By Jayoo Hwang, Vansh Ramani, and Shourya Vir Jain
Ramain is a browser agent. It can automate any UI task by performing mouse and keyboard actions on the same screens a person would use. It logs into portals, fills forms, and moves records between systems without requiring a platform-specific integration. Today we are publishing how well it does that.
- 4
- Public benchmark suites
- 89.4
- WorkArena L3 score
- 84.2
- WebArena Hard score
Four public benchmarks, strict pass or fail
We evaluated Ramain on four public benchmarks that test reliability across very different tasks and websites. In every suite, the agent receives a starting URL and a task instruction, then must finish the entire task autonomously without guidance.
Scoring is strict pass or fail. A task counts as successful only if every requirement is completed perfectly.
- WorkArena L3: complex multi-step enterprise tasks inside a real ServiceNow instance.
- WebArena-Verified Hard: 258 rigorously checked tasks in realistic web apps, including storefronts, project trackers, and forums.
- WebVoyager: everyday tasks on real, live websites.
- Online-Mind2Web: 300 live-web tasks across 136 sites, from quick lookups to long multi-step jobs.
The results
Ramain achieves state-of-the-art performance across all four suites: 99.3 on Online-Mind2Web, 99.2 on WebVoyager, 89.4 on WorkArena L3, and 84.2 on WebArena-Verified Hard.
It also outperforms Fable 5 when Fable is driven from a basic computer-use harness that uses screenshots with mouse and keyboard actions.
The model is not the whole product
We use efficient context management and an optimized browser representation to improve reliability. These techniques squeeze more value from every token and push performance beyond what frontier models alone can buy.
In practice, these results mean Ramain can navigate the vast majority of interfaces and reliably carry out multi-step tasks unaided. Accuracy is only the first production constraint, though. Browser agents consume a significant number of tokens, so cost-efficiency decides which workflows are practical to automate.
Ramain achieves SOTA results across four major browser agent benchmarks.
