Epsilon now writes UI tests as plain sentences. It kept Playwright, and that is the lesson

A typical UI test at Epsilon, the marketing company owned by Publicis Groupe, used to be 80 to 150 lines of Playwright code. The same test can now be written as 18 sentences describing what a user does: open the campaign page, filter by region, check that the totals match. Epsilon’s internal figures put authoring at 5 to 10 minutes instead of 30 to 90.

That is the headline in the implementation AWS published on October 29. The more instructive part is what Epsilon did not change.

The service, not the tool

Epsilon’s Agentic Platform team, about 50 engineers, built intent-driven testing as a shared multi-tenant service, so product teams do not each run browser infrastructure. Underneath are Amazon Nova Act, which turns natural-language steps into browser actions, and the AgentCore Browser Tool, which provides managed headless browser sessions with live view, session recordings stored in S3 and CloudTrail audit.

The service is exposed internally as an agent with two skills: one records a test, one executes it. Each tenant runs under a scoped role assumed across accounts, so one team’s tests cannot touch another team’s environments, and every session is attributable.

Explore once, then replay

The design choice that makes this usable in CI is the split between two states.

In draft, the agent explores: it reads the sentences, works out how to perform them on the current page and retries when it fails. Epsilon reports that a test typically needs four or five prompt iterations to pass this phase. Exploration is where the model’s flexibility pays off, and where it is slow and costs model calls.

In active, the service replays the saved trajectory of the successful run instead of asking the model again. Epsilon reports a 100 percent pass rate on trajectory replay across repeated runs. Replay is faster, cheaper and, above all, repeatable, which is what a test needs to be.

That is the same idea we described in when agent work should become ordinary software: let the model solve the problem once, then keep the solution and take the model out of the loop. A test that is reinterpreted by a model on every run is not a regression test; it is a new experiment each time.

Why Playwright stays

Epsilon keeps its Playwright suite as the regression backbone. Natural-language tests are additive: they make it cheap to cover journeys that never got a coded test, and to express intent that product owners can read. They do not replace deterministic regression, accessibility, performance or security testing.

There are practical reasons beyond principle. The Nova Act signing endpoint sits in US East regardless of where the browser session runs, which matters for teams with regional constraints. Browser viewport defaults have to be accounted for in scrolling steps. And all the figures above are Epsilon’s internal benchmarks, not independent measurements of defect detection.

What transfers to other teams

Treat computer-use testing as platform infrastructure. Browser sessions, credentials for test environments, recordings and audit are the same for every team. Building them once is where Epsilon’s leverage came from.

Make the trajectory the artifact. Review and version the recorded trajectory like code. When the UI changes and replay fails, the test goes back to draft and a person approves the new trajectory.

Keep the deterministic suite. Intent tests widen coverage. Coded tests keep the floor. As coding agents write more of the application, both get more important, because the evidence that software works has to come from outside the model that wrote it.

The sentence-based test is the visible novelty. The replayed trajectory is the reason it can run in a pipeline every night.