Chainguard's AI attacks your running app to prove a flaw is real. Evaluate the sandbox before the model

A static scanner reads code and guesses. A vulnerability service that runs the code and tries to break it can prove. The second kind is far more useful to a security team drowning in maybes, and it changes what you are buying: no longer a report generator, but an environment where a frontier model executes attacks against your software.

Chainguard Harness, announced on October 14, is that kind of service. Chainguard builds the customer’s application from source, as it would ship to production, boots it in an isolated microVM and lets Claude Mythos 5.1 probe the running instance for exploitable flaws: logic errors, injection, resource exhaustion. The output is a report with severity and impact, steps to confirm each finding and a recommended fix written for the customer’s codebase. It describes issues and how to confirm them, not a working attack.

Harness is in beta, with a waitlist and access initially limited to a small group of organizations. It will also be sold through the Claude Marketplace, where customers can apply part of an existing Anthropic commitment.

Containment is the architecture

When a model runs commands against a live application, the sandbox design matters more than the model’s skill. Chainguard’s announcement is specific about it, and the specifics are what a buyer should check:

Where the attacking model can and cannot reach

Chainguard operations

• A named engineer reviews every fix and report

Agent loop

• Runs outside the sandbox
• Every tool call and command logged

Hunting microVM

• No network egress
• No credentials
• Shell and Chainguard-built packages only

Application under test

• Built from source
• Treated as untrusted data

• Setup and hunting are separate phases. A setup model builds the app and fetches dependencies; then the microVM is suspended. Hunting resumes it with no network egress and no credentials.
• The agent loop runs outside the sandbox, and every tool call and command is logged.
• The code under test is treated as untrusted data, a defense against prompt injection hidden in comments or strings.
• Repository access is read-only, through a GitHub App or an uploaded source archive.

That list is a good template for evaluating any AI system that executes code, from security testers to coding agents. It is the same reasoning behind containment as the real security boundary for agents: assume the model can be persuaded, and decide in advance what a persuaded model can reach.

Reproducible, or it does not count

The second design choice addresses the main complaint about AI security findings: they are noisy. Chainguard replays each exploit in a fresh microVM, and findings that do not reproduce every time are excluded. Each recommended fix must make the exploit fail against the patched code.

This is the opposite of a scanner that lists everything that might be wrong. It trades recall for evidence. The trade is sensible for findings that reach developers, because a reproducible exploit is hard to dismiss, and it fits the pattern of the week: Snyk’s line that the generator cannot be the validator, and GitHub keeping its AI findings advisory.

Chainguard publishes no accuracy, false positive or false negative figures, no pricing, no supported languages and no run times. A clean report tells you what Harness proved, not what it missed.

Questions before you hand over source code

1. Where does the build happen, and in which jurisdiction? Your source code and dependencies leave your environment for Chainguard’s.
2. What happens to the microVM, its logs and the model’s working notes after the run? Custody of artifacts matters as much as custody of code.
3. Who sees raw model output? Only Chainguard staff operate the model; customers cannot instruct it or see its raw output. That limits misuse, and also limits your own audit.
4. What does a fix recommendation commit you to? Nothing is deployed automatically. The fix is a proposal that goes through your review like any other change.
5. How are upstream flaws handled? Chainguard says it coordinates disclosure with open-source maintainers and ships patched builds through its repository; agree in advance how you are told.

AI security testing is moving from reading code to running it. Buy it the way you would buy a penetration test with a very fast tester: check the rules of engagement first.