AI malware scanners miss what they cannot read. Databricks closed the gap with evidence, not a bigger model

A common assumption in AI-assisted security is that a smarter model will catch more. Databricks Threat Research tested a case where that assumption fails, and the reason is mundane: the model cannot reason about what it was never shown.

The case is malicious npm packages that bundle a compiled binary. A language model reviewing a package reads its JavaScript, its manifest and its metadata. A compiled executable is, to that model, an opaque blob. If the payload lives there, the code the model can read may look perfectly ordinary.

A small share with a large miss rate

In a corpus of 11,413 malicious npm package-versions confirmed against OSV malware advisories, only 3.8% (432 versions) bundled a binary. The rest hid their payloads in source or fetched them at install time. Small share, outsized difficulty: across the OSV-confirmed set, packages carrying binaries failed to reach a decisive malicious verdict 14.4% of the time, against 6.2% for source-only packages.

A lab experiment made the gap sharper. The researchers ran 10 binary payloads across nine models of the cheaper tier, including Claude Haiku 4.5, Gemini 3.5 Flash, GPT-5.4-nano and Llama 3.1 8B. Without help, half of the 70 tests, 35, failed to reach the escalation threshold, and 20 of those were judged benign.

The fix is an evidence pipeline

Databricks did not answer with a larger model. It wrote capability profiles: declarative playbooks, one per malicious technique such as credential harvesting or persistence. Each profile is a set of weighted conditions over static evidence produced by a binary-analysis module, the existing scanner and metadata checks. The profiles score that evidence against their own thresholds, independently of the model. When one fires, the engine adds a structured summary of about 1,800 tokens to the prompt before the model reasons. The model can still overrule it.

With profiles, 32 of the 35 lab failures became detections, and the escalation failure rate fell from 50% to 4%. The effect was largest on the smallest models: GPT-5.4-nano went from 0 of 10 payloads detected to 7, Llama 3.1 8B from 4 to 10.

Read that result as an architecture lesson rather than a model ranking. The domain knowledge sits in reviewed, versioned playbooks a security team can audit, the deterministic analysis produces the evidence, and the model weighs it. That is the arrangement we keep finding in AI work that holds up, where proof has to come from outside the model.

What production looked like

On a separate validation set, 425 OSV-confirmed malicious package-versions with native binaries and 2,449 benign versions from 287 popular packages (over a million monthly downloads each, none in OSV), the scanner escalating at “highly suspicious” or above produced 345 true positives, 80 misses, 104 false positives and 2,345 true negatives. That is 0.77 precision, 0.81 recall and 0.79 F1, with false positives on 4.25% of benign versions. Of the misses, 60 were downgraded to “suspicious” and 20 cleared as benign.

Those numbers describe a triage tool, not a gate. One in five malicious versions in that set still fell below the escalation bar, and roughly one escalation in four was benign. Databricks also notes that OSV labels whole packages, so some cleared versions may carry their payload in JavaScript or metadata rather than the binary, and that hosted models change continuously, which makes the comparison between conditions more reliable than any absolute score.

What security teams should take from it

Measure what your AI reviewer can see. Before comparing models, list which artifact types reach the model as evidence and which arrive as opaque bytes: binaries, minified bundles, install scripts that download later, images. Each blind type is a class of attack the model will wave through with confidence.

Put domain knowledge where people can review it. Profiles are versioned rules, not a prompt someone tuned once. When a new technique appears, the security team adds a profile, and every model benefits.

Plan the escalation queue. At a 4.25% false positive rate on popular packages, a large dependency estate generates real review work. Size it before turning the scanner on, the same verification budget we described for AI security findings in general.

Keep the established controls. None of this replaces lockfiles, internal registries, allowlists, provenance checks or blocking install scripts by default. An AI triage layer decides what deserves a human look sooner; it is not the reason a malicious package cannot reach a build.