The malicious LiteLLM release was signed. The AI gateway compromise shows what a signature cannot prove

The poisoned LiteLLM images were signed. A team that verified every container signature before deploying on October 21 would have accepted them, because the release workflow that built the malware also signed it.

That is the detail worth sitting with in the compromise of LiteLLM 1.107.0, the open source gateway many companies put between their applications and every model provider they use. On October 21, from 11:39 to 13:23 UTC, the PyPI package carried credential-stealing code; from 11:46 to 14:20 UTC, the official v1.107.0, latest, main-latest and main-stable Docker Hub tags pointed to compromised images. Microsoft published a technical investigation on October 23. LiteLLM rotated its signing key on October 24. LangChain blocked the version in its integration on October 25.

How a gateway’s own logs became the way in

According to the LiteLLM maintainers, the attacker used developers’ GitHub CLI credentials, one of them belonging to an organization owner, which allowed branch protections to be disabled. They ran the project’s release workflow against a lookalike repository, Berri-AI/litellm instead of BerriAI/litellm, and that workflow built and signed the artifacts. The tokens, the maintainers say, were reportedly taken from the logs of an exposed internal LiteLLM deployment, where a coding tool’s responses had carried them. How the attacker reached that deployment is still unconfirmed.

1. Tokens exposed in the logs of an internal gateway
2. Developer credentials reused, branch protection disabled
3. Release workflow run against a lookalike repository
4. Artifacts built and signed by that workflow
5. Published to PyPI, Docker Hub and GHCR tags
6. Code runs at Python startup and collects secrets

The signature was validThe trusted pipeline did the work

Read the chain from the end and every control looks fine: the package came from the official index, the image from the official registry, the signature from the official key. The failure sat upstream, in who could run the pipeline. A signature proves which key signed an artifact. It does not prove that the right people, from the right source, asked for it.

Uninstalling was never going to be enough

Microsoft’s analysis explains why this release was worse than a typical malicious package. It placed files in the Python library directory that run whenever the interpreter starts, so the code executed without anyone importing LiteLLM. A CI runner that merely had the package installed, a developer laptop, a test container: all of them ran it. Deleting the LiteLLM folder can leave those startup files in place.

The collector went after what a gateway host tends to hold: cloud credentials for AWS, Azure and Google Cloud, Kubernetes and Docker configuration, SSH, GPG and CI/CD keys, Vault material, environment variables, configuration for developer and AI tools including MCP integrations, and records from the LiteLLM proxy database, where provider keys live. Microsoft saw the staging and outbound connections on endpoints, but is careful about the limits: startup alone “did not prove collection or upload completed,” a second stage was not confirmed, and Microsoft has not independently determined how the attacker first got in.

LiteLLM’s own guidance is blunter: if 1.107.0 ran on a host with outbound internet access, assume the data left. Rotate the instance role, every provider key stored in the proxy, the master and virtual keys, and the database password. The private enterprise release was not affected, and earlier public versions are safe; the maintainers recommend pinning 1.106.2 or another explicit pre-incident version.

The follow-up releases change your configuration, not your exposure

Two later releases deserve attention because they touch deployment pipelines:

• 1.106.3 (October 24) signs every Docker image and Helm chart with a new cosign key. Verification against the old key now fails, so any pinned copy of the public key in scripts, CI checks or admission policies (Kyverno, Sigstore policy-controller, Connaisseur) has to be replaced. The release also removes the eMudhra root CA from the trust bundles. The release notes do not mention the incident, and a new key does not make anything already deployed trustworthy.
• langchain-litellm 0.11.2 (October 25) sets a LiteLLM floor of 1.106.2 and excludes 1.107.0, which matters because many teams never chose a LiteLLM version: their framework did.

Neither replaces recovery. Microsoft’s line is the one to remember: “Removing the package or a release tag is not sufficient recovery.”

What the incident asks of teams that run a gateway

An AI gateway is built to concentrate things: keys for every provider, routing rules, spend limits, logs of prompts and responses. We have argued that it is becoming a policy enforcement point, and that is exactly why its host deserves the protection of identity infrastructure. Four practices follow from this incident:

1. Pin by digest and lock by hash. Mutable tags such as latest and main-stable were among those repointed. A deployment that references an image digest and a lock file with hashes would not have picked up the new artifact silently.
2. Treat gateway logs as secret stores. If responses can carry tokens, logs inherit their sensitivity: restrict access, redact credentials and keep retention short.
3. Limit egress from build, developer and gateway hosts. The collector needed an outbound connection. Most of these machines need a short allow list, not the internet.
4. Inventory where the package actually runs. Pip and uv caches, internal mirrors, base images and framework dependencies are where a pinned version gets undone.

The uncomfortable lesson is not that open source gateways are unsafe. It is that the controls most teams rely on (official source, valid signature, known publisher) all passed, and the attack still worked.