Reflection's Beam weights are out. How you serve them now matters as much as the model
Beam shipped on October 16 in three precisions, with day-zero recipes from AWS and Google Cloud. For coding agents, the serving architecture now decides cost as much as the model does.
Listen to this article · 5 min
AI-generated narration of the full article.
When Reflection announced Beam on October 5, the weights were still in final red-teaming. Our first analysis was about what an enterprise inherits when it runs a 501-billion-parameter model itself. On October 16 the weights arrived, and the question has moved: not whether you can run Beam, but which way of serving it you are paying for.
What shipped
Beam is on Hugging Face under Apache 2.0 in three precisions, BF16, FP8 and NVFP4, plus a small drafter model for speculative decoding, with a model card and a safety technical report. The architecture is as announced: a sparse mixture of experts with 501 billion total parameters and about 23 billion active per token. AWS’s and Google’s posts give the context window as about one million tokens.
Reflection also released Mirror, an open-source command-line harness for running Beam as a coding agent. It lets a developer inspect what the agent is doing, approve tool calls, steer it mid-task and run parallel agents in separate Git worktrees, and it is extended through project instructions, skills, MCP connections and Python. More than twenty platforms announced support on day one, among them AWS, Google Cloud, Microsoft Foundry, Databricks, Oracle and Cloudflare, plus Dell and HPE for on-premises deployment. Self-distillation recipes are promised “in the coming days.”
Three precisions are three quality decisions
The precision you choose is not only a hardware decision. On AWS, the FP8 weights run on H100 or H200 instances (ml.p5.48xlarge, ml.p5e.48xlarge, ml.p5en.48xlarge), NVFP4 runs on B200 (ml.p6-b200.48xlarge), and BF16 needs a multi-node setup. As rough arithmetic, 501 billion parameters take about a terabyte at 16 bits, about half that at 8 bits and about a quarter at 4 bits, before the KV cache that long agent sessions fill.
AWS’s own guidance is to recheck output quality after changing precision. That is the right instinct: a 4-bit model that fits on fewer GPUs is a different model for evaluation purposes. Run your coding evaluation set on the precision you will actually serve, not on the one the benchmark used.
| Who operates it | What you choose | What you still prove | |
|---|---|---|---|
| Reflection API or a hosted partner | The provider | Plan and region | Quality on your tasks, data terms |
| SageMaker JumpStart | AWS manages the endpoint | Instance type and precision | Quality at that precision, capacity quota |
| GKE with llm-d or Dynamo | Your platform team | Routing, caching, disaggregation | Throughput and latency on your traffic |
| Your own data center (Dell, HPE) | You, end to end | Everything | Everything, plus patching |
Why the serving layer now matters as much as the model
Google’s post is the most instructive document of the launch, read carefully. On GKE, Google reports up to 5x higher throughput and 8x lower latency using NVIDIA Dynamo with SGLang on 32 GB300 GPUs, and up to 2x throughput and 8x lower latency using llm-d with vLLM on eight RTX PRO 6000 GPUs. Both figures compare against a plain deployment on the same hardware with stateless round-robin routing, on a multi-turn agentic coding workload. The 5x figure is marked as based on customer benchmarking, and the two configurations are not comparable with each other.
The gains come from three techniques: routing each multi-turn session back to the replica that already holds its KV cache, separating the prompt-processing phase from token generation so bursts of long prompts do not stall output, and spilling KV cache to host memory and local SSD. None of them changes the model. All of them change what a completed coding task costs.
This matters because coding agents are the worst case for naive serving. A session sends a long, growing context on every turn. Without cache-aware routing, each turn can land on a replica that has to process the whole history again. The same weights, on the same GPUs, can therefore cost several times more per task depending on a routing decision your platform team makes, which is the point we made about pricing inference per completed task.
Two cautions. Google’s figures are “up to” figures against a weak baseline, and gains grow with deployment size, so a small pilot will not see them. And Reflection’s claim of 3x to 4x fewer reasoning tokens than comparable open models is Reflection’s own; test it on your tasks before it reaches a business case.
What to do with the release
For teams that evaluated Beam from the announcement, the work is now concrete. Pick the path from the table by who should operate the model. Choose a precision and evaluate at that precision. If you self-host, treat KV-cache-aware routing as a requirement for agent traffic, not an optimization. And look at Mirror’s tool-call approvals before your developers point it at a repository with production credentials: an open harness is useful precisely because you can see and set where it stops to ask.