Postman's agent got worse once it could see about 40 tools. Its fix treats tool choice as retrieval

Give a capable model more tools and it should be able to do more. Postman found the opposite past a certain point: “In Postman’s testing, tool-selection errors increased once the visible toolset exceeded approximately 40 tools.”

The detail comes from an October 9 post on the AWS Machine Learning Blog by Srinivas Kini of Postman’s AI team and Shubham Gupta of AWS, describing how Agent Mode runs on Amazon Bedrock inside a developer platform with eleven years of product behind it. The errors had a recognizable shape: calls to tools that did not exist, wrong arguments, and tools that looked right but were wrong in context.

Every tool is part of the prompt

A tool definition is text the model reads on every turn: a name, a description, a parameter schema. Forty of them compete for the model’s attention, many with overlapping descriptions. The model is not failing to reason; it is choosing from a menu that has grown too long to read carefully.

Postman’s response was to stop showing the whole menu. A root agent queries a vector database of tool embeddings and, in the post’s words, “narrows more than 170 tools to approximately 15 relevant to the request.” It then hands those tools to a sub-agent whose context is isolated to that task, so the model sees only what the task needs.

1. Developer request
2. Root agent searches tool embeddings
3. About 15 of 170+ tools selected
4. Isolated sub-agent works the task
5. User approval before any state change

• Tool surface narrowed per task: Root agent searches tool embeddings · About 15 of 170+ tools selected · Isolated sub-agent works the task

RequestSelection and context isolation

Fewer, better tools

The second move was less glamorous. For the API Catalog, Postman had built several narrow tools, each answering one kind of question. It merged them into a single query tool that lets the agent write queries with joins and filters against the underlying ClickHouse schemas. The team, in effect, traded many per-question tools for one well-modeled data interface.

That trade generalizes. Teams wrapping internal systems for agents often produce one tool per API endpoint, because that is how the systems are organized. An agent is usually better served by fewer tools that map to the questions people ask, with the data model doing the work. We saw the same economy in Google’s developer knowledge API: the cheapest context is the one the agent never has to load.

Controls that travel with the request

The architecture carries its controls with it. “Agent Mode requires user approval before actions that modify application state,” and tools are scoped to each task. Amazon Bedrock Guardrails can “redact personally identifiable information before it reaches the underlying large language model,” a setting enterprise admins enable in Agent Mode.

Two data settings are worth noting for anyone building on Bedrock. Postman set data retention to none for the supported models, while warning that “Availability and behavior are model-dependent,” so each model has to be checked against current documentation. And Agent Mode uses cross-Region inference, where the choice of profile is a residency decision: geographic profiles keep routing within a geography such as the United States or the European Union, while global profiles add burst throughput but have no geographic boundary and suit only workloads without residency requirements.

What to copy, and what not to

The 40-tool figure is Postman’s observation on its workload and models, not a universal constant. The post gives no latency, error-rate or cost numbers, and the “40 million developers” in the title describes Postman’s audience, not confirmed Agent Mode users. Treat it as an architecture teardown, not a benchmark.

What transfers is the method:

• Count the tools each agent can see, per task, and track it like any other configuration. Tool lists grow quietly as teams add integrations.
• Measure tool-selection errors as the list grows. Your threshold may be 25 or 60; you will only know if you measure wrong-tool and nonexistent-tool calls in your evaluations.
• Select tools before the model reasons, by retrieval or by task routing, and isolate the context of each sub-task.
• Keep the write gate where the tool is, not in the prompt, so the approval holds whichever tools retrieval returns.

This also simplifies governance. A tool the agent cannot see in a given task is a tool it cannot misuse in that task, a narrower and more testable surface than the full catalog MCP makes so easy to connect.