Your company has dozens of model deployments. Can anyone say which models, whose they are and what they cost?
Microsoft published a reference architecture for a tenant-wide view of Foundry and Azure OpenAI deployments: inventory, usage, cost and quota, with no prompts collected. Ownership tags come first.
Listen to this article · 5 min
AI-generated narration of the full article.
Ask a platform team at a large company three questions: which AI models are deployed across the organization, who owns each one, and what each one cost last month. In many companies the honest answer takes a week and a spreadsheet, because the deployments were created by different teams in different subscriptions and regions, for different projects, over two years.
Microsoft’s Mission Critical team published a reference architecture on October 16 for answering those questions continuously in Azure. It is not a new product and does not replace monitoring inside Microsoft Foundry. It is a way to build a management-group or tenant-wide view on top of services most Azure customers already have. Its design choices are useful well beyond Azure.
Four kinds of data, kept apart
Four planes of a model operations view
Inventory
- Azure Resource Graph across subscriptions
- Deployments API per account
- Model, version, deployment type, owner tags
Current state
Usage
- Azure Monitor metrics and logs
- Requests, tokens, errors, PTU utilization
Time series
Cost
- Cost Management queries or exports
- Joined by resource ID and tags
Billing data, may lag
Quota
- Per subscription, model and region
A capacity signal, not a spend signal
The architecture treats these as different kinds of data rather than one feed. Inventory comes from Azure Resource Graph, which queries across subscriptions, combined with the deployments API for each account: model, version, deployment type, capacity, region and tags such as owner, environment and business unit. Usage comes from Azure Monitor: requests, tokens, errors, latency indicators and utilization of provisioned throughput units. Cost comes from the Cost Management APIs or exports. Quota is tracked separately, because it is a capacity limit, not spending, and Azure OpenAI quota is enforced per subscription, not per tenant, so an executive summary can hide the subscription that is about to hit its ceiling.
The audiences differ too. Platform engineers get Azure Workbooks for drill-down; finance gets Power BI, with a caveat: the Cost Management connector is documented for Enterprise Agreement and direct Microsoft Customer Agreement customers, and others should use exports. Application owners get a small scorecard: their deployments, usage, cost, quota and next required action.
Metadata only, on purpose
The design principle we would underline is the one Microsoft states first: the dashboard does not collect prompts, completions, customer data or other payloads unless a separate privacy and security design requires it. A central team can see every deployment, its owner, its traffic and its cost without gaining access to what anyone asked a model. That keeps the governance view from becoming a new copy of sensitive data, which is the custody question that most observability designs end up facing.
The hygiene view is the governance view
Of the five views proposed (executive summary, inventory, operations, cost allocation and governance hygiene), the last is the one to build first. It lists deployments with missing tags, deployments with no owner, old model versions and regional concentration.
Each of those is a risk with a date attached. Old model versions are next quarter’s forced migrations, the kind recent retirement notices put on the calendar. An ownerless deployment is a cost nobody approves and a model nobody will migrate. Regional concentration is a resilience and data residency question. A hygiene list turns them into tickets with names on them.
Microsoft is blunt about the precondition: “Dashboards cannot create accountability if resources do not carry owner, application, environment, and cost-center metadata.” Require the tags at deployment time, with Azure Policy or the deployment pipeline, before building any reporting on top of them. Otherwise the most accurate number on the dashboard will be the count of things nobody owns.
Beyond Azure
The four planes are not Azure-specific. Every cloud and every model gateway can produce an inventory, usage, cost and quota view, and most enterprises use more than one. The architecture is a good template for the cross-vendor agent and model inventory that governance teams keep asking for. Three limits are worth stating: inventory and token counts say nothing about whether a deployment produces value or correct output; some Azure metrics have export constraints; and parts of Microsoft’s monitoring guidance still describe the classic Foundry portal, so check what applies to your environment before standardizing on it.