An eval platform and an MCP endpoint shut down on November 30. Your evaluation suite must outlive every vendor

Here is a six-week calendar for a fairly ordinary enterprise AI estate.

Six weeks of vendor lifecycle dates

October 15November 30

1. Oct 15Copilot Studio retires GPT-5.3 Chat and GPT-5.4 Reasoning, extension up to 30 days
2. Oct 31Copilot Studio retires GPT-5.5 Chat, no extension; OpenAI Evals become read-only
3. Nov 12Copilot Studio retires GPT-4.1, GPT-4.1 Mini, GPT-5 Chat and GPT-5 Auto
4. Nov 30OpenAI Evals dashboard and API shut down; Databricks Genie One MCP Beta endpoint ends

None of these is a crisis. Microsoft names a replacement for every retiring Copilot Studio model, GPT Chat Latest, a default that tracks supported chat snapshots. Databricks moved its Genie One MCP Beta endpoint deadline from October 31 to November 30 and points to a generally available replacement in Unity Gateway. OpenAI announced the end of its Evals platform on June 3 and published a migration path to Promptfoo. Vendors are doing what vendors should: giving notice.

What deserves attention is one line in that calendar, because it is different in kind.

The instrument retires too

A model retirement changes the thing you are testing. An endpoint retirement changes how the thing is reached. An evaluation platform retirement removes the instrument you use to know whether either of the other changes broke something.

Consider a team that built its regression suite in OpenAI Evals: datasets of real cases, graders that score answers, thresholds that gate a release. Since October 31, those evals are read-only. After November 30, the dashboard and API are scheduled to stop. If that team also runs a Copilot Studio agent whose GPT-5.5 Chat model was replaced on October 31 with no extension, it had to validate a forced model change in the same weeks it was losing the tool it validates with. OpenAI notes that graders documented for eval workflows are part of the transition.

We have argued since September that evaluation is the control surface of AI systems, and in October that model retirements belong on the change calendar. This calendar adds a refinement: the evaluation suite is itself a dependency, and it is the one that can least afford to be rented.

What “owning the eval suite” means in practice

Owning does not mean building your own evaluation platform. It means the assets survive any platform.

Datasets in a neutral format. Test cases, expected outcomes and the provenance of each case (where it came from, who approved it) live in your repository or data platform, as files or tables, not only inside a vendor dashboard.

Graders as code or as written rubrics. A grader is a definition of “correct”. Keep it as code you can run anywhere, or a rubric precise enough to rebuild. An LLM judge also needs its model and prompt versioned, since a judge on a retiring snapshot will shift scores too.

Thresholds and history. The pass rate that approved the last release, and the results behind it, are the baseline for the next one. Lose the history and every comparison starts from zero.

A second runner. Being able to run the same suite on two tools, even occasionally, is the test that the suite is portable. A migration forced by a deadline is the worst moment to discover what lived only in the old tool.

The broader point

Production AI depends on parts that change on other people’s schedules: model snapshots, harness defaults, connectors, protocol endpoints and evaluation services. That is normal, and the answer is not to avoid vendors. It is to know which parts are interchangeable and which carry the organization’s judgment.

Models are increasingly interchangeable. The definition of a good answer is not. It took subject matter experts to write it, it encodes what the business accepts, and it is what lets you swap everything else safely. It belongs in the short list of things an enterprise should keep after any vendor handover, next to identity and business definitions.

If November 30 is on your calendar for a migration, use it for one more task: check that every evaluation asset you depend on exists somewhere you control, and that you can run it without the vendor who is leaving.