You Designed for One Model. Production Already Routes Across Five.

The architecture slide is clean. One foundation model. One gateway. One risk register entry. Then you open the production logs and find five different model endpoints answering the same class of request.

That is not a temporary mess. It is the default shape of enterprise AI once pilots multiply, vendors discount, and teams ship under delivery pressure. The problem is not that you ended up with more than one model. The problem is that nobody designed for that outcome, so cost, quality, and failure all land on whoever happens to be on call.

How the one-model fiction starts

Procurement wants a single commercial relationship. Security wants a single data processing agreement. Architecture wants a single pattern they can draw once and reuse. So the programme picks a "primary" model and writes it into the reference architecture.

Then reality shows up:

• A specialist model is cheaper for classification, so it gets wired in "just for that job".
• A second vendor wins a pilot with better latency for customer chat.
• A fine-tune lands for an internal knowledge task that general models keep mangling.
• An agent stack brings its own default model, and nobody changes the setting.
• A regional or sovereign option appears for a regulated workload.

None of those decisions feel like architecture. Each one feels like delivery. Six months later you have five routes and a slide that still shows one.

What "five models" usually means

It is rarely five equal peers. It is usually a messy mix of roles that nobody wrote down.

• A general chat model for broad questions.
• A cheaper model for high-volume summarisation or tagging.
• A retrieval-friendly model for RAG answers.
• A reasoning model for harder tickets that humans used to escalate.
• A leftover endpoint from an old pilot that still gets traffic because a hard-coded key sits in a config file.

If you cannot name which traffic should hit which model, and what happens when that model is slow, expensive, or wrong, you do not have a multi-model estate. You have accidental routing.

What breaks without a routing layer

Teams often treat "more models" as progress. Without an explicit routing layer, it produces four predictable failures.

Cost without ownership. Finance sees a rising AI bill and asks which product owns it. Nobody can split spend by use case, let alone by model decision. The cheapest model for one job sits unused while a premium model answers prompts that did not need it.

Inconsistent answers. The same customer question gets a different tone, refusal style, and factual confidence depending on which path answered. Support notices. Customers notice. Trust erodes even when each model is "good enough" on its own eval set.

Policy drift. One path has tool allowlists and PII redaction. Another path was stood up for a demo and never got the same controls. Your security story is only as strong as the weakest route.

No failure policy. When a primary model times out, some calls error, some fall back silently to a weaker model, and some retry into a more expensive one. There is no shared definition of degrade, deny, or escalate.

What a thin routing layer should own

You do not need a grand "AI mesh" programme. You need a thin, boring layer that makes model choice an intentional control plane. At minimum it should own:

• Intent or job classification. Map the request to a job type before you pick a model. Chat, classify, extract, draft, reason, refuse.
• Explicit routes. For each job type, name the primary model, the fallback, and the conditions that trigger a switch (latency, cost ceiling, confidence, region, risk tier).
• Shared policy hooks. Auth, logging, redaction, tool permissions, and human escalation should sit in front of every route, not inside each team's wrapper.
• Observability that names the decision. Log which route fired, why, what it cost, and whether a human rewrote the answer. If you cannot query that, you cannot improve it.
• An owner. Someone must own the routing table the way someone owns DNS. If "architecture" owns the diagram and "product" owns the prompts, the gap in the middle is where five models hide.

How to get there without rebuilding the platform

Start with the traffic you already have, not a greenfield redesign.

1. Inventory the real routes. Pull thirty days of production calls. Group by endpoint, key, and calling service. You will usually find more paths than anyone admits in the weekly forum.

2. Kill or quarantine orphans. Anything still serving traffic from a dead pilot gets a planned retirement date or a locked allowlist. Accidental endpoints are not innovation.

3. Write a one-page routing table. Job type, primary, fallback, cost ceiling, risk tier, owner. Put it where engineers actually look, not only in the architecture pack.

4. Put one gateway in front of the messy middle. Even a thin proxy that stamps route metadata and enforces shared policy is better than five SDKs with five half-finished wrappers.

5. Tie evals and spend to routes. Offline scores and production rewrite rates should be reported per route, not as one average that hides the bad path.

The architecture question worth asking

If your diagram still shows one model, ask a sharper question: who owns the decision when production needs five?

Multi-model estates are normal. Unowned routing is optional. Design for the routes you already have, name the jobs they serve, and put a thin control plane in front of them before the next vendor discount quietly adds a sixth.