Your AI Pilot Worked. Your Operating Model Did Not Notice.
The pilot deck still has applause. Accuracy cleared the bar. The sponsor asked when it would hit next year's run-rate. Then the operating model did nothing. Same roles, same queue, same cycle-time target, same polite handoff from "the AI team" to a unit that never agreed to own a model. Two quarters later the demo is a graveyard slide.
AI is not harder than it looks. It is unfinished. A pilot proves a model can help in a staged slice of work. An operating model decides whether that help is allowed to become Tuesday morning's job.
This piece is about that silence: ownership, process redesign, KPIs, exception handling, and change capacity. A gate you can attach before the next wow moment gets funded as if it were production.
When an Australian bank, insurer, telco, or agency says the pilot worked, they usually mean this mix.
That is a product experiment, not an operating result. The organisation still routes work the old way because nobody rewrote how work is supposed to move. If your scorecard still counts "number of pilots completed", you will keep collecting demos. Accuracy in a lab is a compliment to the sample. It is not a compliment to the floor.
Treat these as a coupled set. Fixing one and ignoring the rest is how you get a clever assistant sitting next to an unchanged queue.
Someone has to own the outcome in the line, not the lab. An AI squad can own the model, the evals, and the platform path. They cannot own a claims decision, a credit exception, or a customer remediation. If the business unit will not put a named owner on the roster, with a deputy, a budget line, and the right to stop the tool, you do not have a product. You have a science fair.
A Sydney insurer can ship a sharp triage model and still watch every "AI suggested" file bounce to a team whose position description says they complete the file from scratch. The model is a hint. The operating model never made it a step.
Dropping a model onto an old process is decoration. If the swimlane still assumes a human reads every document, then a 40 percent assist does not free 40 percent of the team. It adds a screen. You have to redraw the work: what the model does first, what a person must still do, where the file goes next, and what "done" means when the model is confident versus when it is not.
A Melbourne telco's service summariser can look magic in a demo and still add keystrokes on the floor because the wrap-up form, the CRM fields, and the quality sample were never redesigned. Staff use it as a novelty, then drop it when handle time suffers.
People do the work their scorecard pays for. If team leaders are still measured on files per hour, they will skip the model when it slows them down, even if it cuts rework next week. If quality is sampled only on human-authored notes, staff will type the old narrative and ignore the suggestion. Change the measure or the behaviour will not move.
A government payments team that still worships average handle time will treat an assist that adds two minutes of checking as a failure, even when it prevents a wrong payment. The KPI is the operating model in disguise.
Demos live on the happy path. Production lives on the eight percent that is messy: missing documents, conflicting identity, a customer who is also a staff member, a case that spans two products. If there is no explicit escalate path (who, how fast, with what payload, with what audit), staff invent one. That invented path is usually ignore the model, or ping the person who built the demo.
A regional bank's lending copilot can nail vanilla owner-occupier files and still dump edge cases into a shared mailbox nobody owns after 4pm. Exceptions are the operating model. If they are undefined, the pilot was never going to scale.
Even a good design fails if the floor cannot absorb it. Count training hours, super-user coverage, a fortnight where volumes are deliberately lower, and a freeze on other tool roll-outs. Stacking an AI assist on top of a core-system cutover and a new complaints process is how you get quiet non-adoption. Pretending the AI change is "just a UI" is how you burn goodwill you will need for the next release.
These are not technology failures. They are operating-model silences with a successful demo in front of them. If you cannot name the receiving team and the payload they need, you scaled a screenshot, not a process.
Do not fund "scale" on accuracy alone. Before the next gate, the packet should answer these in writing, with names. If a row is blank, the gate is a no. Staging is cheap. An unowned roll-out is not.
Make the operating-model page the first slide, not the appendix. Sponsors who only want the accuracy chart have not noticed the work yet. Believe them. Accuracy can wait in staging until the job, the measure, and the exception path exist on paper.
A working pilot is a compliment to the model and the sample. It is not a compliment to the organisation. The organisation notices when roles, process, KPIs, exceptions, and change load move. Until then you are applauding a demo in a building that still runs last year's job.
Before the next steering pack, ask one blunt question: if we switched this off on Monday, which operating metric would move, and who would be accountable for moving it back? If nobody can answer, you do not have a production candidate. You have a successful experiment that the operating model has not met.
