03 / Precisely WrongAutomation5 min read

AI does not argue with a bad brief.

A machine will not ask whether the record is complete, or notice that somebody quietly changed what a category means. It commits, at speed and with confidence.

Back to The Thinking

Train it on a flattering account of your business and it learns the flattery. The output is fast, consistent and wrong in the same direction every time.

Speed is not the risk. Speed applied to an edited account is the risk.

The failure is quiet, because every individual output looks reasonable. It is the direction that is off, and direction only shows up over months.

01Exposure

The data it learns from

Your systems record what people reported, not what actually happened. If a problem here typically takes six weeks to reach the top, the data says the problem started six weeks late. Train a model on that and it learns the six-week delay as though it were true, then plans around it.

02Exposure

Your disagreements, at scale

If three executives read the strategy three ways, automated decisions will express all three. People reconcile that in a corridor. Systems do not.

03Exposure

Working out what went wrong

When an automated decision fails, the board asks what the business believed. Accident investigation is built to answer that. Most governance frameworks are not.

The question to settle first.

Before you scale a decision with a machine, establish that the account it will act on describes the business you actually run.

That is an investigation, and it belongs before the approval rather than after the incident.

A fast answer built on an edited account is still the wrong answer.

Next

If this describes your week.

Bring the problem as you see it. We will use 30 focused minutes to work out what deserves a closer look.