Where AI automation actually pays off in a business

The gap between AI adoption and AI results

In July 2025 MIT's Project NANDA published a report called The GenAI Divide: State of AI in Business 2025. It drew on 52 executive interviews, a survey of 153 leaders and an analysis of 300 public deployments. One number from it travelled much further than the report itself: 95% of the generative AI pilots studied produced no measurable impact on profit and loss. Enterprise spending over the same period was estimated at 30 to 40 billion dollars.

That number gets quoted as proof the technology is overhyped. It is better read as a description of how the money was spent. The report found the divide was not between companies with good models and companies with bad ones. Adoption was high almost everywhere. What separated the 5% was what they pointed the technology at, and how they measured it afterwards.

Impressive demos and useful systems are different things

The pilots that failed tended to target the work that looks most impressive in a demo: strategy documents, campaign concepts, customer-facing chat. That work is genuinely hard to automate, because the output is judged on quality rather than correctness, and because there is rarely a clean before-and-after to compare.

The pilots that worked went after back-office friction instead. Inbox triage. Invoice extraction. Support ticket routing. Document classification. None of it is interesting to talk about at a conference. All of it shares one property that makes automation measurable: a per-unit cost you already know. If a person currently spends four minutes routing a ticket and you process nine thousand tickets a month, you have a baseline, and after the change you have an answer.

Buy the capability, build the workflow

One of the report's more uncomfortable findings is that teams which bought their AI automation capability externally succeeded more often than teams which built it in-house. That runs against the instinct to treat AI as a strategic asset that has to be owned end to end.

The reason is fairly ordinary. Building the model layer consumes the budget and the attention that the workflow layer needs. The model was almost never the bottleneck. The bottleneck was the plumbing: where the data comes from, what happens when the output is wrong, who reviews the edge cases, and what the system does at 2am when nobody is watching. Teams that bought the capability had budget left to solve those problems. Teams that built it had a working model and no route into daily operations.

Measure workflow change, not licence adoption

A large share of failed pilots reported success on the wrong axis. Seats activated. Queries run. Weekly active users. These measure whether people tried the tool, not whether the business changed.

The metric that matters is the one attached to the workflow itself: hours returned, error rate, cycle time, cost per item processed. If you cannot state that number as it stands today, you are not ready to start, because in six months you will not be able to prove anything either way. Taking the baseline first is unglamorous and it is the single cheapest thing you can do to protect the investment.

A short test before you commit

Before starting an automation project, three questions filter out most of the work that will not pay off. What is the per-unit cost of this task right now, in minutes or in money? Who is the named person responsible for this workflow continuing to run in six months, not the team in the abstract but the individual? And what happens when the system is wrong, since it will be wrong sometimes, and a process with no answer to that question is not ready to be automated.

If a workflow survives all three, it is usually worth doing. If it fails any of them, the problem is not which model you pick.

What this means for a smaller team

Most of the MIT research looked at large enterprises, but the lesson travels down. A team of fifteen has the same choice about where to point the effort, and less room to waste it. The advantage of being small is that the boring workflows are easy to see. You already know which handoff gets dropped, which inbox backs up on Mondays, and which report somebody rebuilds by hand every month.

Those are the places to start. Not because they are ambitious, but because they are measurable, and because a system that quietly removes forty hours a month of copying data between tools will still be running next year.