A few months ago I was chatting in a small circle at an industry event with other leaders and a senior executive described how well his team was using AI. Big gains, he said. Real results.
Out of curiosity and potential for us all to learn more – someone asked him to name one. One thing they had done, what tool used, what metric…
He couldn’t.
No number, no metric, no comparison to how the work went before. He spoke around the inquiry, giving vague answers but with real conviction! So bravo for confidence in a state of uncertainty. The group simply nodded through any awkwardness and the conversation moved on. It was not the moment to press him, so no one did. I have thought about it several times since, mostly about the people who work for him. He had just made a claim upward that someone below him would now have to prove.
To be abundantly clear, I do not think he was lying. He believed what he said. He simply had no way to know whether it was true, because too many organizations have been just rushing to deploy AI to check a box. A box that the organizational leaders are pressing on to exhibit to shareholders and stakeholders that they are progressive, they are technically advanced, they are part of this AI revolution.
But the part most people step around – many AI tools and adoptions are working on items that either were not properly measured beforehand or are not being accurately measured now. Without first capturing a true baseline or understanding your current and verified reality, claiming any genuine progress is impossible.
Most of what AI absorbs is tedious work
Most of what AI absorbs is tedious work, and tedious work is exactly what no one tracks. Data deduplication is a good example. Some days it takes an afternoon. Other times it runs for weeks, depending on how bad the records are and how many exceptions hide inside them. No one ever timed it, because it was grunt work, and because the hours swung so widely that any figure would have been a guess. So when leadership asks what AI saved, the honest answer is an estimate dressed up as a percentage. And everyone in the chain has a quiet reason to round it up.
That is the first gap. There is a second one beneath it that matters more.
Even the teams that do measure tend to measure the easy thing. They measure time. Calls deflected, tickets closed, and hours returned. Time gets counted because time is countable. It became the default metric because it is convenient, not because it is the one that matters most.
The metric that matters is quality, and almost no one tracks it. Partly because it is harder to pin down. Partly because tracking it means admitting what you were getting wrong before.
I will use myself as an example here. The value I get from AI is not speed. It is a second set of eyes. What did I miss working alone? What did I fail to catch because I was buried inside my own team, where we all carry the same assumptions and confirm them for each other? That is the gain I would put in front of a board, and it holds up better than any claim about hours saved, because it is about the errors that never left the building.
Notice why that gain stays hidden. To claim it, I have to admit what I would have gotten wrong on my own. That costs something, and it is why the most useful thing AI does for many of us is the thing we are least likely to say out loud.
The strange part is that we already know how to measure quality. We just do it in one place and ignore it everywhere else. Engineering has tracked quality for years through bug reports, failed tests, and defects that reach production. The rest of the organization tracks time, because time is all it was ever taught to count. The method exists in one corner of the building and is missing from the others.
The tools are not the obstacle. The willingness is.
None of this means AI is not delivering. Sometimes it delivers a great deal. The point is that most organizations have no honest way of knowing, and they are reporting confidence they have not earned, up a chain where every level has a reason not to look too closely. That is not a technology problem. It is a measurement problem, and underneath that, an honesty problem. It will not correct itself, because the incentives run the other way.
Which brings me back to that circle, and to you.
If someone asked you today to name your gains from AI, in front of the people whose opinion you care about, could you – with confidence in the accuracy of your answer? Not the hours you or your team think you saved. The mistakes it caught that you would have shipped. The quality you can point to and defend.
If you can, – that is genuinely impressive and please know you are ahead of most.
If you can’t, now you know what tomorrow’s work should be.
