When Better Performance Doesn't Mean Better Results

Are We Improving the Work — or Just the Metric?

How do you know when something is getting better?

Usually, we measure it.

Productivity goes up. Costs come down. Waiting times fall. Output increases. Engagement rises.

The graph moves enthusiastically in the correct direction and everybody goes home happy.

Except there's a problem.

What if the number improves, but the thing we actually care about doesn't?

Two pieces of research caught my attention recently.

At first glance, they have very little in common.

One looked at the effect of AI-assisted writing on around 1.5 million online petitions. The other examined the effects of incentives using around 2.3 million hospital records.

Yet put them together and they raise the same uncomfortable question:

Are we improving the outcome — or merely getting better at producing the metric?

When Better Writing Doesn't Mean Better Results

Let's start with AI.

Researchers examining petitions on Change.org found evidence of writing changing as generative AI became widely available.

Petitions became longer. The language became more homogeneous. In various respects, the writing became more polished.

Which sounds like an improvement.

After all, if we introduce a writing tool and the writing becomes better, surely the technology is doing its job?

Perhaps.

But there's another question:

Did the petitions become more successful?

The improvement in the writing did not translate straightforwardly into better petition outcomes. Among returning writers, longer AI-associated petitions actually performed worse.

That doesn't mean AI makes writing worse.

It raises a much more interesting possibility.

Perhaps we've confused improving the output with improving the outcome.

If the objective is:

Produce more polished text

then AI may have succeeded.

But if the objective is:

Persuade people to support the cause

then polished writing is not the outcome.

It's merely one ingredient.

And that distinction matters far beyond petitions.

When the Proxy Becomes the Purpose

We measure things because the things we actually care about are often difficult to measure.

Take education.

What does it mean to educate somebody well?

Knowledge? Curiosity? Critical thinking? Creativity? Confidence? Preparation for adult life?

That's complicated.

Exam results are much easier to count.

So we measure exam results.

Nothing inherently wrong with that. They can tell us something useful.

The problem begins when the measurement stops being an indicator of education and becomes the purpose of education.

The proxy becomes the target.

And organisations do this constantly.

Calls answered becomes customer service. Patients processed becomes healthcare performance. Tasks completed becomes productivity. Clicks become journalism. Followers become influence. Hours worked become commitment. Words produced become writing.

Eventually:

The scoreboard becomes more important than the game.

And Then We Add Incentives

The second piece of research looked at incentives within healthcare using around 2.3 million Irish hospital records.

Again, some of the measured outcomes improved.

Public admissions increased. Hospital stays became shorter.

Look at the dashboard and that might appear to be a success.

More patients moving through the system.

Less time occupying hospital beds.

Greater efficiency.

Except hospitals don't exist primarily to maximise patient throughput.

They exist to make people better.

The research raises the possibility that improvements in targeted measures can produce costs elsewhere, including potentially in the quality of care.

That doesn't mean shorter hospital stays are bad.

If someone can safely return home sooner, that's potentially better for the patient and the health service.

The interesting question is what happens when we attach rewards to particular numbers.

People adapt.

Processes adapt.

Priorities adapt.

Resources move.

The system gradually organises itself around whatever we're rewarding.

Which takes us towards the principle usually associated with Goodhart's Law:

When a measure becomes a target, it can stop being a good measure.

Efficient at What?

Whenever somebody tells me something has become more efficient, there's a question worth asking.

Efficient at what?

A customer service centre reduces average call time.

Excellent.

It's become more efficient at ending telephone calls.

But are customers getting their problems solved?

A company automates large parts of its workforce.

Costs fall.

Excellent.

But what happens to service, innovation, organisational knowledge, resilience and customer relationships?

A school concentrates relentlessly on examination performance.

Results improve. But what happens to curiosity?

A hospital shortens patient stays.

Beds become available more quickly.

But what happens after patients leave?

And increasingly:

An AI system allows someone to produce twice as much work in half the time.

Fantastic.

But...

Did anybody need the extra work?

That's going to become a very important question.

AI Could Make This Problem Much Bigger

Artificial intelligence is going to make many forms of output extraordinarily cheap.

• Reports.

• Emails.

• Presentations.

• Images.

• Code.

• Research summaries.

• Marketing copy.

• Meeting notes.

• Analysis.

• Ideas.

• And, presumably, enormous PowerPoint presentations that nobody particularly wanted in the first place.

AI can help us produce more, faster and often better.

I use it extensively myself.

But if we continue measuring work primarily through output, something rather strange could happen.

Our productivity figures may look wonderful... while actual contribution barely moves.

Imagine somebody previously produced five reports every week.

AI allows them to produce twenty. Fantastic.

Productivity has quadrupled. But did anybody read them?

Did they improve a decision? Did customers benefit? Did the organisation learn anything? Did anything actually change?

Because if the answer is no, we've simply invented a considerably faster way of producing things nobody needed.

Artificial intelligence can increase output without increasing value.

That's not necessarily an AI problem.

It's a measurement problem.

And it's one of the reasons I increasingly think the coming AI revolution will force us to rethink what we mean by productivity in the first place.

Activity. Output. Outcome.

There's a simple distinction I think organisations could make much more often.

Activity is what you do.

Output is what you produce.

Outcome is what changes because you did it.

Imagine a training department.

Activity: Ten leadership workshops delivered.

Output: Two hundred managers completed the programme.

Lovely.

But what was the outcome?

Did those managers behave differently? Did their teams experience better leadership? Did staff retention improve? Did conversations change? Did people receive better feedback?

We frequently stop measuring at the easiest point.

We did the thing.

Tick.

But doing the thing isn't necessarily the point of doing the thing.

Training delivered isn't behaviour changed.

A report written isn't a problem solved.

A strategy published isn't a strategy implemented.

Information produced isn't knowledge used.

And:

Better output isn't necessarily a better outcome.

Successful Failure

There's a phrase I like for what happens when this goes wrong:

Successful failure.

The system hits its targets while failing its purpose.

And that's potentially more dangerous than obvious failure.

The dashboard looks wonderful.

Targets achieved.

Productivity increased.

Costs reduced.

Turnaround times shortened.

Management congratulates itself.

Perhaps somebody gets a bonus.

Meanwhile, the thing the organisation actually exists to accomplish hasn't improved.

It may even have become worse.

Obvious failure creates pressure to change.

Successful failure creates evidence for carrying on.

Don't Throw Away the Dashboard

None of this means measurement is bad.

Quite the opposite.

Without measurement, we're easily fooled by anecdotes, assumptions and somebody declaring their latest initiative a roaring success because everybody seemed enthusiastic during the presentation.

Metrics matter.

Targets matter.

Incentives matter.

But a metric is a representation of reality.

It isn't reality itself.

So every important measure should probably have a companion question:

What is this number supposed to tell us?

Followed by another:

Could this number improve while the thing we actually care about gets worse?

If the answer is yes, you've discovered the limitation of the metric.

Try the Three-Level Test

Pick something you're currently measuring.

It could be at work, in your business or even in your personal life.

Then look at three levels:

ACTIVITY — What are we doing?

OUTPUT — What are we producing?

OUTCOME — What is actually changing?

Then ask:

If activity and output increased but the outcome didn't improve, would we still call this success?

Because if the answer is yes, there's a fair chance you've built a system that rewards busyness rather than progress.

And finally:

What is the system actually rewarding?

If you reward speed, people will optimise speed.

If you reward volume, they'll optimise volume.

If you reward engagement, they'll optimise engagement.

If you reward cost reduction, they'll reduce costs.

And if you reward one number without considering what happens elsewhere, don't be surprised if something elsewhere gets sacrificed.

Not because people are necessarily stupid, lazy or malicious.

They're responding to the environment we've created.

Which means when a system repeatedly produces behaviour we don't want, perhaps we shouldn't only ask:

“What's wrong with the people?”

We should also ask:

“What have we rewarded them for doing?”

Did Anything That Matters Get Better?

These two pieces of research appear to concern completely different worlds.

AI-assisted online petitions.

Hospital incentives.

But underneath them sits the same warning.

A system can produce more.

Move faster.

Cost less.

Look better.

Hit every target.

Make every dashboard glow reassuringly green.

And still become no better at fulfilling its purpose.

That's why our relationship with productivity is going to become increasingly important as artificial intelligence makes output cheaper and easier to create.

Producing more isn't necessarily contributing more.

Efficiency isn't necessarily effectiveness.

And better performance against a metric isn't necessarily a better result.

So perhaps the question we should ask more often isn't:

“How much did we produce?”

Or even:

“Did we hit the target?”

But something considerably simpler:

Did anything that actually matters get better?

Because ultimately, that's the metric that matters.