Using Goodhart’s Law to Ensure Your KPIs Are Not Misused
Goodhart's Law

Written by: Jon Stafford

I have over 13 years experience in creating effective BI that turns data into insight and value. I’ve worked in many industries, combining my extensive business knowledge and background in solution development to deliver successful BI strategies, solutions, training and coaching.

Back in the 1970s, economist Charles Goodhart popularised a saying which has become known as

Goodhart’s Law:

“When a measure becomes a target, it ceases to be a good measure.” *

The law warns against the natural human tendency to ‘game the system’. A popular example uses performance targets in a nail factory:

  • If you measure people on the number of nails they make, then they will make lots of tiny nails to achieve the target.
  • If, instead, you measure people on the weight of nails they make, then they will make a few large, heavy nails to achieve the target.

This is obviously not meant to be taken literally, but it does illustrate the point well. Sometimes your targets and KPIs don’t help you achieve your objectives and goals – human nature inevitably conspires against you.

So What Can You Do to Avoid This?

For a start, just being aware of Goodhart’s Law puts you at an advantage. If your KPIs aren’t achieving what they should, you can look at whether people are interpreting them differently to how you expected, and plug any holes.

You can also try to formulate better targets in the first place – ones that aren’t subject to gaming. This is easier said than done, of course, but there are a few rules of thumb you can follow:

  • Divide and conquer – develop a range of KPIs that work together. It’s easy to work out how to stretch the definition of one or two targets to game the system, it’s a lot harder when there are several of them. So try to develop your KPIs strategically, foreseeing how they’ll be used, so that they cover all scenarios. A balanced scorecard approach works well here.
  • Be clear and complete – leave no wiggle room. The clearer and more complete your KPIs are, the less people will be able to misinterpret them to their advantage in ways you didn’t foresee. So make sure you haven’t left any gaps that could be exploited.
  • Measure and control. Make sure that you have the appropriate governance and controls in place to monitor your KPIs and make sure that they’re working as you intended.
  • Finally, you can look to the experts for help. Let organisations who specialise in BI help you develop effective KPIs, leaving you to concentrate on what you do best.

Better Performance Targets in the Nail Factory

So how can the nail factory we saw above use these rules of thumb to improve its KPIs?

  • By targeting people on both the number and the weight of nails they produce (divide and conquer), they can stop the balance shifting too far in either direction.
  • By specifying the weight and size of the nails that should be produced (clear and complete), they can ensure there is no room for misinterpretation. As a bonus, it’s also now sufficient to go back to a single KPI (either number or weight of nails produced).
  • Finally, by installing quality controls at key points in the manufacturing process, and employing someone to be responsible for measuring output quality and ensuring KPIs are met (measure and control), they can apply governance as a further defence against targets being misused.

By taking a similar approach – being aware of Goodhart’s Law, and using the rules of thumb above – you too can move towards more effective KPIs that help you achieve your objectives and goals.

If you would like more insights like this, check out our blog page here for more articles!


* This is Marilyn Stratham’s popular phrasing of Goodhart’s Law (Marilyn Strathern (1997). ‘Improving ratings’: audit in the British University system. European Review, 5, pp 305-321).

You may also like…

This Is Data-Driven Genius

This Is Data-Driven Genius

KPIs and why a fuel-gauge is data-driven genius. While driving you take a quick glance down to your dashboard and...

1 Comment

  1. Phil Husbands

    Great article Jon! It’s still often true that when a business decides what its KPIs are, not enough consideration is made to how the KPIs will be interpreted and responded to by the business’s people. I agree that human factors must always feature in decisions about how beneficial KPIs can be to a business.

    Reply

Submit a Comment

Your email address will not be published. Required fields are marked *