Software Measurement

Watch out for these unexpected negative outcomes of measurement

It’s important to find healthy ways to measure

Photo by Jeremy Bezanger on Unsplash

“When a measure becomes a target, it ceases to be a good measure.” — Goodhart’s law

The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”— Campbell’s Law

Which measures to choose?

Honestly, the first thing I would ask you before attempting to answer is “what do you want to achieve?”

Here be dragons

Photo by Vlad Zaytsev on Unsplash
  1. The measures are not maintained.
    Since you have no clear goal, you don’t think about how to evolve your measures. You also don’t question their value, and you don’t know when to deprecate them.
  2. You don’t know what to do with the measures or you abandon them.
    Many of your measures will either provide you with no actionable information or no value at all. Eventually, you will abandon them, because measuring takes time.
  3. You don’t know what to do with the measures, but you try anyway.
    And as a consequence, you spend effort in fixing what doesn’t need fixing, wasting your precious time (and time from others). If it ain’t broke, don’t fix it.
  4. Local optimizations do not always translate to global optimizations.
    Let’s say that during your development process you update an artifact that your team never uses. Then you decide to streamline things and stop updating the artifact. Well, that’s your troubleshooting guide, that helps the support team handle user requests. They don’t know it’s not being updated anymore, they think there’s simply nothing to update. Now more incidents are coming in and they can’t handle them. They ask your team for support. There goes your productivity away (that one you were trying to improve).
  5. You don’t know who will see the measures.
    Measures require analysis to be interpreted. This often takes into consideration the context of what happened at a given moment or characteristics of where they come from. One example is comparing velocities from different teams. If a team compares each story to a 2-points story that took 30 minutes to complete and another team compares each story to a 2-points story that took 1 hour to complete, caeteris paribus (all other things being equal), the first team will have a velocity twice the value as the second. If you don’t know who these measures serve, you don’t know what story the numbers need to tell.
  6. You don't know the behavior you want to stimulate.
    Remember our initial discussion, around Campbell’s Law? You want to choose measures that are neutral or stimulate a positive behavior. Positive behavior is one that produces results aligned with goals. If there are no goals to achieve, how do you decide in which direction you want to influence the behavior?

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Peter P. Lupo

Many management blogs focus on soft skills. This blog is about hard skills! Measurement, indicators, approaches, etc., for Software Engineering Management.