In today’s business world, it is almost impossible to find an organization that does not establish and track metrics for any number of business purposes. Accordingly, many relevant and beneficial methodologies for implementing metrics exist, such as The Balanced Scorecard.
The assumption is that an organization that regularly monitors important measurements is poised for innovation and success because it:
1) Quickly adapts to changing situations;
2) Understands the on-going impacts of its major initiatives; and
3) Plans ahead for important decisions and potential risks to achieving results.
With such significant potential benefits, it behooves an organization to establish and track the best possible performance metrics. Therefore, it is incumbent upon practitioners to recognize warning signs that their metrics may be lying to them.
Below are 5 potential warning signs:
Warning Sign 1: The person establishing or monitoring the metrics says, “I’m not a data person, but here are the numbers and this is what they mean.”
When someone admits numerical incompetence just before developing metrics or presenting numbers, you should always interpret any follow-on information with extreme caution.
Warning Sign 2: A coworker asks that a number be changed or omitted to appease a manager and avoid consequences
Successful metrics depend on objectivity. The moment a number is omitted or obfuscated due to idiosyncratic whims, the remaining numbers become questionable.
Warning Sign 3: Assumptions
Be aware of situations where an unsubstantiated assumption is made about the relation between a metric and something else (e.g., “Our customer service rating is suffering – this means customers are probably not buying as many units.”) Without additional data, these types of assumptions are difficult to justify and may drive improper business decisions.
Warning Sign 4: Short duration data are used to describe long-term trends
In his article, Measurements that Mislead, Jonah Lehrer lays out a phenomenon in which predictive errors occur when short duration (or peak performance) assessments are used. For example, employees may artificially increase customer service speed if they are aware they are being measured over a short duration. Similarly, employee attitude assessments often represent momentary or situational opinions, whereas the use of real-time data collection could provide more insight into how employees truly feel.
Warning Sign 5: Monitoring systems place extreme accountability on “hitting the numbers”
Organizations and individuals that “live or die” by the numbers have a high level of motivation to fake it. This scenario has played out in numerous ways recently. Managers at the VA turned a blind eye to number fudging that made them look better from an accountability perspective. Also, in response to onerous “No Child Left Behind” metrics requirements, teachers across the country have been accused of unethical behavior in order to raise test scores.
How do you create reliable measures?
If any of the above warning signs start flashing, consider the following remedies:
- Assign people knowledgeable in data analysis to the team charged with developing and monitoring metrics (if you don’t have any of these people, you may have bigger problems).
- Never comply with requests to alter numbers to make someone “happy.”
- Measure what is actually important. If “number of units sold” is what is truly important, do not simply infer this by measuring something else.
- Develop and implement real-time performance dashboards to track true trends.
- Establish a culture that emphasizes partnership over accountability.
Remember, working toward achieving results as a team is almost always more important than expending too much energy and time chastising those who were holding the accountability stick.
Dr. Paul Eder is a Lead Consultant with The Center for Organizational Excellence, Inc. (COE). The opinions in this piece are the author’s and do not necessarily represent the views of COE or Innovategov.org.