Serious Lessons About Bad Metrics

We have all lived and died by the statement:

“If you can’t measure it, you can’t manage it”

This statement has been attributed to W. Edward Deming – a leading management thinker in the field of quality.

Deming actually wrote the exact opposite:

“It is wrong to suppose that if you can’t measure it, you can’t manage it — a costly myth”

According to Robert A. Berenson MD, Deming consistently cautioned against requiring measurement to guide management decisions, observing that the most important data needed to manage often are unknown and unknowable.

Now whilst I am not against metrics and measurement – if fact I am all for them – they need to be well thought out to be effective.

Quantitative versus qualitative

We should not measure performance based purely on quantitative (numerical) measures. This stifles performance. Once we have reached that number we stop seeking to increase performance! Quantitative measurement breeds mediocrity.

Organisations that remain ahead of the competition and relevant in a world of constant and unpredictable change, are those that also measure qualitative aspects of performance. These could include creativity, innovation, self-management, experimentation, passion, enthusiasm, motivation and emotional intelligence.

Organisations embracing these aspects of performance are leading organisations that realise without behaviours such as creativity and innovation; they will be dead in the water.

The challenge with qualitative metrics is that the environment in which they are used has to be conducive to their use.

When there is a lack of trust in managers to do the right thing in assessing the performance of their staff, we revert to the safety of numbers. We are more comfortable with the black and white as opposed to the assessment of something with extensive variations in tone.

We think it would be too hard to measure but it really isn’t.

When you are watching gymnastics I bet you have an idea of who will win the competition without being an expert

Your assessment is not just based on the difficulty of the routine (i.e. quantitative) but also the artistic impression (i.e. qualitative).

You know a great performance when you see it, you don't have to be able to measure it!

So the organisation needs to trust its managers to do the right thing.

Employees also need to trust the judgement of managers. When measures of performance are subjective people can moan about it and say it’s not fair. But it is not the high performers that are moaning!

Managers need to help employees develop the capabilities that are not measured in numbers and lead by example. 

Driving wrong behaviours

When metrics are not well thought out they can drive the wrong behaviours. Metrics need to be smart and reviewed on a regular basis.

We have seen examples of metrics driving wrong behaviours here in Australia in recent weeks.

Driver breath-tests

 It has come to light that Victoria Police officers have been faking roadside breath tests. There was no financial gain for police officers to fake the tests, and the behaviour has been linked to stretch quotas that officers are expected to reach.

According to the ABC, a highway patrol unit in suburban Melbourne is meant to complete 50 breath tests in a single shift.

In the same report, the ABC stated:

“To reach their targets, police divisional vans and highway patrol vehicles often set up makeshift breath testing sites on the side of the road.

This can be unsafe, particularly on night shifts in dark conditions.

The ABC understands that when the end of the month is approaching and targets haven't been met, lots of officers are sent out to conduct as many tests as possible — sometimes hundreds.”

The Transport Accident Commission (TAC) has suspended $4 million (AUD) funding for Victoria Police roadside operations, after an investigation revealed officers falsified more than 250,000 roadside breath tests in the state over a five year period.

Whilst there has been no impact on prosecutions etc. the standing of the Victoria police has been severely damaged, resulting in both adverse reputational and financial implications.

Dollarmite

The Governor-General of the Commonwealth of Australia established a Royal Commission into Misconduct in the Banking, Superannuation and Financial Services Industry in Australia on 14 December 2017.

The Royal Commission has recently exposed The Commonwealth Bank Australia (CBA) of shady dealings (along with other major financial organisations).

In one of these dealings thousands of children’s CBA Dollarmite accounts have been fraudulently used for the staff’s financial gain.

Staff at a number of the big bank’s retail branches made transactions in the junior saver program without the knowledge of parents, by putting just enough money in each account to accrue bonuses and meet targets, according to an investigation by Fairfax Media.

As Fairfax Media reported, this widespread scam was instigated to meet aggressive performance targets.

The investigation also revealed that the scam is part of a broader culture of gaming financial incentives at the bank where staff were caught faking customer referrals to boost performance targets and earn rewards.

Whilst there was no financial harm to CBA customers, the activity has resulted in reputational damage to CBA and loss of customer trust.

The Fairfax report stated:

Finance Sector Union president Louise Arnfield said that for over a decade union members have been calling out the risks of systemic practices and bonus structures that put profit before all else – “a culture where speaking up was often a dangerous thing to do”.

“It wasn’t front-line workers who benefited from these systems, it was the people who created and enforced them that got the huge bonuses and pay,” she said.

What gets measured gets done!

What gets measured gets done but is the ‘done’ thing the right thing?

When you set out to measure something without some deliberation, expect the measurement to drive people to do whatever is needed to meet the metric.

Serious consideration needs to be applied when creating metrics and reflection on the behaviours that you wish to drive. If you don't, they may turn out to be counterproductive and cause considerable damage as per the recent examples in this post.

Employee involvement

If we want to set the ‘right’ goals and targets, perhaps we should consider involving employees in the process?

Set the right targets and link them to the larger business goals and objectives so employees can see how they contribute. This linkage also helps drive the right behaviours. 

If a CBA goal was one of customer trust and engagement, and employee targets were linked to it, this may have dissuaded the behaviour recently exposed. 

If a Victoria Police goal was all about customer and community, transparency and integrity, and employee targets were linked to that, this could also discourage undesired behaviours.

When employees are involved in setting targets and have a clear understanding of how they help achieve organisational ambitions not only do the targets have more meaning but it can also motivate, engage and achieve better outcomes for both the organisation and the individual.

In Short

Encourage and measure the desired behaviours through carefully considered measurements and metrics.

Review the metrics on a regular basis and check that they are driving the behaviours that you intended.

If they are not, fix them before they cause unintended damage!

Involve employees in setting their goals and targets. 

Do it with them, not too them. Now where I have I heard that before?