What if there are no metrics?

In some circumstances, people and organizations feel that they don’t have hard metrics to use for an assessment. In fact, there are almost always factors that can have metrics applied to help you assess the risks from qualitative factors.

Without data, you’re just another person with an opinion.”

Edwards Deming, US academic and father of the continuous quality improvement movement in the US.

Over the years, many people have said something to me along the lines of ‘we don’t have metrics to use for our assessment. We aren’t technical’. So, what do you do when it seems that you don’t have the metrics you need to complete an assessment?

In my experience, every organization has metrics, you might just need to dig a bit deeper to find them. The trick is to work backwards from your objectives and think about the data you track to measure progress and success.

Although you might not have failure rates or LTI (loss-time incident) statistics, that doesn’t mean that you don’t have things you can measure. Moreover, you’re probably already measuring lots of things. After all, how else do you measure success?

• Sales
• Graduations
• Customer satisfaction
• Workplace accidents
• Uptime
• The number of refund requests

These are all metrics that an organization might track. So even if you started off thinking ‘we don’t have metrics’, you probably do.

But how do you use these in a risk assessment?

The first thing to remember is that you don’t measure risk as an absolute value: it’s not a percentage or a unit of measure. Rather, you’re conducting a comparative process to put risks in order and prioritize them for action. Therefore, as long as you are using the same scale to assess each factor, you’ll be able to complete an assessment.

Once you have your factors, you can drop them into your methodology to get your risk ratings and subsequently put those in order.

But what kind of factors? Here are a few examples showing where you put these in the risk = threat * vulnerability * impact formula.

• Delays shipping a product (impact)
• Time a factory might be closed (impact)
• Sales performance increase / decrease (impact)
• Number of recorded accidents (threat)
• Number of refunds/product returns (impact)
• Number of days with inclement weather (threat)
• Days interrupted due to inclement weather (impact)
• Number of firewall attacks identified (threat)
• Number of firewall breaches identified (vulnerability)
• Downtime/data lost through breaches (impact)

Correlate these with some appropriate values, descriptions, and color codes to complete the assessment. As we saw in the previous section, this doesn’t need to be complicated to give you some usable data.

Furthermore, even though your data is relevant and applicable to your business, it’s still a small data set and vulnerable to subjectivity. If you try to make the process too complicated, you are building on top of a relatively weak platform.  Even large, comprehensive data sets, such as market history, don’t guarantee accurate results.

So please keep it simple. Avoid giving the impression of absolute certainty by getting into single dollar amounts where, in reality, $100s / $1000s might be a more realistic unit.

Finally, avoid ‘weighting’ factors. Weighting – where you attribute more value to certain factors automatically – is very subjective and puts the cart before the horse by prejudging your assessment.

So, even when you think you think you have no metrics, just take a moment and look at the measurements you use to judge the success of your business or the effectiveness of your operations. These are going to give you the factors you need to design your own metrics, analyze your risks and complete your assessment.