ERM and the Kaplan-Mikes (Harvard) heresy: ISO 31000 is “not relevant”

On a chat site recently, US-based performance management specialist Robert Kaplan was quoted as saying to a conference in the Middle East that rules-based risk management was “not relevant”.  When the interlocutor (Domenic Antonucci) pushed him for specific clarification on how this applied to ISO 31000, Kaplan is quoted as saying he found it to be “rules-based” and “not relevant” to the development of ERM.

A furore ensued.  As part of this, a number of luminaries of the risk world corresponded with his colleague, Anette Mikes, and Prof. Kaplan.  A lengthy and considered response from Dr Mikes included the following statement directly from Kaplan:

Standards and innovation have an inherent tension between each other, in some cases they can be mortal enemies. We standardize when we understand a process very well and want to ensure that everyone follows the same processes and measurements because they have been proven to yield superior results. But in an environment with limited knowledge and experience, premature standard setting will inhibit innovation, exploration and learning. We can standardize around preventable risks because managers do understand them well, and have developed excellent processes to prevent them from occurring. But we are just learning about the management of strategy risks and external, non-preventable risks. To think we can standardize the “best practices” for managing these two risk categories through an ISO-based process seems like a highly risky proposition for risk professionals to be engaged in with our present body of knowledge.

To add to the apparent heresy, Dr Mikes reportedly stated recently, at a conference in London, that risk management should focus on downside risk only.  Challenged on this, she stated in the correspondence referred to that “[a]s a result, we find that, on balance, risk management is primarily about understanding what can go wrong as opposed to what can go unexpectedly right.”  The ISO enthusiasts are lighting pyres.

Let’s address each of these propositions in turn.

First, Bob’s point about standardization.  The purpose of standards is, well . . . standardization; that is “to ensure that everyone follows the same processes and measurements . . .”  But it is the next statement that is the most revealing and introduces the third key proposition: “. . . because they have been proved to yield superior results.”  The next key insight is that the body of knowledge in risk management is not sufficiently developed or settled to justify standardization and that attempts to do so may have adverse consequences.

Some may feel that ISO 31000 is a guide.  This is one view that has been expressed by proponents of ISO 31000 during this debate.  The first substantive sentence of the Scope section of the Standard is “This International Standard provides principles and generic guidelines on risk management.”  That is, it IS a Standard that claims it is a guide. This is pretty unequivocal.  The same section notes that the Standard is not “intended for the purpose of certification.”  But it is still a Standard.  It’s there in black and white.  On the cover.  At the top.  In big letters.

This reinforces Bob’s fundamental point: the body of knowledge around organizational management of risk it is not ready for certification.  Personally I do not believe it ever will be or should be.

ISO 31000 is but three years or so old.  But its origins or ‘DNA’, in the words of some proponents, lie in AS/NZS 4360 : 1995, which has considerably greater vintage.  Has that document “been proved to yield superior results?”  I was involved in risk management in that jurisdiction at the time and have paid close attention.  But I am yet to see any substantiable evidence that application of AS/NZS yields superior results.

Simply put, Australian and NZ firms using the Standard have not been shown to defend themselves against uncertainty more effectively or produce better results over the long term than those elsewhere who are not using it.  Some will have; others will not have. But where is the evidence that, on balance across a range of firms, use of AS/NZS 4360 produces superior management of risks over a sustained period?  When is it more effective; when is it less effective or wholly ineffective?  I may have missed such evidence, but I have a strong suspicion that it is because none has ever been collated or produced.  The same is true, and likely in my estimation to remain true, for ISO 31000.

Proponents have been quick to laud successful implementations, but there are two problems here: (i) those from whom evidence is collected on the success of the implementation are seldom unbiased as many will have instigated or been involved in the implementation and (ii) successful implementation does not equate, in any way, to ‘yielding superior results’ over time.  Any number of reports of successful implementations may be reported but the reality will be something less and may disappoint over time.  Again, there is little or no evidence that this is not the case; that the benefit of such a system is durable in the face of its primary purpose: to improve the firm’s response to uncertainty.

Perhaps most controversially for ISO devotees, Kaplan and Mikes appear to disregard so-called ‘upside risk’.  The response to this assertion has been dismissive and decidedly impolite.  However, this appears to be a classic case of people talking past one another and the dangers of self-reference.  ISO 31000 defines risk as: “effect of uncertainty of objectives.”  While there are material problems with this rather summary assertion, we will ignore these for now.  Note 1 to the definition states:

An effect is a deviation from the expected — positive and/or negative.

Upside risk is (re-)born.  But the problem is that this is not how most people use the word risk.  We have terminology and a vocabulary for variance or volatility.  We do not need to distort or contort the word risk to get to opportunity.  We do not need to transliterate from Chinese.  We could just use standard vocabularies drawn from everyday usage.  Just like most managers.

‘Upside risk’ is a misinterpretation of the nature of variance and commercial gain.  There is undoubtedly the possibility of gain from potentially adverse environmental conditions that manifest unexpectedly benignly or commercial performance or demand that exceeds expectation.  That does not need to be called ‘upside risk’; it is merely commercial advantage from risk-taking in which conditions were better than expected.  Taking commercially well-considered risks of loss brings the possibility of loss or gain.  Gain from risk-taking is the universally desired outcome.  This does not require a glossary on hand to interpret.

If ISO 31000 advocates wish to isolate themselves by creating vocabularies that defy ordinary usage, the Oxford English Dictionary or Merriam-Webster, they are entitled to do so.  But when the limitations of this approach are exposed, ad hominem attacks are not an effective retort.

The heresy of Prof Kaplan and Dr Mikes is to ignore the imprimatur of ISO and, following the advice of Marcus Aurelius, look to the essence of the thing.  Just as with the recent COSO redraft, stating that a document is principles-based does not make it so.  Let us consider an example at the heart of the Kaplan-Mikes focus.  Principle 4 states: “Risk management explicitly addresses uncertainty.”  Nowhere is it explained how uncertainty is folded in to the framework or its implications.  Instead, ISO 31000 advocates that its users:

generate a comprehensive list of risks based on those events that might create, enhance, prevent, degrade, accelerate or delay the achievement of objectives. It is important to identify the risks associated with not pursuing an opportunity. Comprehensive identification is critical, because a risk that is not identified at this stage will not be included in further analysis.

Quite.  But in the face of irreducible uncertainty, how can analysis of its effects ever be comprehensive?  In workshops, I have had participants proudly proclaim that they have identified over 10,000 risks.  This seems more like a bureaucratic make-work scheme than an efficient corporate routine for addressing uncertainty.  And if a risk does not make it on the list, as ISO states, it “will not be included in further analysis.”  This seems to me to be the antithesis of explicitly addressing uncertainty; it seems to be reducing irreducible uncertainty to what we already know we do not know.  This is a partial definition of uncertainly only.  That is not enough.

If Bob Kaplan did say that ISO 31000 is not relevant, he is only partially correct.  The problem is that, as long as people continue to advocate its use, it will be relevant.  The greater problem is that, at its core, ISO 31000 is conceptually limited and thus flawed.  In specific instances and with broadly-thinking users, that may not prevent it being useful.  But it does fall well short of being definitive.  And being definitive is what Standards are for.