ERM and the Kaplan-Mikes (Harvard) heresy: ISO 31000 is “not relevant”

On a chat site recently, US-based performance management specialist Robert Kaplan was quoted as saying to a conference in the Middle East that rules-based risk management was “not relevant”.  When the interlocutor (Domenic Antonucci) pushed him for specific clarification on how this applied to ISO 31000, Kaplan is quoted as saying he found it to be “rules-based” and “not relevant” to the development of ERM.

A furore ensued.  As part of this, a number of luminaries of the risk world corresponded with his colleague, Anette Mikes, and Prof. Kaplan.  A lengthy and considered response from Dr Mikes included the following statement directly from Kaplan:

Standards and innovation have an inherent tension between each other, in some cases they can be mortal enemies. We standardize when we understand a process very well and want to ensure that everyone follows the same processes and measurements because they have been proven to yield superior results. But in an environment with limited knowledge and experience, premature standard setting will inhibit innovation, exploration and learning. We can standardize around preventable risks because managers do understand them well, and have developed excellent processes to prevent them from occurring. But we are just learning about the management of strategy risks and external, non-preventable risks. To think we can standardize the “best practices” for managing these two risk categories through an ISO-based process seems like a highly risky proposition for risk professionals to be engaged in with our present body of knowledge.

To add to the apparent heresy, Dr Mikes reportedly stated recently, at a conference in London, that risk management should focus on downside risk only.  Challenged on this, she stated in the correspondence referred to that “[a]s a result, we find that, on balance, risk management is primarily about understanding what can go wrong as opposed to what can go unexpectedly right.”  The ISO enthusiasts are lighting pyres.

Let’s address each of these propositions in turn.

First, Bob’s point about standardization.  The purpose of standards is, well . . . standardization; that is “to ensure that everyone follows the same processes and measurements . . .”  But it is the next statement that is the most revealing and introduces the third key proposition: “. . . because they have been proved to yield superior results.”  The next key insight is that the body of knowledge in risk management is not sufficiently developed or settled to justify standardization and that attempts to do so may have adverse consequences.

Some may feel that ISO 31000 is a guide.  This is one view that has been expressed by proponents of ISO 31000 during this debate.  The first substantive sentence of the Scope section of the Standard is “This International Standard provides principles and generic guidelines on risk management.”  That is, it IS a Standard that claims it is a guide. This is pretty unequivocal.  The same section notes that the Standard is not “intended for the purpose of certification.”  But it is still a Standard.  It’s there in black and white.  On the cover.  At the top.  In big letters.

This reinforces Bob’s fundamental point: the body of knowledge around organizational management of risk it is not ready for certification.  Personally I do not believe it ever will be or should be.

ISO 31000 is but three years or so old.  But its origins or ‘DNA’, in the words of some proponents, lie in AS/NZS 4360 : 1995, which has considerably greater vintage.  Has that document “been proved to yield superior results?”  I was involved in risk management in that jurisdiction at the time and have paid close attention.  But I am yet to see any substantiable evidence that application of AS/NZS yields superior results.

Simply put, Australian and NZ firms using the Standard have not been shown to defend themselves against uncertainty more effectively or produce better results over the long term than those elsewhere who are not using it.  Some will have; others will not have. But where is the evidence that, on balance across a range of firms, use of AS/NZS 4360 produces superior management of risks over a sustained period?  When is it more effective; when is it less effective or wholly ineffective?  I may have missed such evidence, but I have a strong suspicion that it is because none has ever been collated or produced.  The same is true, and likely in my estimation to remain true, for ISO 31000.

Proponents have been quick to laud successful implementations, but there are two problems here: (i) those from whom evidence is collected on the success of the implementation are seldom unbiased as many will have instigated or been involved in the implementation and (ii) successful implementation does not equate, in any way, to ‘yielding superior results’ over time.  Any number of reports of successful implementations may be reported but the reality will be something less and may disappoint over time.  Again, there is little or no evidence that this is not the case; that the benefit of such a system is durable in the face of its primary purpose: to improve the firm’s response to uncertainty.

Perhaps most controversially for ISO devotees, Kaplan and Mikes appear to disregard so-called ‘upside risk’.  The response to this assertion has been dismissive and decidedly impolite.  However, this appears to be a classic case of people talking past one another and the dangers of self-reference.  ISO 31000 defines risk as: “effect of uncertainty of objectives.”  While there are material problems with this rather summary assertion, we will ignore these for now.  Note 1 to the definition states:

An effect is a deviation from the expected — positive and/or negative.

Upside risk is (re-)born.  But the problem is that this is not how most people use the word risk.  We have terminology and a vocabulary for variance or volatility.  We do not need to distort or contort the word risk to get to opportunity.  We do not need to transliterate from Chinese.  We could just use standard vocabularies drawn from everyday usage.  Just like most managers.

‘Upside risk’ is a misinterpretation of the nature of variance and commercial gain.  There is undoubtedly the possibility of gain from potentially adverse environmental conditions that manifest unexpectedly benignly or commercial performance or demand that exceeds expectation.  That does not need to be called ‘upside risk’; it is merely commercial advantage from risk-taking in which conditions were better than expected.  Taking commercially well-considered risks of loss brings the possibility of loss or gain.  Gain from risk-taking is the universally desired outcome.  This does not require a glossary on hand to interpret.

If ISO 31000 advocates wish to isolate themselves by creating vocabularies that defy ordinary usage, the Oxford English Dictionary or Merriam-Webster, they are entitled to do so.  But when the limitations of this approach are exposed, ad hominem attacks are not an effective retort.

The heresy of Prof Kaplan and Dr Mikes is to ignore the imprimatur of ISO and, following the advice of Marcus Aurelius, look to the essence of the thing.  Just as with the recent COSO redraft, stating that a document is principles-based does not make it so.  Let us consider an example at the heart of the Kaplan-Mikes focus.  Principle 4 states: “Risk management explicitly addresses uncertainty.”  Nowhere is it explained how uncertainty is folded in to the framework or its implications.  Instead, ISO 31000 advocates that its users:

generate a comprehensive list of risks based on those events that might create, enhance, prevent, degrade, accelerate or delay the achievement of objectives. It is important to identify the risks associated with not pursuing an opportunity. Comprehensive identification is critical, because a risk that is not identified at this stage will not be included in further analysis.

Quite.  But in the face of irreducible uncertainty, how can analysis of its effects ever be comprehensive?  In workshops, I have had participants proudly proclaim that they have identified over 10,000 risks.  This seems more like a bureaucratic make-work scheme than an efficient corporate routine for addressing uncertainty.  And if a risk does not make it on the list, as ISO states, it “will not be included in further analysis.”  This seems to me to be the antithesis of explicitly addressing uncertainty; it seems to be reducing irreducible uncertainty to what we already know we do not know.  This is a partial definition of uncertainly only.  That is not enough.

If Bob Kaplan did say that ISO 31000 is not relevant, he is only partially correct.  The problem is that, as long as people continue to advocate its use, it will be relevant.  The greater problem is that, at its core, ISO 31000 is conceptually limited and thus flawed.  In specific instances and with broadly-thinking users, that may not prevent it being useful.  But it does fall well short of being definitive.  And being definitive is what Standards are for.

18 thoughts on “ERM and the Kaplan-Mikes (Harvard) heresy: ISO 31000 is “not relevant”

  1. Hi Peter, what framework, standard or principle to you mainly advocate in your practice and training?

  2. Mike

    I do not come at this from an off-the-shelf perspective. My principal concern is that all the frameworks I have seen treat risk management as a linear process to be bolted on to the work of a firm. I find this a bit of a nonsense.

    There is a simple test that can be applied to risk management practice: do you use it yourself? If you are going on a camping holiday, do you (or anyone else) make out a risk register? It is simply, well, . . . ‘anti-behavioural’, unnatural.

    Outside financial markets and credit risks, we need to get a whole lot smarter about recognising the roles people play in processes and how they act. Workshop-based elicitation of risks is, and can be shown to be, a tremendously distorting exercise. By far the most effective form of risk identification is sitting down over a coffee and thinking about a problem or letting your mind roam free in the shower or listening to a Bach cantata or driving past a bus station . . . Then, talking to someone you trust about the problem. Then trying to marshall some evidence to support your thoughts. That is what happens in the real world.

    Risk frameworks assume our next step is assessment then evaluation of treatment options then treatment then monitoring.

    The linear representations do more harm than good precisely because they displace all the good stuff above. I have frameworks I use but they are in preparation rather than ready to hit the road. My tentative proposition is naturalistic organisational risk management or NORM. More in due course.

  3. Hi Peter

    The simple truth is that writing a list of ‘risks’ and trying to ‘manage’ them is no substitute for managing well in the face of uncertainty/limited knowledge. The idea of making risk lists, promoted by AS/NZ 4360 and still encouraged by ISO 31000 (though not so explicitly) is very much a minor and recent fad in the context of man’s efforts to deal with uncertainty. Those efforts can be traced back hundreds of years to early work on probability theory, then forwards to an explosion of powerful ideas and techniques within management science in the 20th century.

    Happily, although ISO 31000 is a risk lister’s approach, most people (according to one of my surveys last year) prefer techniques for managing under uncertainty that do not involve making lists of risks.

    • Matthew

      Thanks for the note. I am very happy to hear it. However, in my experience, while the ‘preferred’ technique may be different, most firms still produce lists of risks, even if just for form. Doesn’t this then represent rather a collosal waste of money?

      • Most firms don’t do anything like that, but of course I’m counting small firms that are under no pressure to go through the risk listing rigmarole and so they don’t. There are also countless sub-units within larger organizations that escape the risk listing programmes.

        However, in UK plcs this sort of risk listing exercise is now sadly very common. A risk manager is employed to facilitate the creation of a risk list, and the next person in that role may have the job of sorting out the mess of thousands of higgledy-piggledy ‘risks’ that were listed initially. Provided this is just a waste of time then the damage is limited because people usually do their best to spend as little time on it as they can get away with.

        My great fear is that it might one day be seen as an alternative to managing uncertainty through the way core management activities are done, so that people feel free to take whatever reckless and stupid macho management gambles they can persuade others to accept because, surely, risk management has risk covered.

  4. Hi Peter, Mike, Matthew,

    Just wanted to jump into this quite interesting discussion. First, when I read heresy, I hear “against dogma”. Now, let’s be clear that dogmatic behaviour is not good under most circumstances. Especially in developing areas such as risk management, which Matthew called the new Wild West only a few years ago (Matthew, I’m paraphrasing, but I really liked the snake oil salesmen reference you made ;-)) we need to make sure that we don’t hold on to dogma’s that are unproven.

    However, and this is important as well, what I feel that Kaplan fails to address is the error in expectations we all appear to have with respect to risk management. While not by far the perfect risk management approach, we need to look beyond the limitations of ISO 31000 and look at what it does bring to the table. However, it’s easy to dismiss an approach based on the problems perceived by the experts, while within certain limitations COSO ERM, ISO 31000, AS/NZS 4360 assist in developing a better and better view on what good risk management should be.

    Compare this to physics, for example. Any theory which explains even part of what we see and internally and externally shows consistency is considered as a valuable addition to the overall body of knowledge. It explains perhaps only part of the issue, but at least it does that. It may be wrong but it will give us a basis to sharpen our insights. The steady state theory, for example, even while mainly wrong, has significantly contributed to our understanding of how elements were created in the early universe.

    This being said, I believe that COSO ERM, ISO 31000 and other risk management approaches will gradually make way for newer approaches that build on the lessons learned from these approaches. Pretty much like Sarbanes-Oxley showed us what not to do to avoid future Enrons or Worldcoms.

    Just kicking them to the curb as irrelevant is an easy and even cheap trick which is unworthy of an academic heavyweight such as Kaplan. He certainly has a number of points where he makes a case, but he should look at how each of the current frameworks contributes and how it can be adapted, amended or even completely turned around to be used for the better of risk management.

    For the record, I am a reformed list-maker. I don’t agree that ISO 31000 is all about making lists. For me, and how I teach it, it is more about an awareness that there are issues we know, issues we are aware of and issues we are completely unaware of. And that communication and consultation, in whichever form is relevant for your organization (cfr. some of Matthews excellent surveys, by the way) is a key factor in truly treating risk.

    That said, we still like to use our little checklists to make sure we have not forgotten anything. They are no longer risk models, they are just simple risk checklists. By ‘relegating’ them from model to checklist we aim to clarify to the users they are merely one of a set of tools we use to assist them in thinking about and discussing risk on a regular basis.

    Just my 2 cents. Given I consider this very important, I’m actually copying this on my little blog as well, with a link to yours.

    Cheers, Ben Broeckx

  5. Ben

    Thanks for the note. I wrote a long response, only to realise I was actually writing my next blog. Will post shortly and return the favour with a link.

    Great addition, thanks. Worth way more than your valuation suggests!

  6. Pingback: Of shoes and ships and ISO wax … « The risk debate

  7. Bizarrely, Robert Kaplan seems to have gone down the risk listing route in his own theorising about how to improve on the balanced scorecard management approach. He makes ‘risk management’ a whole new area that’s sadly separated from the rest of management.

    He would have done much better to develop the good material already in his books on ‘strategy as hypothesis’ and so discourage people from getting too attached to the beliefs enshrined in their strategy maps and targets.

    • A thought on Matthew’s reply:
      The point on enshrined beliefs is oh so valid, and a key barrier to getting good risk management adopted, so I’m glad you bring it up.
      In the context of assisting risk management implementation, be it in my own organization (the Belgian Development Agency), be it when I’m teaching, people often approach us very willing to make an initial investment … but very weary of a continuous effort over a long period.

      There is an enshrined belief that risk management takes an initial effort (developing the risk model) which is a high cost with the benefit that after that, the model exists and need not be re-examined. I think the mechanical approach mainly preached by COSO is to blame for that set of false expectations.

      What I’m trying to get across when talking to these people is that what they are looking at is the tools, not the practice. And while tools may be supportive of embedding a practice, risk management is in part also a state of mind. It’s about being aware of and attentive to the fact that as an organization, we are not invulnerable and live in a world where problems may occur.

      In essence, risk management is like trying to build an amygdala (the lizard brain, if I’m not mistaken) into an organization. Trying to develop an almost intuitive, embedded, learned reaction to certain types of issues. Again, given an organization has no real brain, it’s a challenge. The only way is through training and experience. This is the point where most people start to understand that a commitment to risk management is not a commitment to a tool or a methodology, but rather a new way of thinking about and with your organization.

      Peter, possible 3 cents worth?

  8. Ben

    Interesting thoughts. Neurophysiology is not my strong suit, so cannot comment on the evolutionary origins of the limbic system or our common structures with lizards. And if I could, I’m not sure I’d want to (which is not to suggest I would not like to be able to!).

    I agree fully with your observations. I think they warrant deeper consideration – the different and differentiable motivations of people in promoting risk management in its many and varied guises, or the politics of risk management, if you will – and I can sense another blog thereon.

    My only observation would be that, if at all possible, I avoid now using the word ’embed’ or any participle related to it. In the hands of a skilled operator (as I suspect you are) it is a useful shorthand; but it is horribly overused.

    Going back to the brain issue, there is a lot of fertile ground for thinking about risk in neurophysiology, as Iain McGilchrist took great pains in explaining to me one evening (I was buying the Rioja). But I am sure that your central point, about distributed perception, sensing and imagination is a vital one. Far too many commentaries talk about the firm as if it were singular or homogenous; there is far too little attention given to the different aspects of sub-firm life, of divergence of interest, of distribution of knowledge and intellect and analytical ability and attention and . . .

    All these things are both fertile ground for thought and important to understanding how to get organisations to work better with uncertainty.

    Over to you . . .

  9. Ben

    Great post. I am sure this is fertile and very important ground although, as stated earlier, my knowledge of neurophysiology is limited. Two thoughts come to mind:

    1. One of the metaphors Gareth Morgan uses in his great work Images of Organization he calls the brain metaphor – that organisations work in ways that can be compared to neurological systems.

    2. In addition to your amygdala response proposition, there is also the very insightful work of neuroscientist and psychiatrist Iain McGilchrist in The Master and his Emissary about brain hemispheres and their influence on our individual and group behaviour. Iain has demurred so far from applying this work to risk but I will keep encouraging him.

    I am not sure that neuroscience will ever lead us to prescriptions but it can and should encourage us to reflect on our approaches to behaviour including the very important areas of agency (which you describe as free-rider problems) and control systems more specifically. Until we grapple with this stuff, we just keep inventing layers of rules that are counter-productive.

  10. I am in partial agreement and admittedly amused by the purposely exagerated negativity expressed by you on all existing RM standards. I think the strong opinions and antagonistic tones are all appropriate responses to some very questionable practices of quelching dissent among devout followers of some of these standards. Just to be balanced, however, I see ISO31000 and all other standards as valuable to the field of RM, albeit with a very judicious eye toward the complete and full applicability of any of these in the real world. None of them should claim to be the ‘end-all and be-all” of RM–even if only as a guide!!! (As you said in your blogs, the field is too young and there are still new ground-shifting concepts to be discovered such that any prescriptive rules or principles would be premature).

    The scandalous act and cardinal sin of some followers of these standards is to tout any one of them as exactly that, and next resort to personal attacks rather than engage a rigorous discourse and exchange of ideas. (I wonder if these folks are mindful of the rate of obsolescence of newfangled concepts in a young field.) Given this kind of “bullying-them-to-submission” discussions going on, I do support the dissenting voice, along with the equally strong tone–but without falling into similar traps. In the case of Kaplan and Mikes, I think that categorizing ISO31000 as rules-based and therefore irrelevant is skirting that line of reasonableness. However, I would like to reserve my opinion til more facts or discussions ensue.

    In the meantime, It’s all good. IMHO.

  11. The well known crop of ‘risk management standards’ are mostly guidance documents issued by organisations that issue standards. I still think they are presenting themselves as ‘standards’ but it also means that they can be compared with guidance issued by other authoritative sources.

    Some literature searching I’ve done in the past few months has revealed something fascinating and important, which is that there is a ton of other guidance on how to manage risk and uncertainty issued by authoritative sources (not just single authors). Some of it is really good.

    This material is often to be found in documents on other topics, such as decision making or policy review. That’s why, when you Google for ‘risk management standards’ you don’t get it.

    Nevertheless, this material is important and I suspect some of it is required practice rather than just guidance. I’ve made a list of things I’ve found so far, with links and brief descriptions of the relevant content here:

  12. Pingback: Why ISO 31000 is dangerous « Critical Uncertainties

  13. My perspective on ISO 31000 is essentially a Weickian one, that it embodies a process paradigm which is an anathema to mindfulness. And mindfulness, a willingness to look at the small signals that may overthrow our grand theories, is what you need to go hunting black swans.


  14. I’m a bit late to the party, but this post and the conversation are like finding out where the smart kids have been hanging out. Love the concept of “building an amygdala” for an organization.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s