I knew from the very first episode that I would love the US TV show ‘The West Wing.’ I studied economics. I think the very first lecture I had on economics included reference to the ‘post hoc‘ fallacy – post hoc, ergo propter hoc (after the fact, therefore because of the fact). This was the name of the very first episode of The West Wing. It was nerdy. I was hooked.
My former colleagues at Ernst & Young have recently published a study called ‘Turning risk into results‘. It has generated the usual buzz of a glossy, big-four publication on the various risk-related chat sites. One contributor described the study as “interesting”.
Well, it’s interesting up to a point. If you read it carefully, my former colleagues are not really making any profound point in this report. The difficulty is that the report does not do what it says on the tin. The evidence they present really says: if you do risk well (according to their – unspecified – yardsticks), chances are you also make money well. They are presenting a correlation. But, chances are you also make widgets well, service customers well, attract and retain quality staff well, manage contracts well and plan and communicate well.
What the report does not present is causation – that firms get superior results BECAUSE they do risk well. That is, they do not show that risk management is EFFECTIVE, just that they do a range of things that, by E&Y’s estimation, suggest that they do it repeatably and with a focus on why they do it – the usual definition of process maturity.
They use a title that suggests a causation but does not state it directly, and a subtitle that goes even further but still does not cross the line. This is classic post hoc, ergo propter hoc thinking (or marketing, depending whether you would prefer to regard them as intellectually lazy or cynical).
Of course, there are some very sound suggestions in the report. Definition of risk thresholds at corporate and sub-corporate levels is an essential discipline, especially if related to the firm’s risk tolerance. Stress tests are clearly useful tools across businesses in all sectors. Linking planning to risk – levels of uncertainties in underlying planning assumptions – as well as to planning process is likely to encourage acceptance and addresses the nature and extent of uncertainties facing the business. Understanding formally how the risk in firm’s business model translates to result volatility is a rarely-used but particularly useful exercise. Getting the board to consider uncertainty and senior executives and manage to discuss differing understanding of risk is even more useful again.
But none of these demonstrates that, in the firms E&Y reviewed, risk management was effective – that losses or other negative outcomes were averted. Nor does it seek to understand or explain the circumstances or conditions in those firms in which it was not effective or why.
Until we move from demonstrating correlation to seeking to understand causation, risk management or ERM, if you prefer, will not move forward.