Many organizations today try to assess the risks they face. External pressures have caused organizations to increase the breadth, depth and thoroughness of risk assessments. Numerous forces have caused many to increase both monitoring and evaluation of risk assessment effectiveness, especially with regard to regulatory compliance. Boards and senior executives desire a deeper understanding of how they can enhance risk management to create the greatest shareholder value. The focus on risk management is growing for other stakeholder groups as well, with rating agencies, advocacy groups and enterprise partners demanding more transparency. We would expect that organizations are now routinely conducting risk assessments on an enterprise-wide basis with value-added resultsï¾—but this is not the case. Why not?
The OCEG GRC Strategy Study, conducted earlier this year, evaluated governance, risk management and compliance (GRC) efforts in more than 250 organizations globally and found overwhelming value in overcoming fragmented, siloed approaches to GRC and establishing integration of these efforts. Even so, the study also found that most companies still have a long way to go in achieving integration and the maturity that it brings about.
Weakness in risk assessment amongst those with fragmented GRC is apparent in the Strategy Study, where nearly half of the selfidentified fragmented entities report being worse than their peers in risk identification and assessment, and nearly three-quarters describe their management of risk appetite as ad hoc or only at an initial level of development.
In light of these findings, we undertook our next study, the OCEG Effective Risk Assessment Study (the Risk Study), sponsored by Ernst & Young, to further analyze approaches to risk assessment and identify those taken by companies that find value from their risk assessments. Preliminary findings show that many of the contrasts between those who say they find sufficient value and those who don’t are striking, while at the same time, some of the similarities between these two populations are equally revealing.
A Lack of Value
Fully 80 percent of the more than 250 participants in the Risk Study say their organizations are not getting as much value from their risk assessments as they should. They see that they will both improve value and protect it better if they increase the use of risk assessments across a range of activities.
About half use their risk assessments to evaluate operational and financial performance and to influence internal audit planning. Less than onethird, though, use risk assessment to evaluate board or c-suite performance, define monitoring or metrics, or challenge leadership decisions and business plans. Yet, when asked if these applications of risk assessment should be used, and if they would add or protect value, almost all say yes, and they are right.
Virtually all the participants who do believe their risk assessments create full value say that their risk assessments add to or preserve value in operational performance, improve or enhance performance metrics, influence internal audit planning and enable them to challenge, refine or modify business operating plans. Organizations that effectively identify a complete set of risks, tie them to enterprise and business objectives, articulate the causes and effects and have a structure for consistently assessing risk find they are able to make more informed business decisions.
More than three-quarters of those who are not getting full value from their risk assessments believe that the scope of their assessments is not as comprehensive or vigorous as it should be, and again they are right. They are not taking many of the steps essential to a mature process.
For example, these organizations have a desire to do a better job of linking risks to business objectives (both corporate and at the business unit level) and using criterion for classifying the appropriate management response to risk. They also see the need to better analyze competitor risks and events. In fact, less than one-fifth of all of the study participants analyze the risks of their competitors.
Even many of those who say they are getting good value from their risk assessments see room for improvement. More than one-third say that their assessments should be more comprehensive and vigorous. They see the need to improve many areas , including how they link risks to business objectives and how they use criteria for classifying the appropriate management response to risk.
Most of the study participants fail to validate their assessments by determining whether detected risk/opportunity events occurred with the projected probability. Only one-quarter of those who find value in their assessments, and only one tenth of those who do not, engage in this critical step. While the adage, “you can’t effectively drive ahead while looking in your rear view mirror,” has its truth, it is important to look at past events to validate the accuracy of assessments and adjust the process if necessary.
Setting the Stage for Failure
The most striking difference between those who find great value in their risk assessments and the vast majority who do not is the degree to which they apply standardized processes to risk assessments throughout the entity. While more than two-thirds of the satisfied group applies a single set of risk classifications or categories across all risk assessments, the same proportion of the dissatisfied population do not do so.
While more than two-thirds of those who find value use a single set of risk vocabulary terms, the same proportion of the dissatisfied population does not have a common vocabulary.
Similar numbers fail to analyze opportunities as part of the assessment process or to reconcile findings at an organizational level. Additionally, nearly one-third fail to cover all geographies in which they operate, address a broad range of parameters or establish assessment criteria on multiple impact parameters.
As a result, nearly two-thirds of the satisfied population believes it can aggregate or consolidate risk assessments with a high degree of confidence, while barely more than one in ten of the dissatisfied group can do so. The reasons given for failure to use standard approaches offer some insight into the problem. By far, the most common reasons cited are the failure of the entity to have established a common approach, the absence of consistent rating and evaluation criteria and the lack of technology.
This siloed approach to risk assessment is common in many organizations. Business leaders want an assessment that is relevant to their business. Although there is a natural linkage from the organization’s enterprise objectives to those of the siloed business, there is not always central ownership to aggregate and interpret the results. But this leads to unintended consequences that lessen the value of the risk assessments and make it virtually impossible for the board and senior executives to know what the relative risks really are.
It’s in the Details
The study reveals a significant lack of attention to details that would enhance the value of risk assessments in both groups, but with a greater failure (as expected) in the dissatisfied population. For example, when evaluating impact, three-quarters of the satisfied group say their criteria is consistent with the metrics used to measure performance and achievement of business objectives.
Also, overall, the dissatisfied group report a much poorer job of tracking key indicator metrics , such as the number of key assets , business operations or locations, and more than a third never do so.
 |  |
The same difference arises when the study looks at weaknesses in the risk control universe. A third of the dissatisfied population never measures the number of unique identified risks they face. Even the correctness of the measurement by those who do is suspect, since half say they don’t know how to do so and another third say they know how but cannot accomplish the task. Nearly one-third of the dissatisfied group (three times the amount in the satisfied group) also say that they never measure the number of unique general controls that had to be redesigned due to identified design weaknes ses and one-quarter (five times the number in the satisfied group) never measure the number that were redesigned due to operating weaknesses.
Satisfaction Does not Indicate Success
When we look beyond the satisfaction level, we see that even those who say they are satisfied may be lacking in completeness, and therefore reliability, of their risk assessments. In many ways, although they feel more confident about their efforts than the majority, this group may still be blind to the less apparent risks they face and opportunities for improvement.
About two-thirds of those who are satisfied with their risk assessments fail to take some critical steps to improve their processes. Like their less satisfied peers, only a few of them use all available sources to identify events that introduce risk. They pay little attention to analyst reports, industry profiles, competitor public filings, debt ratings or other valuable sources of information.
They also are not using their detected risks to validate the accuracy of their impact and residual risk analyses or to refine their identification methodologies. Barely more than half monitor whether their controls mitigate the risk events as projected. They also largely fail to consider their explicit risk tolerance, or the ways they might reduce likelihood or impact of an event by shifting risk through various means. Perhaps most important, they heavily depend on consideration of likelihood and impact of risks arising, over many other important considerations.
Assess Does not Mean Guess
Perhaps the most apparent failing of a vast majority of participants is the dependence on qualitative analysis above all else. While those who value their assessment process are nearly four times more likely to use quantitative techniques like net present value and statistical modeling, most study participants primarily rely on qualitative analysis in their evaluation of likelihood, probability and most other factors. They only rarely use other evaluative means or modeling of any type. This indicates that their r i sk analyses resul t s may well be little more than a guess or a hunch based on anecdotal information, so-called managerial intuition and limited direct information.
Less than half of those who find insufficient value in their risk assessments can say that their assessments establish accurate data and results. Only one in three finds that the results are effective and properly align activities with objectives in light of risk tolerance. Fully half cannot say that their risk assessments are even valid. By contrast, virtually none of those who find value in their assessments says they are inaccurate, ineffective or invalid. The common thread in this group seems to be that they develop and implement a clear set of processes that link risk to organizational and business objectives and they use a consistent set of metrics to articulate the variability of risk.
