Are you ready for Guideline E-23?

In September 2017, the Office of the Superintendent of Financial Institutions (OSFI) Guideline E-23 Enterprise-Wide Model Risk Management for Deposit-Taking Institutions came into effect. This guideline, which falls under the category of “Sound Business and Financial Practices”, sets out OSFI’s expectations regarding sound policies and practices related to enterprise-wide model risk management.

Fast forward to 2023 – the models used in financial services organizations continue to increase in complexity, relying on larger and more varied data sets as well as advanced analytics such as machine learning and artificial intelligence. Models are increasingly embedded into operations. As decision-makers place more reliance, directly or indirectly, on the outputs of models, there is a corresponding increase in model risk. In recognition of this, OSFI announced in May 2022 that it was seeking to revise Guideline E-23 to:

1. extend the guideline to other federally regulated financial institutions (FRFIs), including insurers;
2. address emerging model risks; and
3. provide clarification on how the guideline should be applied.

The proposed timeline would see final guidance published by the end of 2023, with target implementation by June 2024.

Although insurers, reinsurers, and fraternals have not been subject to Guideline E-23 so far, use of models has long been embedded in the insurance industry. After all, the business of insurance requires quantifying the impact of uncertain future events, usually relying on the specialized modeling skills and professional judgement of actuaries. Consequently, all FRFIs conducting insurance business should already have a model risk management (MRM) framework in place. However, this does not mean that these institutions should be complacent about the implications of being included in the scope of Guideline E-23. Here are some things to consider:

The definition of models set out in Section 2 of Guideline E-23 is very wide. It is easy to identify “big” models, such as those that are used for actuarial purposes, including pricing, financial reporting, risk and capital management. However, the definition of a model under Guideline E-23 includes any quantitative tool that uses inputs (data and assumptions), a processing component, and a results component. Therefore, the entity would need to consider whether the current MRM framework has captured all tools which would potentially meet the OSFI definition.

Once the universe of models in the organization has been identified, any model that could materially impact the risk profile of the entity should be within the scope of the MRM framework. This can include, for example, a spreadsheet tool that is used to perform so-called “out of model” adjustments to the output of the “big” actuarial models.

What about models that are used to support operations, such as a model used in making underwriting decisions; or to inform strategic direction, such as a spreadsheet that forecasts sales and revenue? A sound assessment of models in scope should at a minimum:

1. trace both upstream and downstream model dependencies, from external as well as management reporting,
2. Examine key processes and the tools that support them, and
3. assess how the models identified in a) and b) affect the organization’s risk profile.

Section 2 of Guideline E-23 defines certain roles within an MRM framework, and Sections 3 and 4 set out the scope, key characteristics, and components of a robust approach. It is expected that there will be amendments to these sections given OSFI’s stated desire to update the guideline for emerging model risks, as well as clarify expectations.

Guideline E-23 currently makes a distinction between expectations for institutions using approved internal models for regulatory capital purposes vs. other “standardized institutions”. There is an implicit assumption that this distinction will also capture “large and complex” vs. “smaller and simpler”. However, given the extension of Guideline E-23 to insurers and other FRFIs with different regulatory capital frameworks, there would need to be more clarity on which institutions are expected to comply with all components of the guideline as opposed to approaching it on a best-efforts basis. Smaller insurers and fraternals are likely to struggle with populating the various roles in an MRM framework and may need to consider outsourcing one or more of the components (for example, independent model validation).

The MRM framework needs to cover the entire model management cycle set out in Section 5. This includes model development, model change management, and model decommissioning. A model inventory that is kept up to date is critical to an effective MRM framework. Tracking models at various stages in the model management cycle can be a challenge. The challenge increases with the number of models to be tracked and the wider the range of model users and model owners. Model change management places a lot of emphasis on vetting and validation activities – these can be very time and resource intensive, and extending the models that are now in scope can lead to capacity issues for organizations.

It is recommended that entities review their MRM framework against the roles and structure set out by OSFI. For example, it is not uncommon to see the role “model steward” which, at different entities, may be mapped to different OSFI roles, or even to a combination of roles. Another example is where the role of reviewer and approver may be combined under the current framework, whereas OSFI would prefer that they are separate. If they are combined, OSFI requires that there are mechanisms in place to ensure independence and conflicts of interest are managed. These measures would need to be addressed in the governance structure and corresponding documentation of the MRM framework.

The pace of change for tools and approaches used by insurers to manage their business has picked up considerably since Guideline E-23 was first issued. Recent developments, such as IFRS 17 adoption, have also contributed to an environment where models are becoming increasingly complex and specialized. One example is the deployment of new stochastic models to meet IFRS 17 requirements related to confidence level estimation, or to model financial options and guarantees embedded within insurance products. As it may take time to build capability in new modeling skill sets, this can lead, at least in the short term, to increased reliance on third parties. The incorporation of advanced analytics, machine learning, artificial intelligence, and other complex processing techniques lead to results that are harder to validate and explain. As a result, a fresh look at how entities assess and manage model risk is needed. OSFI’s task of incorporating these developments while applying proportionality in how entities of various sizes and complexities address similar risks is not to be understated.

The model risk management landscape continues to evolve and change for insurers. The update to Guideline E-23, which is expected by year end, will extend applicability to insurers and set out OSFI’s expectations of how model risk is managed within the organization. KPMG’s advisory professionals can help insurance organizations with assessing their preparedness for compliance with the updated Guideline. Beyond compliance, we can advise on preferred practices that can drive better MRM outcomes, while managing the administrative burden. Our modeling and model risk professionals have the knowledge, skills and experience to help with components of the MRM framework using leading technologies. Organizations of all sizes should take this opportunity to revisit their MRM frameworks, approach, and model inventory so that they are well-positioned to address model risk as part of their overall risk management framework.