Introduction to Model Risk Management

Publication 1 in our Model Risk Management Thought Leadership Series

Welcome to our Thought Leadership series where we explore the world of Model Risk Management. Throughout this series we will break down what Model Risk Management really is, taking a look from a broad industry perspective, and then through the lens of insurance to understand its growing importance within the industry.

We will provide a deep dive into model validation and share our insights on what good practice looks like. We’ll also tackle the exciting realm of Artificial Intelligence (AI), examining how to manage the additional model risks AI can present and how AI can help reduce model risk. We hope you find the series insightful.

In this first publication of a four-part series, we will explore Model Risk Management from a broad industry perspective.

What is a model?

When you think of models, big financial companies such as banks, investment houses and insurance companies come to mind, but in today’s world models are being used across all industries.

A model is any tool which takes inputs and processes these to ultimately produce an output result to assist with the decision-making process within an organisation.

With the introduction of AI and Machine Learning (ML), the number of models being used is increasing and so too is their complexity.

Where does model risk arise?

Model risk arises from decisions made based on incorrect, misused, or misinterpreted model outputs. A simple error can lead to a model failing which can lead to significant financial or reputational damage to your company. Irrespective of the sector in which a model is being used, incorrect, misused, or misinterpreted model outputs increase the risk of poor business decisions.

As models become more widespread across different areas of businesses, the exposure to model risk is heightened.

Some examples of model errors that caught the headlines:

An NHS spreadsheet error meant that close contacts of 16,000 Covid-19 patients were not traced in a timely manner.
The London Olympics oversold Olympic event tickets by 10,000 tickets due to a typo in Excel.
A private equity fund in the US was sanctioned for failing to accurately discount management fees.

The increased reliance on models to support business decisions and the connectivity of models used within an organisation highlights the need for companies to implement a robust model risk management framework.

How can model risk management help?

A model risk management programme provides the framework for managing model risk and gives model stakeholders an understanding of the weaknesses and limitations of the models used within the business. Strong model risk management reduces the risk of model errors and gives your organisation more confidence in your models.

For a model risk management framework to be effective, it must be consistent with the culture, strategy and risk appetite of your business. Developing effective governance, clear and up to date policies and procedures is crucial to building this robust framework. Having a controlled environment where model risk is understood and managed will benefit your business, whether model practices within your industry is regulated or not.

There has been increased regulatory focus on robust model risk management in recent years. In the UK, the Prudential Regulation Authority (PRA) has developed requirements for an effective Model Risk Management framework for banks, which came into effect on 17th May 2024.

Also, with the incoming introduction of the EU AI Act, the focus from regulators will be heightened by the developments in the AI field, where emphasis on building security, trust, and compliance across these systems is being introduced.

How can you implement an effective model risk management framework?

The key components of a robust Model Risk Management framework typically include:

Model inventory

The first step in creating a comprehensive and complete model inventory is setting a firmwide definition of a model’s materiality. Usually, the models with the largest financial impact come to mind first but there are other important areas to consider such as regulatory impact, customer and operational impacts and those models that feed into decision making.

The inventory should capture both internal and vendor models, and span across all departments. The materiality of a model can be measured using the model risks such as the potential of financial misstatement, reputational damage, customer impact or regulatory fine. Sometimes the simplest model can carry a lot of risk and often because these are viewed as simple models, they can get overlooked.

The exercise of creating a model inventory requires each model to be allocated a model owner and provides a starting point to embedding a strong model governance culture within the company.

Model documentation

The next step involves implementing a consistent model documentation process for the models within an organisation. This should clearly outline the standards required by the organisation when creating defined model processes, approval for model implementation and proper application of models across the organisation.

The term ‘Model Documentation’ is used a lot and can have varying definitions – it can mean anything from detailed process notes to higher level user guides. It is important to set clear guidance on the level of documentation required and who the readers of the model documents will be.

This ensures accountability around the effective execution of the model risk management programme including support from the board and senior management.

Model review cycle

The success of a business will be heavily influenced by the quality and integrity of the models used. The model review process is a vital step to ensure the models remain appropriate. Reviews on a model can vary depending on the type of model and what stage of the lifecycle a model is at.

Typically, a model should undergo a series of checks by the user every time it is run and then be reviewed by a senior member. Independent checks or full model validation exercises are crucial for quality and integrity purposes, and these checks can be completed less frequently, and should ensure that there is no ‘model drift’ occurring.

Model risk assessments should be carried out to evaluate inherent and aggregate risk presented by a model. These assessments should cover the quality of model inputs, the effectiveness of the validation process, and provide ongoing challenge and review of models through quantitative and qualitative assessments.

Model development & enhancement

Models used by a business will regularly be enhanced to reflect the economic outlook and changes in market conditions. Updates to a model should trigger a set of model change checks and any changes in methodology must be brought through the correct governance channels.

The materiality and sensitivity of a model should determine the frequency and extent of testing that should be carried out when developing and enhancing models. The roles and responsibilities should be clearly defined and evidenced – particularly around approval of changes to models and where judgements are being made.

Common pitfalls

The common pitfalls found in the model risk management area include:

Outdated model design, theories, and logic to support the model’s purpose and intended use.
Inadequate controls around data quality for its appropriateness to model the subject matter.
Poor model documentation of approximations, assumptions and expert judgements.
Obsolete test plans and output validation frameworks used to assess the model’s expected performance prior to approving a model for use.

Conclusion

Demonstrating not only the validity of models but also the effectiveness of the controls covering the design, development, revision, and use of models is of paramount importance to an organisation.

Establishing a comprehensive, robust and fully embedded Model Risk Management framework can help demonstrate this and mitigate the model risks presented.

How KPMG can help

KPMG has a successful track record of providing a broad range of financial and strategic advisory services to clients across a wide array of industries related to model risk management.

We have developed KPMG’s Model Risk Management approach which can help you create a well-controlled, integrated, and comprehensive Model Risk Management programme and offers a practical framework for identifying, quantifying, and mitigating model risk by addressing the sources of risk head-on.

Depending on your specific needs, KPMG can assist with any combination of the components of a successful Model Risk Management programme including: