Model risk management

MRM is a structured, iterative approach to identifying, assessing, mitigating, and monitoring the risks associated with the use of models. It involves a set of practices designed to ensure models are developed, implemented, and used appropriately, especially in relation to their potential impact on decision-making processes. KPMG outlines a model risk management lifecycle that encompasses six key stages:

1. Model Planning: This initial phase involves defining the scope, objectives, and design of the model. It includes identifying the decisions the model will inform and the potential risks associated with its application, laying the groundwork for a risk-aware development process.
2. Model Development: In this stage, models are built according to the specifications established in the planning phase. This includes selecting appropriate data, methodologies, and algorithms. Documentation of the development process is an expected output from this stage to facilitate both transparency to independent parties and subsequent validation efforts.
3. Model Validation: The process that evaluates a model's theoretical framework, data input, output, and underlying assumptions to confirm its appropriateness for the intended purpose(s). It involves both quantitative and qualitative assessments to identify any issues in the model's design, data quality, or operational functionality. Validation is an ongoing necessity, extending beyond initial deployment to address any changes in model use, data, or surrounding circumstances that could affect performance.
4. Model Implementation: Once validated, the model is deployed within its operational environment. This stage requires integration into existing systems and processes, ensuring that the model operates as intended in real-world conditions.
5. Model Use and Monitoring: Models should be monitored on an ongoing basis to ensure that they continue to perform as expected over time. This includes tracking the model's output, assessing its impact on decision-making, and identifying any drift in performance or changes in the underlying assumptions that may necessitate adjustments.
6. Model Changes and Adjustments: The lifecycle is iterative, and models may require adjustments in response to performance issues, changes in the operational environment, or shifts in the model's objectives. This stage involves reviewing and implementing necessary changes to the model, followed by re-validation to ensure continued accuracy and reliability.

• Governance Structure: A well-defined governance framework ensures accountability and facilitates coordinated action in managing model risks. This structure should define the roles and responsibilities of individuals involved in the model's lifecycle, from development and validation to deployment and ongoing monitoring.
• Model Inventory: Keeping a detailed inventory of all models, including their development history, purpose, relatively risk, etc., serves as the foundation for managing model risk. The inventory serves as the central system of record, enabling tracking of governance adherence, model performance, and the firm’s outstanding model risk.
• Validation and Testing Protocols: Independent validation and testing protocols are at the heart of a model risk management program. These protocols should include components such as performance, stability, sensitivity, and explainability testing. The goal is to ensure that models are robust, perform as expected under a wide range of conditions, and do not introduce unintended biases.
• Documentation and Transparency: Detailed documentation for each model, including its design, development processes, and performance metrics, supports transparency and independent model validation.
• Ongoing Monitoring and Review: Models must be regularly reviewed and monitored for performance against real-world outcomes. This ongoing process helps identify when models deviate from expected behaviors, allowing for timely adjustments or model recalibration to address emerging risks.