The CECL Challenge: You Spoke We Listened
Challenger Models, Challenger Methodologies, and Model Validation are closely related concepts that may sometimes be considered one and the same. However, it is worth defining what each is, exploring their relative virtues, and identifying how to make the best use of these tools for optimal model performance. Based on our survey results from last week's email, we decided to expand on these concepts to help you determine the best way to manage your model risk.
Challenger Modeling is a full-blown challenge that involves comparing the champion model to an independent platform. This allows you to test the entire model: from data inputs, key assumptions, and statistical techniques to user controls, model performance, and model outputs. This dual approach also permits a sandbox environment where different methodologies and concepts can be tested without the possibility of compromising the integrity of the live production model. Unlike one-time model reviews, challenger modeling can be repeated as often as desired and provide for ad-hoc analysis. This is especially valuable as a financial institution grows, evolves, or encounters unforeseen circumstances, such as COVID 19.
Challenger Methodologies are reconfigurations to your existing champion model’s methodology to prove, disprove, or provide alternative viewpoints to current results. For example, in a CECL model, methodology challenges could be made at a structural level such as cohort groupings and loss modeling approaches; at a quantitative level such as economic forecasts and reversion techniques; or even to more subjective assumptions such as qualitative overlays. Challenge methodologies are often used during the development of your champion approach, when reacting to an unexpected event, or more periodically when planning for future changing events such as a new acquisition.
Lastly, a Model Validation or Independent Model Validation is a point-in-time review, often performed annually, intended to verify that the champion model is performing as expected and producing results in accordance with predefined model objectives and documentation. Model validations can help ensure both model and methodology performance as well as compliance with GAAP and regulatory guidelines.
Given that many financial institutions have already implemented CECL and that others will soon be implementing, verification, confirmation, and flexibility are necessary as you proceed forward and respond to auditors, regulators, growth, and the unforeseeable future. Which of these three tools is right for you? We've taken the “Good, Better, Best” approach to help you answer this question.
- Good – Model Validation satisfies auditors, regulators, management, and directors by confirming the model and methodology alignment to documented policies, controls, and expectations.
- Better – Challenging your champion methodologies by analyzing the impact of changes in specific assumptions allows you to better understand the effects of your modeling choices on model outputs.
- Best – Challenger Modeling provides an independent framework for validating a champion model, establishes a dedicated environment for testing alternate methodologies, and allows for greater flexibility and confidence throughout the growth and economic changes your financial institution will experience.