.png){kind=logo}
The number of models your organization must manage is increasing every year. Models are often registered in tools like Excel that are excellent for a particular type of task but are certainly not ideal as inventory tools to track current status, search for different criteria, or manage a variety of workflows associated with the model's life cycle.
Ad hoc requirements and report preparation for the regulators are then a complete nightmare. Moreover, model risk management framework is often incomplete or even non-existent. So how do you get the maximum value from a properly set model inventory? We believe that in order to achieve this, you need to go through the following two phases gradually.
This phase is about nothing more than creating and setting up the essential infrastructure of model governance. It's the natural and inevitable first step banks and other financial institutions must take in order to establish the model risk management framework.
What are the indicators that model governance might be something worth considering?
Dozens, hundreds, or even thousands of models managed as a static shared table in MS Excel.
No single source of truth for your basic model information with multiple “latest” versions of the documentation.
Ad hoc data collection processes frequently asking for overlapping data sets.
Poor data quality.
Confusion in the portfolio overview.
Non-existing, formal, process definition and workflow management enforcing these processes.
Very time consuming and error-prone reporting.
We believe that every model inventory should contain no less than the following features:
Enforcement of model definition by configurable business rules.
Registration of model validations in a structured format including all the deadlines, enclosed processes, and registration of validation outcomes.
Flexible workflow management system to cover the entire model lifecycle from the model idea until model decommissioning.
A single searchable place to store all relevant documentation and data.
Ability to easily configure the inventory to match your own standards and methodologies.
If the inventory is easy to use and user-friendly, then why not increase its value by identifying further possible sorts of models? You can take advantage of more than just obligatory credit models; organize other functions driving your firm’s decisions and register them as well - CRM, marketing, sales.
Define a model map to illustrate their relations - this map could be later visualized or queried.
So, you've already created the essential infrastructure of your model governance system, you're probably thinking about the next step. Great!
The goal of phase 2 is to enhance the use of your model governance by automating parts of the process. When processes are formalized and well defined, or even enforced, by the workflow engine, then many of the steps may even be done automatically!
Some examples of tasks that could be automated include:
Validation data with a predefined structure could be checked for data quality automatically after upload.
Backtests could be executed right after the data sample is uploaded and validated, the final report is evaluated for basic aspects, and published. Some of the backtest results could be processed by a decision tree with a predefined warning system sending a notification.
Validation reports could be pre-generated based on the structured database of validation findings.
...and many many more. All these automated steps might not be game-changers when considered alone, but just think about all the unnecessary time that your experts need to invest in all of them...
Risk reduction, an immediate overview of portfolio status, single source of truth, documentation, communication, workflows, validation all in one place, formalized validation process, sophisticated model risk management, etc. etc. etc.
The list of benefits of a successfully implemented model governance is long. Many organizations realize a problem that threatens their business only when it actually occurs. All these benefits would help you avoid threatening situations and bring you a competitive advantage.
Here are some of our suggestions about how to further capitalize on your model governance:
Based on the newly gathered metrics that you get after implementing the first two phases, you can work on process efficiency tracking and further optimizations.
Automated model tiering based on your own methodology would let you optimize your validation resources.
Do your validation data exist in the system already in the up-to-date, structured format? Then, for example, let the AutoML agent automatically do its job and propose an alternative model which might be even performing better. The discussion of how such alternative models might be useful even in a strictly regulatory supervised environment will be a topic for another discussion.
Model risk governance requires a proactive approach and is necessary for internal or external requirements. Successful implementation is currently bringing competitive advantage and indisputable business value to those using it.
Let us know if you have comments, or if we can support your company in building your own effective model inventory, an essential part of model risk management framework.
.png){kind=logo}