Remote Collaboration of Model Governance & Model Risk Management Teams

The current pandemic has revealed the importance of remote work. Teams that were used to collaborating together on site have been forced to switch to a remote mode. While this might be a seamless process for professions like programmers, with their setup already prepared for remote work (including VPNs, tooling, communication channels, CI/CD pipelines...), the situation may have been a bit different for professionals involved in model governance and model risk management. 

The following text explores just a few of the ways to enable remote work within these professions.

Team coordination

Yes, the almighty email will most likely still be here even in the 22nd century, however, our great-grandchildren would probably laugh a little imagining our 2020 work being totally coordinated by this useful, yet slightly ancient tool. And why should we go through so much trouble when there are much more powerful and sophisticated means of coordination?

Regulatory model validation is typically a well-defined, rigorous process with multiple stakeholders involved, so it’s only natural to introduce some kind of workflow management system in place. What benefits does a management system bring?

To have a reliable, real-time overview of the current state of work (development, validation, internal audit, MRM exercises...).

Each step in the workflow can contain a precise description of the work to be done. The expert performing the task knows exactly what to do and what result she/he is expected to provide, even without a manager standing (in the current situation unacceptable) 30 centimeters behind.

Workflows spread the methodology consistently across teams in the organization. The result is better alignment, for instance between model developers and validators. Workflows act as an information base for ad-hoc reporting, process optimization, and measuring teams' efficiency.

Ideally, such a workflow management system would be connected to a company-wide model inventory to optimize the context switching for end-users and to allow reporting (e.g. Gantt chart of validations planned for the next year).

Source code management

Good software development practices can easily solve difficulties like model source code version management and lingering doubts such as “is the version of the source code of this model about to be validated really the right one?”

An example of a good software development practice is considering model development as a variety of software development. The most useful practice for achieving this is introducing a version control system, like Git. Model developers can use it for collaboration on the source code of models. 

The introduction of such a system substantially more convenient for model validators, as model developers can send them a unique, unchangeable "Commit id" (tag) of THE FINAL VERSION (otherwise often represented in e-mails as model-source-code_final_v23_most_final_not_kidding_really_this_is_the final_v2.py), which shall be validated. Git can serve as a single source of truth for all these development artifacts, and even for managing source codes of model performance monitoring scripts.

Provisioning of virtual analytical or development environments

Teams can struggle even with having access to model development/analytical environments (like RStudio or JupyterLab). Having them installed locally might seem like a sufficient solution until one needs to perform an analysis on a very large dataset located on an external server using a rather slow home internet connection to download it. 

The solution can be granting access to this kind of environment hosted on a powerful corporate server (or in the cloud). This way, the IT resources can be shared across the team, providing computing power far exceeding common localhost when needed. It can additionally optimize costs if designed efficiently. 

Resources like datasets or blocks of code can be also shared across teams with no security and throughput struggles with data transfer.

Validation envelope

Imagine having a single envelope encapsulating one or more models that are about to undergo a validation. Such an envelope would contain all information (both qualitative and quantitative - e.g. whole datasets) needed to perform the validation. 

After the validation is started, the validation workflow would guide validators through each step according to company policies, with no need for physical interaction. After the validation is finished, validators would then upload the validation report and update the validation metadata. All stakeholders involved in validation could easily get a link to this envelope.

Let’s wrap it up

I'm convinced that the events we went through in the past several months will have a lasting and positive impact on the way we work, catalyzing changes to remote modus operandi, including model governance & model risk management teams. 

It’s true that this is something that such teams in internationally operating institutions had been forced to deal with in the past, however, this is a unique opportunity to accelerate this transformation in a way that will be beneficial to both employers and employees.