As organizations hand over more and more of their business decisions to AI, there’s a growing concern about the ethical, legal and compliance implications.

While many think of computers as accurate, neutral, unemotional and purely logical machines, the human-designed AI algorithms they run are susceptible to error and bias, especially if the data they rely on is incomplete or contains inherent biases.

Financial services companies are subject to a significant number of regulatory requirements and compliance mandates. Avoiding unintended biases with age or race in decisions about extending credit or determining credit limits, for example, is key to meeting those mandates. Investment decisions, of course, can also have significant financial consequences. Therefore, it’s essential to have confidence in any decision that AI makes by ensuring that such decisions are free from error or bias.

Understanding, detecting and preventing such errors or biases, however, can be remarkably challenging. There’s often a lack of transparency into how AI is making its decisions — indeed, one of the reasons we turn to AI is precisely because it can detect patterns amid chaos that humans are incapable of seeing or even understanding.

Responsible AI is the practice of bringing transparency, accountability and trust to AI. Our goal was to help our client stand up an effective and efficient responsible AI program, powered by the technology required to validate its AI/ML models and data to support longer term scale.

The challenge in this case was that each of the institution’s five separate business units had already implemented their own AI/ML solutions. But without any formal or centralized responsible AI program, or technology specifically designed to detect AI/ML model or data errors or biases, the client was at risk.

This included custom Python scripts, open source libraries and other tools designed for this purpose.

We worked with the client’s preferred AI/ML platform solution. This cloud-based AI workbench is designed to help data scientists and developers prepare, build, train and deploy high-quality machine learning models quickly by bringing together a broad set of purpose-built capabilities. However, by default, it’s not configured with any validation capabilities.

We helped develop the scripts for the platform that would enable the client to validate any AI or ML algorithms and the data on which they depend — essentially, “built-in” validation instead of using a separate add-on solution that typically evaluates only the output of those algorithms.

Our validation customizations were designed to help the firm quickly and easily detect, diagnose and correct problems that can arise with AI/ML models, including both internally and externally developed and trained models. It enabled our client to experiment with its models — providing “levers” they can pull to conduct what-if experiments to help determine the effect various parameters or assumptions have on the models that rely on them, and ultimately a way to quantify errors or biases in those models.

We also reverse-engineered several proprietary third-party models to help our client gain insights into those models that were otherwise a black box.