Reliance on business analytics is growing significantly as advanced data analytics or analytics is becoming critical for an organisation’s overall success. These analytics create business intelligence for better decision making, ensure more efficient processes and enable tailor made customer experiences.
The increased reliance on data analytics combined with increasing complexity opens companies up to risks around data reliability and accuracy, technical robustness, accountability, compliance, and ethics. It forces business leaders to consider critical questions such as: How do my analytics impact the business? Are my analytics performing in line with our criteria? Are they compliant to regulations? And to which risks are my analytics exposing me?
Understanding your analytics
Impact over complexity
When considering risks related to analytics, you should consider a solution's impact and how analytics influences business processes. Decision-making is equally relevant to determining the risk, if not more, than its technological impact. Therefore, it is essential to take a broader view and ensure that the full spectrum of analytics risks is managed. This includes complex artificial intelligence (AI), but also more common automated data processing and other analytics and algorithms.
There is a growing political and societal anxiety around companies’ use of algorithms that make decisions, or facilitate decision making. An increase in regulatory discussions demonstrates this, for example, by the Competition and Markets Authority (CMA) in the UK, and through the recently proposed AI regulation by the European Commission.
CMA paper on AI Risks
The CMA working under the UK Government mandate aims to guide businesses by setting standards, regulatory frameworks and engaging in audits of algorithmic solutions in organisations. It also foresees penalisation, disclosure requests and other more severe actions. It advises companies to keep records explaining their algorithmic systems. This means even complex algorithms need some level of explainability.
EU proposed AI regulation
In April 2021, the European Union unveiled its proposal for the regulation of AI in Europe. The goal of the regulation is to protect citizen's rights while encouraging innovation. The proposed regulation centres around the need for comprehensive risk management, requiring each AI solution to be assessed and categorised into low, medium or high risk. High risk applications are subjected to various requirements in terms of transparency and risk assessment. This encourages a compliance-by-design approach. In addition, the regulation also explicitly bans certain types of AI applications, with the possibility to extend this "blacklist" of AI applications over time.
Start with an inventory
Once you’ve identified what automated data processing, analytics and (AI) algorithms are used within your organisation, you can start using them to answer business questions. It is essential to recognise that both the complexity and the impact of a solution determine the (potential) value and the risks. Therefore, an inventory should capture the function and the context in which algorithms, models and other analytics operate. Based on the technology used, the lifecycle phase, the type of analytics, the domain of application and operational context, we can start to understand how these technologies impact the organisation.
Performing risk assessments
The core characteristics of an analytics solution and the context in which they operate provide vital information needed to manage the portfolio of solutions. It should be the starting point to assess associated risks, then implement solution-specific controls and overall organisation-wide governance.
To assess the risks associated with analytics and prioritise further measures, we can learn from the practice of Risk Tiering: a scorecard-based methodology to classify risks of financial models applied by financial institutions. In this methodology, models are scored on several axes such as (material) impact and complexity and then ranked on overall risk. Typically, financial materiality is the most important factor in a risk assessment. However, analytics can impact organisations in other ways such as regulatory, reputationally, operationally and can impact on consumers or citizens.
Third party alignment
Analytics solutions may be developed in-house, in close collaboration with third parties or bought off the shelf from other developing parties. Analytics with a significant impact on the organisation should be subject to the same level of control, just as analytics developed internally are.
With all this, it is also important to look beyond your own organisation. Developing new AI applications often requires cooperation with other parties, both concerning the retrieval of datasets used for analytics, as well as the actual procedures performed on these datasets to build new algorithms. In such cases, there will be a dependency on partners.
When entering collaborations, it is important to translate the organisation-specific context into demands on the third party. Questions to ask include:
- Who has access to the algorithms?
- To what extent can a partner ensure that the organisation's standards are met?
- What are the possibilities to obtain information about the operation of the algorithms
The assessment should focus on the risk stemming from an analytical solution in the absence of actions that the management may take to decrease risks. This helps organisations understand the risks arising from solutions as a baseline, to understand the residual risk once implemented.
Selecting mitigating measures to control algorithmic risks
As analytics are almost always embedded into your existing processes, the desired level of control should be tailored and optimised in relation to overall risk appetite. It is crucial to select controls that mitigate risks across the different stages in the analytics lifecycle, from ideation to development, consumption and retirement. There is no ‘one size fits all’ approach. Each solution requires a specific set of controls, most likely derived from a common library of mitigating measures.
How can we help?
We have designed a methodology and built a tool to detect and index key risks related to analytical solutions in use. This allows us to provide insights on the analytics used by organisations, the risks surrounding these solutions, and how they are controlled. The tool identifies inherent risks and governance maturity for algorithm-based tools, models, calculation templates, and analytical assets in an organisation. It provides an overarching comparative view, with deeper dives into each one of the tools. By understanding and having a comprehensive inventory of your organisation’s analytics will support you in managing risk and governance.
To find out more and learn about how we can support you on your risk analytics risk inventory journey, please connect with us.
