Trusted AI & Systems

With the core principles as a base, federal agencies will continue to apply existing and new guidance, regulations, and frameworks toward managing the risks related to AI and systems. Multiple public-private initiatives are underway to inform (through information sharing, testing, transparency) the understanding of, and promote innovation in, AI model development and related regulatory guardrails. Related state activity is also gaining momentum at both the legislative and regulatory levels.

The focus on risk management will cover the full AI lifecycle and include:

Frameworks

Cross-agency evaluation of risk management practices under:

• Existing laws, regulations and frameworks (e.g., consumer and employee protection laws and regulations, model and third-party risk management guidance).
• New, evolving, and anticipated frameworks, standards, and regulations (e.g., NIST’s AI and Cybersecurity Risk Management Frameworks, ISO 42001, application of TCPA to AI-generated voice).

Governance

Cross-agency focus on robust, and effective governance practices, including:

• Understanding the inter-relationships among the core principles, changing societal dynamics and human behaviors, and AI risks.
• Implementing practices/parameters for development, implementation, and use (e.g., clear statement of purpose; sound design, theory, and logic).
• Testing and validation of systems and risks, including third parties.
• Promoting transparency (e.g., what data is used, how data is used, impact assessments) and accountability (e.g., claims, ethical application).

Purpose Limitation & Data Minimization

Driven by the proliferation of available consumer data, the volume of data needed to train AI models and systems, and the increasing number of applications of AI and systems, regulatory attentions and enforcement will focus on:

• Compliance with data privacy laws and regulations, including protections against disclosure of sensitive data including biometric, health, location, and personally identifiable information.
• Responsiveness to consumer data requests (e.g., corrections/revisions, consent, opt in/out, authorization(s), deletion).
• Protections against bias, including data enrichment, as well as protections against adverse threats such as cybersecurity breaches, data poisoning, and misuse of the model/data.
• Limitations, including collection, access, and use as well as permission(s), consent, opt in/out, and/or authorization.
• Retention, safeguards, and disposal practices (e.g., disposal of devices/ assets containing customer data).

Continual Improvement

Regulators will expect companies to demonstrate continual improvement of the risk governance/management/controls framework. Better practices are expected to evolve based on public/private information sharing (within and across organizations as well as across regulators) especially in areas such as risk management, decision making processes, responsibilities, common pitfalls, and TEVV (testing, evaluation, validation, verification).

• Establish and maintain a governance framework: Implement tools and technology to support and operationalize a scalable governance framework that guides the design, use, and deployment of automated systems ensuring adherence to ethical standards, regulatory requirements, and best practices.
• Conduct pre-deployment testing and ongoing monitoring: Perform thorough pre-deployment testing, risk identification, and mitigation for automated systems to ensure their safety and effectiveness. Conduct runs in parallel with existing processes and have demonstrable uplift from a regulatory perspective (e.g., decrease in false positives) before full deployment. Stay up to date on regulatory developments; implement continuous monitoring and evaluation practices to identify potential issues, biases, and undesirable outcomes in a system’s performance; and adjust accordingly.
• Promote transparency and accountability: Foster a culture of transparency and accountability within the organization, clearly communicating the goals, functionality, and potential impacts of automated systems to both internal and external stakeholders.
• Implement effective MRM: Adopt a robust MRM framework to ensure models are reliable, accurate, and unbiased. Conduct regular validation, testing, and monitoring of the models, and timely address any identified issues to minimize adverse impact on investors and comply with regulations. Provide transparency regarding model performance and risk exposure to the board and management.
• Provide human alternatives and remediation: Offer human alternatives and fallback options for customers who wish to opt out from using automated systems, where appropriate. Establish mechanisms for customers to report errors, contest unfavorable decisions, and request remediation, demonstrating the organization’s commitment to fairness and responsible use of technology.
• Understand system strategy and roadmap: Align the organization’s vision, strategy, and operating model for system solutions with their broader goals. Assess the board-level oversight and maintain an inventory of the system landscape within your organization. Monitor third-party risks associated with data protection, storage, and access to confidential data, and evaluate software tools acquired to maintain data security and privacy.
• Adapt to the speed of AI development: Adopt a dynamic approach to new applications of existing laws/regulations, evolving standards, and new requirements by implementing streamlined processes for development, testing, and validation; robust training programs; arrangements to leverage external third-party expertise and technology; tailor strategies to meet unique demands and regulatory requirements across industries and geographies.