Applicable legal frameworks
International
Measure 2.5, 2.7
Voluntary AI risk management framework structured around four functions: Govern, Map, Measure, Manage. A common reference in AI governance.
A.6.2.4
Certifiable standard describing the requirements for establishing an AI management system. Relevant for voluntary certification processes.
UE
Article 15 (exactitude, robustesse)
European regulation establishing a harmonized framework for AI, based on a risk-based approach (unacceptable, high, limited, minimal risk). Relevant for Quebec organizations doing business in the EU.
Quebec sector examples
Transport
Un système d'aide à la conduite hivernale entraîné principalement sur des conditions sèches échoue sur les routes québécoises lors d'épisodes de verglas.
Banque et assurance
Un modèle de détection de fraude perd brutalement en performance après un changement saisonnier dans les habitudes de consommation, sans alerte automatique.
Recommended mitigations
- 2.3Model Safety Engineering
Technical methods and safeguards that frame model behaviors and protect them against exploitation and vulnerabilities.
- 3.1Testing and Audits
Systematic internal and external evaluations that examine AI systems, infrastructure, and compliance processes to identify risks, verify safety, and ensure performance meets standards.
- 3.4Phased Deployment
Implementation protocols that deploy AI systems in stages, requiring safety validation before expanding user access or capabilities.
- 3.5Post-Deployment Monitoring
Processes for continuous monitoring of AI behavior, user interactions, and societal impacts after deployment to detect misuse, emerging dangerous capabilities, and harmful effects.
- 4.1System Documentation
Comprehensive documentation protocols that record technical specifications, intended uses, capabilities, and limitations of AI systems to enable informed assessment and governance.
Documented risks (126)
Entries from the AI Risk Repository (MIT) classified under this subdomain. Original content in English.
126 entries
01.02.00Type 2: Bigger than expected
Harm can result from AI that was not expected to have a large impact at all, such as a lab leak, a surprisingly addictive open-source product, or an unexpected repurposing of a research prototype.
01.03.00Type 3: Worse than expected
AI intended to have a large societal impact can turn out harmful by mistake, such as a popular product that creates problems and partially solves them only for its users.
04.03.00Ethics and Morality Issues
LMs need to pay more attention to universally accepted societal values at the level of ethics and morality, including the judgement of right and wrong, and its relationship with social norms and laws.
06.01.00Incompetence
"This means the AI simply failing in its job. The consequences can vary from unintentional death (a car crash) to an unjust rejection of a loan or job application."
07.02.00Accidents
"Accidents include unintended failure modes that, in principle, could be considered the fault of the system or the developer"
08.04.00AGIs with poor ethics, morals and values
"The risks associated with an AGI without human morals and ethics, with the wrong morals, without the capability of moral reasoning, judgement"
09.01.01Unethical decision making
"If, for example, an agent was programmed to operate war machinery in the service of its country, it would need to make ethical decisions regarding the termination of human life. This capacity to make non-trivial ethical or moral judgments concerning people may pose issues for Human Rights."
09.02.04Safety
"Are AI safe with respect to human life and property? Will their use create unintended or intended safety issues?"
09.02.05Law abiding
"We find literature that proposes [38] that early artificial intelligence should be built to be safe and lawabiding, and that later artificial intelligence (that which surpasses our own intelligence) must then respect the property and personal rights afforded to humans."
09.06.02Human-like immoral decisions
"If we design our machines to match human levels of ethical decision-making, such machines would then proceed to take some immoral actions (since we humans have had occasion to take immoral actions ourselves)."
12.07.00Performance & Robustness
"The AI system's ability to fulfill its intended purpose and its resilience to perturbations, and unusual or adverse inputs. Failures of performance are fundamental to the AI system's correct functioning. Failures of robustness can lead to severe consequences."
14.04.00Complexity of the Intended Task and Usage Environment
"As a general rule, more complex environments can quickly lead to situations that had not been considered in the design phase of the AI system. Therefore, complex environments can introduce risks with respect to the reliability and safety of an AI system"
14.07.00System Hardware
""Faults in the hardware can violate the correct execution of any algorithm by violating its control flow. Hardware faults can also cause memory-based errors and interfere with data inputs, such as sensor signals, thereby causing erroneous results, or they can violate the results in a direct way through damaged outputs."
15.01.02Misapplication
This is the risk posed by an ideal system if used for a purpose/in a manner unintended by its creators. In many situations, negative consequences arise when the system is not used in the way or for the purpose it was intended.
15.01.03Algorithm
"This is the risk of the ML algorithm, model architecture, optimization technique, or other aspects of the training process being unsuitable for the intended application.Since these are key decisions that influence the final ML system, we capture their associated risks separately from design risks, even though they are part of the design process"
15.01.05Robustness
"This is the risk of the system failing or being unable to recover upon encountering invalid, noisy, or out-of-distribution (OOD) inputs."
15.01.06Design
"This is the risk of system failure due to system design choices or errors."
15.02.01Safety
This is the risk of direct or indirect physical or psychological injury resulting from interaction with the ML system.
19.01.06Immaturity of AI technology can cause incorrect decisions
19.05.01AI sets rules without ethical basis
Evaluate this risk for your use case
Our risk evaluation wizard is coming soon.