Algorithmic impact assessment

An algorithmic impact assessment (AIA) is a structured process for identifying, documenting and mitigating the likely effects of an automated decision system before it is deployed, and for reviewing them afterwards. It adapts a familiar idea from environmental and data protection law: those who propose a potentially harmful activity must study its consequences, consult those affected and publish their findings.

An AIA typically asks: what decisions the system makes or supports; who is affected and how severely; what data it uses; how accurate and biased it is; what human oversight exists; how decisions can be explained and appealed; and what mitigations are in place. The result may be a risk level that triggers specific obligations.

AIAs rest on several assumptions about how to govern algorithmic systems.

• Ex ante over ex post. Harms from automated public decisions can be large and fast, as scandals have shown, so prevention is preferable to litigation after the fact.
• Proportionality. Not every system needs the same scrutiny. A tiered approach applies stronger requirements, such as peer review, human decision-making or public consultation, to higher-impact systems.
• Accountability through documentation. Writing down reasons, data choices and risk judgements creates a record that auditors, courts, ombudsmen and the public can examine.
• Participation. The AI Now Institute's 2018 framework argued that public agencies should invite public comment and external researcher review, not only conduct internal self-assessment.

Metcalf and co-authors point out that "impacts" are not simply discovered: they are co-constructed by whoever defines the scope and methods of assessment. Selbst argues that because assessors depend on information only system developers hold, their discretion and incentives will make full compliance hard to achieve, but that AIAs remain useful because the documentation they produce supports policy learning even without full compliance.

Examples of impact-assessment regimes

• Canada. The Treasury Board's Directive on Automated Decision-Making took effect on 1 April 2019. It requires federal institutions to complete an Algorithmic Impact Assessment questionnaire, with 65 risk questions and 41 mitigation questions, before deploying an automated decision system. The score assigns one of four impact levels, from little or no impact to very high impact, each with escalating requirements such as peer review, notice, explanation and human involvement in final decisions. Departments must publish final results on the Open Government Portal.
• European Union. Under the AI Act, deployers that are bodies governed by public law, or private entities providing public services, must carry out a fundamental rights impact assessment before using high-risk AI systems, describing affected groups, risks of harm, human oversight and complaint mechanisms.
• Data protection impact assessments. GDPR Article 35 already requires an assessment for high-risk processing of personal data, including systematic profiling with significant effects, which covers many public algorithmic systems.
• Health and local government. The Ada Lovelace Institute piloted an AIA for access to NHS imaging data in the UK, and various cities have adopted AIA templates for procurement.
• Cautionary cases. Australia's Robodebt scheme, an automated debt-recovery system using income averaging, and the Dutch childcare benefits scandal both caused widespread harm. The Dutch parliamentary inquiry found that fundamental principles of the rule of law had been violated by a mass, all-or-nothing approach to fraud, and the Dutch data protection authority found the use of dual nationality in risk-selection models unlawful and discriminatory.

• Self-assessment. Most AIAs are completed by the agency deploying the system, which has an interest in a low risk score. External review is rare.
• Box-ticking. Questionnaires can become compliance rituals that produce documents without changing designs.
• Unknown unknowns. Many harms emerge in deployment, from interaction with changing data, users and institutions, and cannot be fully predicted beforehand.
• Scope gaps. Security, policing and migration systems are often excluded or subject to weaker rules.
• Weak participation. Affected communities are seldom consulted meaningfully, partly because assessments are technical and timelines short.
• Enforcement. Where non-compliance carries no consequence, AIAs remain aspirational. Early reviews of Canada's directive noted that relatively few assessments had been published compared with the number of systems in use.

Impact assessment has become the default policy tool for governing public-sector AI: Canada created the first binding government-wide regime in 2019, the EU made fundamental rights assessments mandatory for public deployers of high-risk systems, and many other jurisdictions now require some form of assessment. The approach has improved documentation and created institutional responsibility for algorithmic risk. Its weaknesses mirror those of environmental assessment: dependence on self-reporting, uneven quality and limited enforcement. AIAs work best when combined with public registers, independent audit, rights to explanation and appeal, and bodies with power to stop deployment.

• Government of Canada, Algorithmic Impact Assessment tool
• Metcalf et al. (2021), "Algorithmic Impact Assessments and Accountability: The Co-construction of Impacts", FAccT
• Regulation (EU) 2016/679 (GDPR), Article 35
• Regulation (EU) 2024/1689 (AI Act), Article 27
• Dillon Reisman et al., Algorithmic Impact Assessments: A Practical Framework for Public Agency Accountability (AI Now Institute, 2018)
• Andrew D. Selbst, "An Institutional View of Algorithmic Impact Assessments", Harvard Journal of Law & Technology 35(1) (2021)