Registro de sistemas de IA e regra de suspensão

AI system register and suspension rule

Purpose

Members should be able to know which automated systems take part in public decisions about them, who is responsible for each system, and how a system that goes wrong is stopped. This rule creates a public register and a suspension procedure. It complements, and does not replace, prior impact assessment and individual appeal rights.

Definitions

• Automated system: software that, using statistical models, machine learning or rules, produces outputs (scores, classifications, recommendations, drafts or decisions) that are used in a decision of a body of the polity about a person, a group or the allocation of public resources.
• Responsible body: the body that decides to use the system and answers for its use.
• Role level: advisory (outputs inform a human decision), administrative (the system issues routine decisions within set rules), or autonomous (the system issues binding decisions without case-by-case human approval).

Article 1. Register

1. No automated system may be used in a decision covered by this rule unless it is entered in the public register before use.
2. Each entry states: the system's name and purpose; the responsible body and a named contact; the role level; the decisions it affects and roughly how many people a year; the supplier and model type; the categories of data used; the human oversight in place; how to ask for human review; and the date of the last evaluation.
3. Entries are updated within 30 days of any material change. Retired systems stay on the register, marked as retired, for five years.

Article 2. Suspension

1. The responsible body must suspend a system's use, and switch to the documented manual procedure, when it has reasonable grounds to believe the system is producing systematically wrong or discriminatory outputs.
2. Every registered system must have a documented manual fallback that has been tested within the previous 12 months.
3. Suspensions and their reasons are recorded on the register.

Article 3. Independent suspension authority

1. The oversight board, which is independent of every responsible body, may order suspension of any registered system on its own initiative, on a referral from the ombudsman, or on a petition of 1% of members.
2. Suspension is mandatory when any of the following is found: the error rate in a monitored sample exceeds the threshold stated in the register entry; outcomes differ between protected groups beyond the tolerance stated in the entry without a justification accepted by the board; the system is used outside the purpose registered; or the responsible body cannot produce decision logs.
3. A suspended system may resume only when the board is satisfied that the cause has been fixed. Its decision is published with reasons.

Article 4. Review of past decisions

1. When a system is suspended because of systematic error, the responsible body identifies all decisions that may have been affected since the error began and reviews them, starting with those that imposed a debt, penalty or loss of a benefit.
2. Recovery of money on the basis of affected decisions stops until the review is complete.
3. People whose decisions are changed are told in writing and compensated under the budget rule.

Article 5. Autonomous systems

1. A system at the autonomous role level may be registered only after approval by the assembly, on the basis of a published impact assessment and an independent audit.
2. Its authorisation lapses after two years unless renewed by the same procedure.
3. Every decision it issues carries a notice that it was made automatically and how to obtain review by a human within 30 days.
4. No autonomous system may decide on loss of liberty, expulsion from membership, the removal of children, or the denial of essential subsistence.