An experiment in which about 1,000 representative Americans used the Polis platform to draft principles for an AI chatbot. Anthropic trained a model on the resulting public constitution and compared it with one trained on its own; the public model was less biased on a standard benchmark and equally capable.
Constitutional AI is a method developed by Anthropic for training large language models: instead of relying only on human ratings of individual answers, the model is given a written list of principles (a "constitution") and uses it to critique and revise its own outputs and to generate preference judgements for further training. The method makes the values a model is trained on more legible, but it also highlights who writes them. In Anthropic's words, "we wrote the constitution ourselves".
In 2023 Anthropic and the Collective Intelligence Project (CIP), as part of CIP's Alignment Assemblies programme, ran an experiment to have members of the public write such a constitution. The results were published on 17 October 2023 and, as a peer-reviewed paper, at the 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT).
About 1,000 adults in the United States, recruited by the survey firm PureSpectrum to be representative by age, gender, income and geography, took part through Polis, an online platform in which participants submit short statements and vote agree, disagree or pass on statements written by others. The prompt asked them to help pick rules for an AI chatbot. Participants submitted 1,127 statements and cast 38,252 votes, an average of 34 each. Polis identified two main opinion groups.
The researchers then turned the public input into a constitution. They removed statements below a consensus threshold across groups, deduplicated and combined similar statements, and mapped the remaining 275 statements into principles in the format used for training, comparable with the 58 principles of Anthropic's own constitution. About half of the concepts overlapped. The public principles placed more weight on objectivity, impartiality and accessibility, and tended to encourage desired behaviour rather than prohibit undesired behaviour.
The project was funded by the two organisations; participants were paid by the recruiting firm. The public goods are the methodology, the published public constitution and the evaluation results, which other groups can reuse.
The researchers stressed the limits themselves. The sample was small and not globally representative. Many subjective judgement calls were made by researchers in moderating, deduplicating and translating statements into training principles, so the final constitution was co-authored by the organisers. Both models were trained on the same database of prompts even though the public constitution contained principles those prompts may not exercise. Training proved harder than expected; early versions were judged "annoying" because of an imbalance between harmlessness and helpfulness data. Critics of participatory AI add that a one-off survey cannot provide ongoing accountability, and that the company retains the decision whether to use any of the results.
Anthropic trained two models of the size of Claude Instant: a "Public" model on the public constitution and a "Standard" model on Anthropic's own. They performed equivalently on standard language-understanding and mathematics benchmarks and on helpfulness and harmlessness ratings. The Public model showed lower bias than the Standard model across nine social dimensions on the BBQ benchmark, most clearly on disability status and physical appearance, and both models showed similar political leanings. The models were research artefacts; the post does not report that the Public model was deployed. The experiment is widely cited as one of the first cases of a representative public directly shaping the training rules of a large language model.
1,127 statements and 38,252 votes produced a public constitution of principles overlapping about 50% with Anthropic's. A model trained on it matched the standard model on capability, helpfulness and harmlessness tests and showed lower bias on nine BBQ dimensions. Published in October 2023 and at FAccT 2024.
A one-off research pilot rather than a standing institution. The organisers made many judgement calls in turning statements into training principles, the sample was US-only and small, and there was no mechanism binding the developer to adopt the results. It ended as planned after publication.
About 1,000 US adults recruited by PureSpectrum submitted 1,127 statements and cast 38,252 votes on Polis; after moderation 275 statements remained, overlapping about 50% in concepts with Anthropic's 58-principle constitution.[1]
The model trained on the public constitution performed equivalently to the standard model on MMLU, GSM8K and helpfulness and harmlessness ratings, and showed lower bias across nine social dimensions of the BBQ benchmark.[1][2]
The organisers acknowledge that many subjective judgement calls in moderating and translating public statements into training principles shaped the final public constitution.[1]