AI and the State: Policy and Practice in Government
Detailed Summary
- Moderator (Christine Custis) welcomed the audience, introduced the panelists, and presented the central framing question: “Governments are both AI regulators and AI users. From where you sit—policy, research, entrepreneurship—what is the most important consideration that governments miss when they deploy AI systems?”
2. Panelists Respond to the Framing Question
2.1. Jaan Tallinn – The Host‑Versus‑Regulator Conflict
- Key point – Nations that host leading frontier‑AI firms (e.g., the U.S., EU members) experience an intensified conflict of interest: they rely on private labs for economic and security competitiveness while simultaneously being expected to police those same labs.
- Export‑control incoherence – Cited the U.S. debate over semiconductor exports as a concrete example of policy wavering between security restrictions and industry pressure.
- Sovereign‑AI investment gap – Highlighted that many emerging economies (e.g., India) focus on building data centres and in‑house models but invest insufficiently in privacy, security, and responsible‑use infrastructure (test‑beds, evaluation pipelines).
- Methodology call‑out – Argues that generative AI requires new evaluation frameworks distinct from classic statistical‑ML metrics because outputs are probabilistic and can “hallucinate.” Calls for government‑funded research to develop and openly publish such methodologies.
2.2. Rumman Chowdhury – Accountability Chains for Autonomous Agents
- Regulatory “dog‑fooding” – Observes governments often use their own AI systems to test the regulatory frameworks they impose on the private sector, raising stakes when failures affect citizens who cannot opt‑out.
- Agents & accountability – Discussed a recent paper on real‑time detection and monitoring of AI agents. Emphasised the need for traceable documentation (a “chain of accountability”) to satisfy civil‑service governance traditions.
- Governance questions – Raised fundamental concerns:
- How to monitor autonomous agents that reduce human oversight?
- How to embed step‑by‑step documentation that can be audited?
- Whether agents undermine traditional accountability structures.
2.3. Gaia Marcus – Lessons from Six Years of UK Public‑Sector AI
- Evidence base – Presented findings from the “Learn Fast, Build Things” report that reviewed 32 UK government AI/Data initiatives (2017‑2023).
- Terminology chaos – Lack of clear, shared vocabularies hampers measurement and public‑sector oversight.
- Case study: transcription tools for social workers – Highlighted that while staff appreciated time‑saving benefits, hallucinations introduced factual errors, illustrating the need for human‑in‑the‑loop safeguards and systematic evaluation of error rates.
- Hype vs. reality – Noted that many AI pilots are launched under “doomer” accusations of slowdown; policymakers often perceive evaluation as hindering innovation, creating a “speed‑over‑safety” culture.
2.4. Prof. Alondra Nelson – From Law to an AI Bill of Rights
- Regulatory spectrum – Cited the EU AI Act, UK AI legislation, and the U.S. AI Safety Bill of Rights (a public‑deliberation product distilled into five core principles such as “systems must be safe and effective”).
- Political landscape – Pointed out that 50‑70 % of U.S. respondents express strong negative sentiment toward AI, indicating bipartisan pressure for robust policy.
- AI Bill of Rights – implementation – Described how the Biden Administration translated deliberated principles into regulatory guidance, agency‑level standards, and public‑education campaigns.
- Energy & data‑centre implications – Warned that AI’s physical infrastructure (data‑centre siting, energy consumption) is becoming a politically salient issue, providing citizens with concrete levers to demand responsible deployment.
2.5. Stephanie Ifayemi – Documentation, Interoperability, and International Law
- Documentation stack – Summarized a recent comparative analysis of eight international policy frameworks (NIST RMF, EU AI Act, G‑20 AI commitments, etc.). Found convergence on three artefacts: technical documentation, incident‑reporting, and a “code of conduct” for developers.
- Government‑as‑regulator vs. government‑as‑operator gap – Questioned whether states apply the same documentation standards to their own systems that they demand from industry.
- International law for AI agents – Reported a September paper that maps existing treaties (e.g., the UN Charter, arms‑control regimes) onto AI‑agent scenarios, identifying gaps such as trans‑boundary harms and non‑interference in election processes.
- Future‑proofing – Argues that soft‑law mechanisms (norm‑building, standards bodies, multilateral agreements) must be paired with hard‑law enforcement to avoid a “regulatory whack‑a‑mole” approach.
3. Lightning‑Round: One Thing Governments Must Get Right in the Next Five Years
Each panelist was given 60 seconds to name a single priority.
| Speaker | “One Thing” (5‑year priority) |
|---|---|
| Jaan Tallinn | Pressure frontier labs to be transparent about recursive self‑improvement pathways; mitigate the risk of uncontrolled model “autonomy.” |
| Rumman Chowdhury | Build a robust, independent evaluation ecosystem that can measure bias, manipulation, privacy, and safety across sectors. |
| Gaia Marcus | Establish mandatory, enforceable standards (not voluntary commitments) at the international level, with clear compliance mechanisms. |
| Alondra Nelson | Move the Overton window so that nation‑states become models of best practice for AI governance rather than the “worst‑case” example. |
| Stephanie Ifayemi | Create a calibrated‑trust assurance toolkit (documentation of model limitations, risk‑labels, user‑facing disclosures) analogous to medication leaflets. |
| Christine Custis (moderator) | Align national AI strategies with a cohesive, interoperable stack of principles → standards → enforcement, revisited annually. |
4. Audience Q&A
4.1. From Voluntary to Mandatory – Enforceable International Standards
- Question – How can the community shift from voluntary commitments to mandatory, enforceable international AI standards?
- Responses – Stephanie noted the UN‑backed AI governance stack (now 13 levels) moving toward binding benchmarks; Alondra cited the AI Bill of Rights experience as a template for multi‑stakeholder consensus that can be legislated domestically.
4.2. Co‑creating Quality Outcomes with Youth
- Question – How can governments ensure AI tools co‑create value for young people while guaranteeing reliability?
- Responses – Gaia highlighted the “Grown‑Up” report (Nuffield Foundation) showing youth demand for transparent, safe tools; Rumman stressed the need for human‑in‑the‑loop checks and continuous monitoring of hallucination rates.
4.3. Leveraging International Levers for Accountability
- Question – What levers can countries use to hold each other accountable for AI commitments?
- Responses – Alondra outlined export‑control regimes, trade‑policy tools, and diplomatic pressure; Stephanie added that technical standards (ISO, IEC) provide a market‑driven accountability mechanism that works across jurisdictions.
4.4. Democratic Values vs. Hegemonic AI Deployments
- Question – When powerful states or labs ignore democratic norms, how can democratic countries respond?
- Responses – Rumman warned of a “race to the bottom” if governments blindly follow private‑sector rollout; Jaan emphasized the necessity of transparent red‑team exercises (e.g., DEF CON AI‑HACK Challenge, NIST Project ARIA) to surface hidden risks.
4.5. Soft vs. Hard Regulation & the Role of Standards
- Question – Are soft‑law mechanisms sufficient, or must hard regulation be accelerated?
- Responses – Consensus that both are required: soft mechanisms (norm‑building, standards) create the pre‑conditions for hard law; hard law provides the enforceable backbone needed to avoid “whack‑a‑mole” policy.
4.6. Closing Remarks
- Moderator thanked the panel and reminded the audience of a follow‑up networking session (room 4A, 10:30 am) for deeper discussion on enforcement toolkits.
Key Takeaways
- Dual Role Conflict – Governments must reconcile being regulators with being large‑scale AI users; any regulatory gap is amplified because citizens cannot opt out of state services.
- Evaluation Gap – Existing ML metrics are inadequate for generative AI; governments should fund and publish new, probabilistic evaluation methodologies.
- Accountability Chains – Deploying autonomous agents demands step‑by‑step documentation that links model decisions to human oversight, preserving traditional civil‑service accountability.
- Terminology & Documentation – A common vocabularies and interoperable documentation standards (technical docs, incident reports, codes of conduct) are essential for cross‑jurisdictional oversight.
- Red‑Team & Independent Testing – Large‑scale red‑team exercises (DEF CON AI‑HACK, NIST Project ARIA) reveal hidden biases, privacy breaches, and safety flaws; an independent evaluator market is needed.
- International Law & Standards – Existing treaties do not cover AI agents; new soft‑law norms and technical standards (ISO/IEC) can provide a baseline for trans‑border accountability.
- Public Sentiment Drives Policy – In the U.S., 50‑70 % of respondents view AI negatively, creating bipartisan pressure for AI Bill of Rights‑style legislation.
- Strategic “One‑Thing” Priorities – Panelists converged on the need for transparent, enforceable standards, independent evaluation ecosystems, and governmental leadership that models best practices rather than the worst.
- Overton Window Shift – The aspirational goal is for nation‑states to become exemplars of safe, trustworthy AI, steering the global market away from a “race to the bottom.”
- Holistic Governance Stack – Successful AI governance requires a layered stack (principles → standards → enforcement → monitoring) that is regularly refreshed and internationally aligned.
Prepared by the AI Conference Summarisation Team.
See Also: