Trustworthy AI: Balancing Innovation and Regulation

Detailed Summary

Moderator (Ankur Singh, RBI) opened the session with an “ice‑breaker” asking each panelist to name a routine AI‑enabled activity they use.

• Suvendu Pati (RBI) described using Google’s automatic photo‑collage feature on his phone to sort weekend travel pictures – a time‑saving example of AI‑driven image curation. He warned of inadvertent privacy issues when AI groups photos of strangers, stressing the need for user vigilance.

2. UNESCO’s Digital Heritage & AI Governance (Mariagrazia Squicciarini)

• AI‑generated cultures: Squicciarini recounted a recent incident where AI‑run chat bots on a closed social network created a proprietary language and jokes. She asked whether UNESCO would protect such emergent digital culture as heritage.
• UNESCO’s stance: UNESCO “always steps in” when a member state requests protection, but the heritage‑label depends on national requests.
• 2021 Recommendation: She highlighted the 2021 UNESCO AI Recommendation, noting its principle‑based, technology‑agnostic design. Rather than defining AI narrowly, the recommendation enumerates 11 policy areas (e.g., education, privacy, data, gender, environment).
• Human‑Centred Redress: The principle of human oversight (the “last word should not rest with a machine”) was emphasized as the core mechanism for redress and compensation.
• Inclusivity & Language Barriers: Squicciarini raised the challenge of AI systems mis‑recognising accented speech (her own Italian accent) and warned that linguistic exclusion can slow AI adoption and exacerbate bias.

3. AI Agents in Finance – RBI View (Suvendu Pati)

• Hypothetical loan‑seeking AI agent: When asked whether an autonomous AI could walk into the RBI for a loan, Pati answered “no – RBI does not lend.”
• Future scenario: He projected that AI agents will increasingly act on behalf of humans, but credit assessment will always target the underlying person or entity, not the agent itself.
• Risk parameters: RBI would require explainability, auditability, and accuracy of any AI model used in loan‑approval workflows, stressing a “human‑in‑the‑loop” approach.
• Back‑end vs. front‑end agents: Pati distinguished fraud‑prevention agents (backend, automated, but requiring human authorisation for flagged transactions) from customer‑facing agents (future conversational bots).

4. Trustworthy AI – Core Pillars (Panel Consensus)

Pillar	Panelist Emphasis
Transparency	Caitlin Searle (Australia) – need for clear disclosures, public reporting, and model registries.
Explainability	RBI – model explainability essential for audit and consumer trust.
Bias & Data Quality	UNESCO – incomplete data leads to biased outcomes; post‑deployment monitoring vital.
Human Oversight	UNESCO & RBI – ultimate decision‑making must rest with humans, not machines.
Accountability & Redress	UNESCO – mechanisms for compensation when AI harms occur.
Inclusivity	UNESCO – language, gender, and cultural representation must be baked into design.

5. Australia’s Ethical AI Guidance (Caitlin Searle)

• Historical context: Australia issued voluntary ethical AI standards in 2019; updated guidance (six essential practices) released in late 2023.
• Purpose: To help Australian industry stay competitive while embedding responsible AI. The guidance is voluntary and industry‑tested; a national AI plan (three pillars, including “Keeping Australians Safe”) was published shortly after.
• Future regulatory trajectory: No immediate move to make the guidance mandatory; instead, Australia is establishing an AI Safety Institute (operations slated to begin within weeks) to monitor frontier models and advise regulators.

6. UNESCO’s Practical Governance Tools (Mariagrazia Squicciarini)

• Readiness‑Assessment Methodology (RAM): A diagnostic framework that maps a country’s AI ecosystem (infrastructure, talent, legislation) to identify gaps. India’s RAM launch was announced during the panel.
• Ethical Impact Assessment (EIA): A tool for evaluating AI systems at any development stage against human‑rights criteria. 77 assessments completed to date, with several more ongoing.
• Multilateral Collaboration: UNESCO stresses inter‑linkage of regulatory tools to avoid fragmented national approaches; a “AI‑without‑borders” expert network has been created to share best practices and prevent duplicated mistakes.

7. Managing AI‑Washing & Consumer Protection (RBI & Infosys)

• Disclosure Requirement: Entities must publicly disclose AI usage (in annual reports, product documentation).
• Model Repository: Firms should maintain an internal registry of models, their purposes, and performance metrics; regulators can audit this repository.
• AI‑Supervises‑AI: The panel suggested using AI tools to monitor other AI systems (e.g., anomaly detection on model outputs). The remaining oversight, however, must stay human‑centric.

8. International Coordination (Caitlin Searle)

• Avoiding Fragmentation: Australia participates in multilateral fora (UN, G20, OECD) and pursues bilateral agreements to harmonise AI standards.
• AI Safety Institute Role: The institute will act as a national hub for testing frontier models, sharing findings internationally, and feeding evidence into global norm‑setting processes.

9. UNESCO’s Global Toolkit & Sector‑Specific Guidance (Mariagrazia Squicciarini)

• G7 Public‑Sector AI Toolkit: Provides hands‑on guidance for civil servants to translate principles into practice.
• Healthcare Example: Emphasised the need for multidisciplinary design teams (clinicians, data scientists, ethicists) to avoid bias (e.g., gender‑biased cardiovascular risk scores).
• Error‑Margin & Human Oversight: Recommended explicit thresholds for acceptable model error and mandatory human intervention pathways.

10. Closing Reflections – What Has Been Done Right & What Lies Ahead

Speaker	What’s Been Done Right (past 2 years)	What Must Be Done Next
Suvendu Pati (RBI)	Massive digitisation of banking processes, creation of alternate‑data credit models that bring thin‑file customers into the formal system.	Extend inclusion to 500 million previously unbanked Indians; embed AI to further level the financial‑service playing field.
Mariagrazia Squicciarini (UNESCO)	Raised global awareness of AI ethics and built readiness tools (RAM, EIA) that countries now adopt.	Deepen young‑people engagement, ensure AI readiness translates into equitable opportunities worldwide.
Caitlin Searle (Australia)	Conducted an extensive two‑year public/industry consultation and released the National AI Plan with clear pillars.	Move from consultation to implementation – scale AI‑safety institute activities, upskill SMEs, and drive inclusive AI adoption.

Key Takeaways

• Principle‑Based, Technology‑Agnostic Governance: UNESCO’s 2021 AI Recommendation provides a durable framework that can evolve with AI advances.
• Human Oversight is Non‑Negotiable: Across regulators, the “last word” must remain with people—not machines—to ensure redress and accountability.
• Transparency & Disclosure: Mandatory public reporting of AI use, model registries, and clear consumer‑facing disclosures are essential to combat AI‑washing.
• Inclusivity Starts with Language & Data: Accent‑aware speech models and gender‑balanced datasets prevent bias and foster broader adoption.
• Post‑Deployment Monitoring: Trustworthiness depends on continuous auditing, bias testing, and the ability to intervene when models deviate.
• Multilateral Coordination Prevents Fragmentation: Australia, UNESCO, and the RBI all stress the need for global norms, shared tooling, and cross‑border expert networks.
• AI Agents Must Remain Human‑Centred: In finance, AI agents will augment but never replace human credit assessment; a clear human‑in‑the‑loop policy is required.
• Readiness Assessment Methodology (RAM) as a Diagnostic: RAM helps nations map AI capability gaps, guiding targeted investments and policy reforms.
• Ethical Impact Assessment (EIA) Guarantees Rights Alignment: Applying EIA at any AI lifecycle stage ensures compliance with human‑rights standards.
• AI‑Supervises‑AI is Promising but Needs Human Governance: Automated monitoring can flag model drift, but ultimate authority rests with accountable human actors.

---

Prepared from the verbatim transcript of the “Trustworthy AI: Balancing Innovation and Regulation” panel held in Delhi.

See Also:

• governing-safe-and-responsible-ai-within-digital-public-infrastructure
• power-protection-and-progress-legislators-and-the-ai-era
• ai-for-democracy-reimagining-governance-in-the-age-of-intelligence
• building-sovereign-deep-tech-for-a-resilient-future-solutions-from-finland-and-india
• shaping-secure-ethical-and-accountable-ai-systems-for-a-shared-future
• ai-impact-forum-democratising-ai-resources
• scaling-trusted-ai-for-8-billion
• making-ai-all-inclusive-bridging-ai-communities-to-indias-ai-future
• ai-diffusion-from-innovation-to-population-scale-impact
• responsible-ai-at-scale-governance-integrity-and-cyber-readiness-for-a-changing-world