OmouAI: Enhancing Policy Discussions with Simulated Personas

Paper: OmouAI: Argumentative Human-AI Policy Deliberation with Simulated Personas

Authors: Stylianos Loukas Vasileiou, Antonio Rago, William Yeoh, Georgina Curto

Executive Summary

Recent research highlights the promise of using large language models (LLMs) in decision-making processes, particularly in public policy discussions. However, these interactions can face challenges, such as excessive agreement (sycophancy) and inadequate explanations of decisions. Addressing these concerns, the researchers have developed a system called OmouAI. This system integrates LLMs with computational argumentation, a method that enhances the representation and analysis of debates.

OmouAI is designed to facilitate discussions around policy claims while simulating various perspectives. Users can interact with simulated personas that represent different stakeholders, domain experts, or even contrarian views, often referred to as "devil's advocates." This approach aims to reduce sycophantic behavior by encouraging diverse viewpoints. Each persona generates distinct arguments that contribute to a collective framework for discussion, allowing users to contest, modify, and build upon these arguments. This process ensures that human participants retain significant oversight over the deliberation.

One of the key features of OmouAI is its ability to evaluate arguments against predetermined external goals, such as those outlined in the UN Sustainable Development Goals. By employing deterministic argumentative semantics, the system assesses how the proposed arguments advance or hinder these goals. This evaluation not only supports more informed decision-making but also guarantees that explanations for policy recommendations are grounded in established criteria.

The research demonstrates that such a system could enhance the quality of deliberations in high-stakes environments by providing a structured way to analyze and discuss complex issues. It suggests that integrating LLMs with rigorous argumentation methods could lead to more balanced and transparent policy discussions. Although the findings derive from simulations rather than real-world applications, they point towards a future where technology aids in more effective public discourse and decision-making.

Organizations that engage in policy formulation may find value in exploring tools like OmouAI to facilitate more inclusive and effective discussions. By utilizing a system that encourages diverse perspectives and provides clear rationale for decisions, entities involved in public policy could improve outcomes and align their recommendations with broader societal goals.

Academic Abstract

Debates amongst agents driven by large language models (LLMs) have demonstrated vast potential in various applications, but when these interactions include humans and take place in high-stakes environments, e.g., in public policy deliberations, they are beset with issues such as sycophancy and a lack of faithful explanations. To tackle these issues, we present OmouAI, an interactive and inclusive deliberation system that uses LLMs in combination with computational argumentation, a field which excels in representing and reasoning within debates. OmouAI allows a human user to deliberate policy claims for real-world challenges with simulated personas, e.g., representing stakeholders, domain experts or devil's advocates, towards reducing sycophancy. Each persona generates its own arguments, and the arguments of all parties form a shared argumentation framework. Users can then contest, add and revise arguments, providing crucial human oversight. Then, arguments are evaluated using deterministic argumentative semantics against external goals, such as the UN Sustainable Development Goals, guaranteeing faithful explanations. The advancement or worsening of the goals thus serve as indicators for the policy recommendations.