Panel Leader: Michael Morris (Columbia)
Michael Morris, Columbia University
ACE: A LLM-based Negotiation Coaching System
The growing prominence of LLMs has led to an increase in the development of AI tutoring systems. These systems are crucial in providing underrepresented populations with improved access to valuable education. One important area of education that is unavailable to many learners is strategic bargaining related to negotiation. To address this, we develop a LLM-based Assistant for Coaching nEgotiation (ACE). ACE not only serves as a negotiation partner for users but also provides them with targeted feedback for improvement. To build our system, we collect a dataset of negotiation transcripts between MBA students. These transcripts come from trained negotiators and emulate realistic bargaining scenarios. We use the dataset, along with expert consultations, to design an annotation scheme for detecting negotiation mistakes. ACE employs this scheme to identify mistakes and provide targeted feedback to users. To test the effectiveness of ACE-generated feedback, we conducted a user experiment with two consecutive trials of negotiation and found that it improves negotiation performances significantly compared to a system that doesn’t provide feedback and one which uses an alternative method of providing feedback.
Harang Ju, Massachusetts Institute of Technology
Does AI Coaching Improve Negotiation Outcomes? Evaluating Warmth, Dominance, and Individual Traits in Negotiation
Artificial intelligence (AI) tools leveraging large language models (LLMs) have enhanced task performance across domains of knowledge work. Yet, their impact on negotiation—a critical component of collaborative work—remains unclear. This pre-registered online experiment investigates how real-time AI coaching, delivered through distinct communication styles (warmth- or dominance-oriented prompts), shapes negotiation outcomes. We hypothesize that AI assistance improves both objective (e.g., deal value) and subjective (e.g., perceived fairness) outcomes, with effectiveness contingent on alignment between coaching style and individual receptivity. Participants were paired and randomized into conditions including no AI, AI with fluency-focused prompts, and AI with combined warmth or dominance strategies. During negotiations, participants received AI-generated suggestions for chat messages. The study evaluates (1) whether AI coaching enhances outcomes, (2) which coaching styles (warmth, dominance, fluency) are most effective, and (3) which individuals benefit most from specific styles. This work advances understanding of how AI coaching can enhance negotiation dynamics in socially complex contexts.
Peter Carnevale, University of Southern California
AI Knows a Deal When It Sees One
Classification of negotiated agreements is important for understanding the basic structure of agreement and for coaching people on what is possible in negotiation. An important step is answering this question: Can people, and AI, reliably recognize and classify agreements? If yes, it suggests new avenues for systematic investigation on agreement and coaching negotiators to consider possibilities (cf. Lax & Sebenius’s, 2006, notion of “deal design”). Pruitt developed a 5-category classification scheme (1981) and Carnevale (2006) expanded it to 8, with a new category called “Superordinate” that derived from Sherif’s (1958) notion of the “superordinate goal” wherein disputing parties forgo an immediate interest in lieu of a common goal, that is, a replaced interest. A Superordinate agreement is one where the parties drop their initial interests in favor of new interests that emerge. A coding manual for the 8-agreement taxonomy was developed that includes definitions and examples of agreements of each type. The OpenAI ChatGPT4o was trained on the agreement taxonomy. Can people, and AI, reliably recognize and classify agreements? Yes. As an example, see Figure 1 that shows an instance of an agreement that had not previously been shown to GPT, and GPT’s classification. It is worth noting that a large sample of human judges concurred that the agreement is Expand. The human-AI concurrence across instances of the 8 different agreement categories was remarkable. As a further test of GPT’s ability to classify agreement, it was asked to read the Mayflower Compact (Figure 2). Then it was asked to indicate which if any of the 8 agreement types describes the Mayflower Compact. The Mayflower Compact, signed in 1620, is a historic agreement that established self-governance in the Plymouth Colony. It is one of the first formal agreements made by European settlers in the Western Hemisphere. The 41 signers, having landed unexpectedly outside their intended destination, faced internal group conflict. We expected it to be difficult to classify. But GPT4o sees the Mayflower Compact as Superordination, one of the eight basic forms of agreement. See Figure 3. Again, human judges concurred.
Amy Schmitz, The Ohio State University
AI as the Fourth Party in Empowering Self-Represented Litigants
JusticeTech is an innovative initiative that explores how AI can act as the “fourth party” to help self-represented litigants (SRLs) navigate negotiations and mediation in vital areas impacting Access to Justice (A2J).
The collaborative efforts between the Moritz College of Law and the Computer Science and Engineering (CSE) department at Ohio State University are unique, involving researchers from both departments and Public Affairs. JusticeTech focuses on developing AI-enhanced Online Dispute Resolution (ODR) systems to expand A2J, particularly for SRLs in eviction and truancy cases. Ethical AI design is a cornerstone, ensuring technologies mitigate bias and safeguard individual rights.
Key projects include the Eviction Project, which involves empirical research and focus groups to understand the legal landscape of evictions, supported by nearly $1.6 million in grants. The Truancy Project addresses truancy issues in Ohio through online mediation and AI-enhanced assistance, with ongoing feedback from community partners and expert mentors. JusticeTech’s interdisciplinary approach involves project-based learning, pro-bono assistance, and robust research to provide insights into how technology can assist SRLs.