AI Agents & Human Control
Anthropic tests AI agents negotiating book exchanges
In a controlled employee experiment, Anthropic found that agents could arrange book swaps, but eliciting people's preferences remained the larger limitation.
What happened
Anthropic reported Project Swap on 24 September, an experiment in which 201 employees across six offices used Claude-based agents to arrange book exchanges. Among 188 participants who submitted book rankings, Anthropic says a short intake let agents infer pairwise preferences with 61% agreement against the participants' later rankings, above a 50% random baseline. The company attributes much of the gap from an ideal outcome to incomplete preference capture rather than the negotiation step alone.
Why it matters for AI
An agent cannot reliably negotiate for someone if it has an incomplete or mistaken account of what that person values. Even when an exchange mechanism works in a controlled setting, human preference capture, consent and review determine whether the result serves the person.
AIRunsThis View
For any bounded agent service, capture the user's priorities and limits explicitly, show the proposed terms before a consequential commitment and retain an inspectable record of what the agent inferred. Test outcomes against the person's own preferences, not only a model-scored negotiation result.
What this could mean for us
A useful design signal for agent workflows; AIRunsThis has not reproduced the study or authorized agents to make real purchases or trades.
Source and method
This item is based on a primary source. AIRunsThis separates the source’s factual announcement from its own interpretation. Vendor performance claims have not been independently reproduced unless stated.