Macbeth, a tour guide, and LangGraph

Building a small grounded multi-model debate harness

A knockoff bard in Stratford

I recently made a pilgrimage to Stratford-upon-Avon. My limited knowledge of the bard’s works ranges from whatever abridged version I read in my school years to the gritty Michael Fassbender version of Macbeth roaming around misty Scotland.

An amazing tour guide (hey Cameron!) who was an actual actor with experience in the Royal Shakespeare Company led the proceedings with our group. He could quote Shakespeare at will and refused to break out of character unless it was for mundane things like letting us knaves know when to reassemble near the tour bus etc. Much to my daughter’s amusement, he stayed in character throughout, and I caught the sharp end of a few zingers as I was bumbling along.

Every time someone in the group asked him a question, his answer got sharper like a supreme debater addressing the key points and skewering the opposition. I committed myself to a higher standard with respect to Shakespearean literature after this encounter.

The encounter made me want a ringside seat to skilled debaters dismantling arguments across topics. This led to a small LangGraph debate harness: two models take opposing sides, search a curated quote collection, and hand their transcript to a third model for synthesis.

Why one model is not enough

If you ask a single model if Macbeth is about ambition, you will get a balanced summary that sits on the fence, commits to nothing, and you feel like you’ve learnt nothing new.

Here is a typical example below. The model opens by calling the claim “not wrong,” then argues both sides for several paragraphs, then closes by calling the claim “true but incomplete”:

Few claims about Shakespeare’s Macbeth are as common as the assertion that it is “a play about ambition.” The statement is not wrong: ambition is the engine that drives the plot from prophecy to regicide to tyranny. Yet to reduce the play to a single theme risks overlooking the dense web of fate, gender, guilt, and political order…

Macbeth is undeniably a play about ambition. But it is not only about ambition. To call Macbeth a play about ambition is true but incomplete.

That is the shape of every single-model answer I have gotten on a contested claim. It agrees, then hedges, then agrees again, which drives me nuts.

if you run the same claim through two agents that have to answer each other, this sharpens the discussion as they both try to corner each other.

The AGAINST agent forces the FOR agent to defend a specific claim (“he names it as his motive”) against a specific counter (“he names it to reject it”). Opposition makes an objection to weaken the argument. A synthesizer / summarizer reads the whole transcript afterwards and can say where each side was strongest, which is a more useful output than fence-sitting equalizing with both sides. It feels more natural than grabbing a single model by the scruff of its transformer neck and asking it to take a position.

The second thing that matters here is grounding. An agent arguing about Macbeth will happily produce lines that sound like Shakespeare without being Shakespeare. So every argument turn in this system is preceded by a search against a vetted dataset, and the model is told to cite what it found. It does not stop hallucination though but it makes the bad quotes more verifiable.

Architecture

The graph is a LangGraph StateGraph. It runs sequential turns. argue_for goes first, argue_against responds, and after each pair a conditional edge checks whether enough rounds have completed. Once they have, a synthesize node reads the full transcript. Both argument agents can call the MCP search tool during their turn.

It has to be sequential as running the two arguers in parallel will mean neither one sees what the other said, and you get two monologues instead of a debate.

State management

class DebateState(BaseModel):
    claim: str
    charlimit: int = 100
    rounds: int = 3
    transcript: list[dict] = Field(default_factory=list)
    synthesis: str = ""
    status: Annotated[str, _last] = "pending"

LangGraph handles the orchestration, but the state itself is a Pydantic model rather than a free-form dict. The transcript is a list of turn dicts, each carrying side, text and model.

The status field is the one that needed some debugging. Multiple nodes write to it in the same step, and without a reducer LangGraph raises InvalidUpdateError because it cannot decide which write wins. The Annotated[str, _last] annotation attaches a two-argument function that returns the second one, so the last write is kept:

def _last(_a: Any, b: Any) -> Any:
    return b

Any state field that more than one node touches in the same step needs a reducer, whether that is last-write-wins, concatenation, or something custom.

The agents and the two-pass tool loop

argue_for and argue_against are async callables that share a _turn function. The function:

1. Builds a system prompt with the stance and the collection name to search.
2. Formats the transcript so far as context.
3. Calls the LLM with tools bound.
4. If the LLM emitted tool calls, executes them and sends a follow-up message with the results inlined.
5. Truncates the output to charlimit.
6. Appends the turn to the transcript and returns the state update.

Step 3 lets the model decide whether to search. If it does, the results come back as citation strings. Step 4 hands those strings back to the model and asks it to write the argument citing them. So that’s two LLM calls per agent turn whenever the tool fires.

LangGraph ships a prebuilt ToolNode that does this for you. Not required for this small example but in production, use ToolNode.

On step 5, charlimit is a hard slice, text[:state.charlimit], applied after generation. It keeps turns sharp rebuttals and it keeps the transcript short for the synthesizer, but it cuts mid-word and it does not save you a single token. If you want short turns cheaply, you can use cap max_tokens instead. I left the slice in because I wanted a guaranteed ceiling on transcript length, and prefer a truncated turn than a 900 character speech.

Grounding: MCP search server

The search tool runs as a separate process over stdio, using the MCP Python SDK’s FastMCP class. It exposes two tools:

• search(query, collection) - substring match against quote text, speaker and theme tags. Returns matching passages with source attribution.
• summarize(topic, collection) - returns a work summary or a themed list of passages.

This is a substring match over a hand-curated Python dict of public domain quotes, each one carrying text, speaker, act, scene and themes. There is no vector store required here and no ranking which you would need on a heavier dataset.

The collection parameter picks the dataset. Shakespeare is the default. Each collection is a Python file in data/ exporting a QUOTES dict, and the registry in data/__init__.py maps collection names to modules. You can add any topic you want and have your favorite LLM generate it as long as you are sure about the accuracy.

Multi-model routing

Each role can run a different model. Which one depends on token availability, the state of my various subscriptions, and whether my local hardware is busy with something else.

MODELS = {
    "argue_for": "ollama:deepseek-v4-pro:cloud",
    "argue_against": "ollama:glm-5.2:cloud",
    "synthesize": "ollama:kimi-k2.7-code:cloud",
}

make_model_config parses the provider:model format and instantiates the right LangChain chat class. Ollama gets ChatOllama, OpenAI gets ChatOpenAI, Anthropic gets ChatAnthropic. Swapping to a hosted model is a config change, or a per-run override with --model-for openai:gpt-4o.

Different models per side is to ensure they are not framing the arguments similarly.

Running it

python -m hark "Macbeth is about ambition"
python -m hark "Climate change is the biggest threat facing humanity" \
    --collection climate --rounds 5 --charlimit 150
python -m hark --chat

Flags:

• --collection which dataset to search
• --rounds how many back-and-forth turns
• --charlimit max characters per agent turn
• --model-for / --model-against / --model-synthesize override models per role
• --chat interactive REPL

run.py starts the MCP server as a subprocess using MultiServerMCPClient, discovers the tools at runtime, and passes them to the graph. Results are written to results.md after each run.

A real debate

Running python -m hark "Macbeth is about ambition" --rounds 2 --charlimit 100:

And the synthesis:

The side arguing that Macbeth is about ambition is strongest when it tracks what Macbeth does after the witches speak. He takes their hint and runs with it. He kills Duncan to seize the throne and keeps killing to hold it. He calls ambition his only spur.

[Snipped for brevity: the weakness paragraphs where each side critiques the other.]

The play gives us ambition alongside fear and the witches’ suggestion. Calling Macbeth a play about ambition captures part of the action but leaves out the rest.

For what it is worth, I think FOR has it. He needs to take ownership.

Some failure modes

A three round debate with tool calls firing on every turn is 6 argument turns at up to 2 LLM calls each, plus 1 synthesis call. That is up to 13 calls for one claim.

If I ran the same 13 calls against a hosted API, it would cost roughly a few cents at current per-token pricing but will depend on your use case.

Three failure modes I have actually hit:

• Both sides confidently wrong. Grounding constrains the quotes, not the reasoning. If both models share a misreading, the debate reinforces it and the synthesizer reports a clean disagreement about the wrong thing.
  
• False balance in the synthesis. The synthesizer is told not to declare a winner in the literary argument. That is right for contested literary claims and wrong for factual ones, where one side is simply correct. For factual claims you want a judge, not a neutral synthesizer.
  
• Retrieval decides the argument. Substring match means a side that phrases its query well gets evidence and the other side gets an empty list and argues from weights. That is a retrieval artifact showing up as a debate result, and this is why a real index behind the MCP server will be needed in a real production deployment. That will be possibly part 2 of this post.

Where this is useful

I can see this being useful for general adversarial testing of a claim, with citations, and a record of the exchange you can analyze.

Another useful use case is red-teaming a model’s output. Debate something a model told you, make it make the claim, and let a second model attack it with the source documents cited. Contract clause analysis and pre-merge code review are other use cases.

The point is not to outsource judgment to a panel of models. It is to make the disagreement visible before you make up your own mind. I also want to build an enhancement that will let me inject my own argument into this and let the synthesizer add that to the final mix.

Repo is here.