Panel of Experts Prompting: Templates and What Research Found
Most people prompt ChatGPT like it’s an intern.
“Write me a marketing email.” “Fix this code.” “Explain this concept.”
You get much more out of it when you use it like a boardroom of experts.
The panel of experts technique
The prompt structure looks like this:
“I want you to simulate a panel of experts discussing [topic]. The panel includes:
- [Expert 1 with specific credentials]
- [Expert 2 with different perspective]
- [Expert 3 with contrarian view]
Each expert should share their perspective on [question], and they should respond to each other’s points.”
Why this works
LLMs have absorbed expertise from countless perspectives. When you ask a generic question, you get a generic answer, a blend of all those perspectives into something safe and mediocre.
When you ask for a specific expert’s perspective, the answer tends to commit to a point of view. And when you create dialogue between experts, you get something closer to real intellectual discourse.
There’s research behind the dialogue part. In Du et al. (2023), researchers from MIT and Google Brain had three copies of ChatGPT answer the same question, read each other’s answers, and revise over two rounds. On 100 arithmetic problems, accuracy went from 67.0% for a single answer to 81.8% after the debate, and grade school math went from 77.0% to 85.0%. Factual accuracy on generated biographies of computer scientists went from 66.0% to 73.8%.
Show the data
| Setup | Arithmetic | Grade school math |
|---|---|---|
| One model, one answer | 67.0% | 77.0% |
| One model, self-critique | 72.1% | 75.0% |
| Three models, majority vote | 69.0% | 81.0% |
| Three models, two rounds of debate | 81.8% | 85.0% |
That study used separate model instances. The panel prompt does it inside one conversation, which is closer to what Microsoft Research and the University of Illinois called Solo Performance Prompting: a single model invents several fine-grained personas and has them collaborate. They found it reduced factual hallucination while keeping reasoning strong, and that several specific personas beat one persona or a fixed few. The catch they reported is that the effect only showed up in GPT-4, and not in GPT-3.5-turbo or Llama2-13b-chat. Use a strong model for this.
A real example
Bad prompt:
“What’s the best database for my startup?”
Good prompt:
“I’m building a startup that handles user-generated content with high write volume. Simulate a panel discussion:
- A PostgreSQL expert who has scaled databases at Stripe
- A MongoDB advocate who has built content platforms
- A contrarian database architect who believes most startups overcomplicate data storage
Each should argue for their approach and respond to the others’ points.”
The second prompt gets you actual trade-offs instead of “it depends,” a contrarian asking whether you need either database, and a synthesis you can decide from.
Variations that work
The Mentor Panel
“Simulate advice from:
- Paul Graham on startup strategy
- Patrick Collison on building developer tools
- Naval Ravikant on making decisions under uncertainty”
The Technical Review
“Review this architecture as if you were:
- A senior SRE concerned about operational burden
- A security engineer looking for vulnerabilities
- A product engineer worried about developer experience”
The Devil’s Advocate
“Make the strongest possible case against [my idea], as if you were:
- A competitor trying to find weaknesses
- A skeptical investor looking for deal-breakers
- A customer who’s been burned before”
Tips for better panels
Make experts specific. “A PostgreSQL expert who scaled Stripe’s databases” beats “a database expert.”
Include contrarians. The best panels have someone who disagrees with the premise.
Ask for interaction. “Respond to each other’s points” generates richer discussion.
Give context. The more specific your situation, the more relevant the advice.
Follow up. “What would [Expert 1] say to [Expert 2]’s point about X?”
Where I’ve used it
The first time was close to an accident. While building an interactive guide with Claude Code, I had a music engine that sounded like a ringtone. I asked Claude to roleplay as the best Super Famicom composers of all time and critique the composition. It sounds absurd, and it produced the best of six rewrites.
The bigger test came in May 2026, on our own outbound sales campaign. I had Claude Code spawn eight well-known sales and positioning experts, by name, each reviewing the campaign on its own and in parallel, then merge their critiques into a fix list. Round one gave it a median of 4.5 out of 10, with critiques specific enough that we fixed a list of them that night. Over four rounds the panel grew to 11 personas and 38 reviews. The GPT-4 caveat above hasn’t been a problem on current Claude models in my use. The personas disagreed with each other, and their scores in round one ran from 3 to 6.
Then the panel showed its failure mode. After round three, the synthesis projected a score around 7. Round four handed all 11 personas the revised campaign blind and asked for a fresh score, and the median came back at 4.0. Each round’s projection had run about 1.5 points above what fresh reviewers gave the same version. Round four’s loudest message was to stop reviewing and start sending: 27 critiques in, we hadn’t sent a single message.
Two things I’d keep. Each persona reviews in its own context and never sees the others, because agreement between reviewers who can’t see each other is the signal worth having. That’s closer to the separate model instances in the Du et al. study than to one conversation playing every role. Round four went one step further and gave the reviewers no history from earlier rounds, which is what caught the inflation. And a panel needs a stopping point. It’s good at finding problems, and it will keep finding them for as long as you keep asking. Ours ended at round four, when the reviewers themselves said to go get real data.
When not to use this
This technique is overkill for simple questions. A study of 162 personas across 2,410 factual questions found that adding a persona to the system prompt did not improve accuracy compared to no persona at all. A panel helps when there’s something to argue about. Don’t use it for:
- Factual lookups
- Straightforward coding tasks
- Summarization
Use it for:
- Strategic decisions
- Architecture choices
- Anything where “it depends” is the likely answer
The averaging problem
My working theory is that the panel fights the averaging described above. Specific personas narrow the search space, and making them argue forces a synthesis closer to what you’d get from actually consulting experts. The model already has the expertise. The panel points it there.
Try it on your next big decision.
If you want more techniques like this in context, my interactive Claude Code guide is free and written for people who don’t code. If you’d rather practice on your real work, I do Claude Code coaching over screen share. Build It Yourself coaching is four 90-minute sessions for $1,997.
