AHMAD BILAL
← AHMAD BILAL / WRITING AI & RESEARCH
AI & RESEARCHSIMULATED RESEARCHWORKFLOW

A simulated user research workflow, end to end

Grounding sources to audit, the whole pipeline: how a simulated study runs when every finding has to trace back to evidence.

Ahmad BilalAug 2026~6 minAI & Research
FIG. 01 · ONE GATE Six pipeline stages from sources to audit; between chain and score the line reroutes through a gate.
Six stages. The gate is where an unlinked claim stops.

A simulated user research workflow uses LLMs to stand in for study participants. The version worth building runs six stages in a fixed order. Ground the personas in real behavioral data. Generate them so they don't collapse into the same helpful voice. Interview them without the interviewer knowing the hypothesis. Chain every claim back to a transcript line. Score confidence by how often participants disagreed, not by how confident the write-up sounds. Then audit the whole thing before anyone acts on it. Skip a stage and you get fluent transcripts instead of a study.

Here's how I actually run it.

01Stage 1: grounding the sources

The persona doesn't start with a prompt. It starts with what you can point to: prior interview transcripts if you have them, survey or CRM data if you don't. Where both are thin, I reach for published behavioral instruments: trait facets, cultural dimension frameworks, a memory model that decays the way human recall decays. The point is that nothing about the persona comes from an adjective I typed. If a stakeholder asks where a trait came from, I want an answer that isn't "it sounded right."

This is the layer most simulated-user tools skip, and it's the reason personas built from a few demographic lines read fine and say almost nothing you could act on. A label like "32-year-old marketing manager, budget-conscious" describes a bracket. It tells a model nothing about how that specific person hesitates, rationalizes, or gets talked out of something, which is most of what an interview is actually for.

The architecture that holds this stage and the next five together is what I call Grounded Simulation, laid out in an SSRN working paper, not yet peer-reviewed, by Ahmad Bilal. The short version of what the term means lives on the research page; this piece is the walkthrough of running it.

02Stage 2: generating personas that don't collapse into each other

Generate ten personas from one template and you tend to get one persona ten times, wearing different names. Ask each of them a hard question and they agree with each other and with you, because a model tuned to be helpful defaults to agreement. A sharper prompt doesn't fix that, because nothing in the generation step is forcing disagreement in the first place.

So I build variance in on purpose. Personas draw from different points across the same instruments rather than the same demographic bracket restated five ways. Then I check the population before I trust it: do these ten actually disagree with each other on the questions that matter, or did I just generate the same helpful voice in different outfits? A batch that comes back too agreeable gets regenerated, not shipped.

FIG. 02 · SPREAD CHECK Two dot fields divided by a variance threshold: a tight cluster marked regenerate, a wide spread marked ship.
Ten personas piled in one cluster get regenerated; ten spread across the instrument axes ship.

03Stage 3: interviewing without leaking the hypothesis

Say the study is about whether shipping fees drive checkout abandonment. Write a persona prompt that already knows that, and you'll interview a price-sensitive shopper who talks about shipping fees. Congratulations, you've measured your own prompt.

Whatever runs the interview never sees the hypothesis under test. It sees the persona and the interview guide, nothing about what I'm hoping to learn. That single separation, generation in one file, hypothesis in another, closes the most common route to a self-fulfilling study. It costs something real, though. I can't nudge a persona back on track when it says something inconvenient, because that nudge is exactly what would contaminate the result. I've written the mechanics of this stage in more depth in the Grounded Simulation essay; here it's stage three of six.

04Stage 4: chaining evidence back to source

A theme in a report is a sentence someone wrote after reading evidence. Treating that sentence as the evidence itself is how an unfounded claim survives review. Every claim in the output has to link to the specific transcript span that produced it, and if it can't, it doesn't ship as a finding.

This is also where "sounds true" and "is traceable" stop being the same test. A theme can read beautifully and trace to nothing. The chain is what tells you which one you're holding.

05Stage 5: scoring confidence instead of asserting it

Models tuned to be agreeable will validate almost any idea you put in front of them, which means a transcript full of enthusiastic responses tells you the model was polite, not that the idea is good. So confidence gets computed rather than reported: I count how often a theme was contradicted across the interview set against how often it was supported, and that ratio is the score. The model doesn't get to grade its own output.

Read the refutation count before you read anything else. A theme participants pushed back on nearly as often as they endorsed it means your simulated population disagrees internally, and a report that flattened that into one tidy finding deleted the most useful part. That count is the cheapest, most skippable signal in the whole artifact, which is exactly why I check it first.

FIG. 03 · REFUTATION LEDGER A ledger of themes with support ticks left and contradiction ticks right of a center rule, ratios at each row end.
Support and contradiction counted per theme; the near-balanced row gets flagged, and flattened findings get deleted.

06Stage 6: the audit

Somebody who didn't write the report opens a finding and has thirty minutes to decide whether they trust it. They should be able to see the quotes, the transcript turns those quotes came from, whether the hypothesis was actually blind at interview time, and the confidence score computed rather than claimed. Any of that missing, and the finding doesn't survive the audit, no matter how well it reads.

Auditability and validity are separate claims, and it's worth being honest about where this workflow stops. A theme can trace perfectly to a real transcript line and still describe nobody who exists. The audit checks whether the study is honest about how it got there. It doesn't check whether the population inside it resembles anyone real, and that second question only real users can answer. Treat a simulated study that passes this audit as one that has earned a real study. On its own, it hasn't earned the right to move a roadmap.

That's the whole loop: grounded personas, checked for collapse, interviewed blind, chained to evidence, scored on disagreement, then opened by someone who wasn't in the room when it was built. The shipped version runs inside the Articos research pipeline, and the broader case for building AI research this way is in auditable AI research.

What would you want to see before you'd trust a synthetic study enough to let it change a roadmap?

07References

08Further study

Branching by what you are actually trying to do next.

If you are about to run a simulated study.

If you are picking instruments to ground a persona.

If you are deciding how much to trust the output.