Articos: research you can audit in 30 minutes
Six months designing an AI research-report generator that survives the strict-reader test. This is what it looks like when a senior researcher has to defend the output. I led the research, the experience, and the production build.
What those numbers mean Point a plain AI at a pile of interviews and it hands back about 141 themes, of which roughly 7 in 100 are real. You then spend your week sorting them. This hands back 18, and about 4 in 10 are real. It finds slightly fewer of the true themes than plain prompting does, and I would rather say that than hide it. What you buy is that the ones it does put in front of you are worth acting on.
Overview
Traditional user research costs $5k–$30k per study and takes weeks. That blocks agencies on five-day timelines, SaaS teams shipping weekly, and founders without budget. Most synthetic user research tools collapsed into chat wrappers that produce fluent-but-unfaithful interviews. The hard problem is generating one a senior UX researcher would put their name on. Articos is built around a single principle: the product refuses to ship a report it cannot defend.
My title is four jobs, and this case study is organised by them. On Articos I set the early positioning and the quality bar, designed the experience, wrote the production code and the LLM harness behind it, and ran the research that tests whether the output holds up. I’m a founding designer, not the founder. I designed and coded the product and its design system, about 119K lines of TypeScript and React with 1,669 passing tests, then handed it to Disrupt Labs’ engineering team.
One number shaped all four jobs. When the product had a one-click report button, every one of those reports was edited before anyone shared it. Here’s one I publish on purpose. Plain prompting still finds a few more of the true themes, 85% against 75%. Articos wins where a researcher needs it to: 42% of the themes it reports are real, against 6.7%.
- ProductThe quality bar, the friction I kept, the positioning and the GTM pack.
- DesignThe report, six checkpoints that let someone say no, the flow end to end.
- EngineerThe 8-stage harness, the lens library, personas that browse the real page.
- ResearcherThe rubric, the 46-study evaluation, Grounded Simulation.
01 · Product: a report a senior UXR would sign
A research report is only worth something if a researcher will put their name on it. This is where that bar was set, and where I chose friction over a faster demo. The screens that hold people to it are in Design.
Articos is for the people slow, expensive research shuts out. Agencies on five-day timelines, SaaS teams shipping weekly, founders without a research budget. The bar was a report a senior UX researcher would sign, and fluent isn’t enough for that reader. Every claim has to survive someone who reads Baymard and NN/g.
Decision · kill the one-click “generate full report” button. Stakeholders asked for it; the reports were polished but undefendable. Telemetry: 100% of one-click reports were edited before share. In the week-six survey, four of five researchers said the friction made them more confident in the output. I lost the demo line. I kept the reason a researcher would put their name on the work.
Trace · one weak claim. Say a draft reads “most teams skip research because it takes too long.” A one-click report would have shipped that sentence as written. The next three chapters follow it.
Positioning, brand and the GTM handover
I also set the early positioning and built the brand. The site’s how-it-works section makes a promise the product can keep: three steps, thirty minutes, one report you can ship. Then I wrote a marketing pack as the handover to the GTM lead, so he could see the positioning, where Articos wins against the alternatives, and where it’s heading in the market.
FIG.04. The brand and the GTM handover, shown as screenshots. // click any tile to enlarge
02 · Design: screens that let someone say no
The bar only matters if the screens hold people to it. Here’s how the interface keeps a weak claim out of an export. The machinery behind each checkpoint is in Engineer.
Five screens carry the work: brief, lens, critique, repair, export. Each one is a decision the user owns. Everything else (settings, history, share) sits in side panels. Brief is where the model becomes accountable; lens is where the user picks which strict reader to invoke; critique is the first place the model can refuse; repair is the first place the user can refuse the model; export is where the audit trail lands.
- The headline is the claim. Every finding is a full sentence, so the headlines alone carry the argument.
- The rail reads as sentences. Findings, not section titles, so you can skim the case in the margin.
- The claim sentence is highlighted. The one sentence the finding rests on, marked inside its paragraph.
- The evidence sits under the claim. A real quote with its source marker, not a footnote at the bottom.
- One accent colour. Saved for “look here” and the main action, Download PDF.
Six checkpoints, each one refusable
Between the brief and the export, the researcher gets six chances to stop the model:
- Brief approval. Nothing runs until the researcher signs off on what the study is for.
- Lens selection. The researcher picks which strict readers judge the work.
- Blueprint review. The narrative angle and its confidence bands, before any prose exists.
- Critique acknowledgement. The model’s own flags, read before the section is.
- Repair approval. Every rewrite arrives as a before-and-after diff the researcher can reject.
- Exec-summary review. The summary for a non-researcher, checked against the claims it restates.
Trace. At the critique checkpoint, the weak sentence surfaces with a missing-evidence flag before the researcher reads the section. Repair can’t keep an unsourced claim, so it comes back pinned to a source paragraph or it’s cut.
- Study type
- Personas
- Persona map
- Script
- Interviews
- Report run
- Report
FIG.06. A study end to end, in the order the product runs it. Frames from my own test runs, across more than one study. // step through, or enlarge
FIG.07. The interview script, desktop and phone. The two shots come from different test studies. // click to enlarge
03 · Engineer: where the model ends and the user begins
Each checkpoint is backed by a pipeline stage that exists because the model fails in one specific way when it’s left alone. How well it all works is measured in Researcher.
I architected this harness and wrote the production code, about 119K lines of TypeScript and React with 1,669 passing tests, before handing it to Disrupt Labs’ engineering team. For that handover I built an interactive map of the pipeline, with every stage’s prompts and data contracts, so the engineers could see how each piece fits.
The report pipeline is an 8-stage service blueprint. Each stage exists because of one specific failure mode the model produces when left alone. So the guardrail and the recovery path are the design surface, and the model is the implementation detail.
- 01 ThemesCatches generic theme driftGuardrail Reject any theme with fewer than three source paragraphsIf it fails Re-run with a tighter source filterYou see You approve the brief, in an editor with a hypothesis-blind toggle
- 02 Web researchCatches citation hallucinationGuardrail De-duplicate, and drop low-authority sourcesIf it fails Replace with theme-based search on a real indexYou see Sources are listed with their provenance
- 03 BlueprintCatches overconfident synthesisGuardrail Cap confidence at 75% when the sources agree unanimouslyIf it fails Inject a refuting-source passYou see You review the narrative angle, in an outline with confidence bands
- 04 CritiqueCatches false authorityGuardrail Block on a critique-failure score: a 13-item cliché blocklist and a 5-part sycophancy checkIf it fails Trigger the repair passYou see A critique panel, collapsed by default
- 05 RepairCatches repair driftGuardrail The repair must keep every source pinIf it fails Reject, and escalate to a section rewriteYou see You approve or reject the repair, in a before-and-after diff
- 06 SectionsCatches section incoherenceGuardrail Citation coverage of at least 95% per sectionIf it fails A section-level repair passYou see You edit inline, with revision history
- 07 Exec rewriteCatches new-claim driftGuardrail No new claims: the summary may only restateIf it fails Block, and fall back to an extractive summaryYou see You review the summary, with claim-level evidence links
- 08 PolishCatches semantic driftGuardrail No prose change that alters what a claim saysIf it fails Reject the polish, and ship the pre-polish versionYou see You give final approval in the export modal, with the audit trail
Trace. The Critique stage flags the weak sentence, and the repair pass can’t keep an unsourced claim. It comes back pinned to a source paragraph, or it’s gone.
Personas are generated from a curated library of 14 lenses. The model picks 4 per
study but cannot invent lenses outside the library. The library is the guardrail, and stable
names let researchers build intuition across studies (e.g. adoption_stance: Skeptic vs
Pragmatist). Three of the fourteen:
Decision · hide the hypothesis from the model during generation. Confirmation bias is the failure mode fluent models produce by default. The script-generation service receives only role names and counts, never the hypothesis, which is contested against the evidence only at synthesis. Researchers asked to feed the model their hypothesis upfront (“it would be faster”). I shipped the friction instead, so the output can answer “did the model already know what you wanted to find?”
Personas that browse the real page
In A/B studies the personas used to read a text description of each landing page and never see it. I rebuilt that so each persona opens its variant in a real cloud browser, scrolls, clicks and thinks aloud with Claude’s Computer Use, then answers the interview in the same session. The model kept wandering off the page, so I blocked navigation in the browser itself instead of asking it nicely in a prompt. Live browsing costs too much to run on every study. It’s off by default, with a replay built from screenshots the study already has.
- Setup
- Personas
- Observation room
FIG.10. An A/B study, frames from my end-to-end test run. // step through, or enlarge
04 · Researcher: measured, including where it loses
None of this counts unless it’s measured. The rubric came first, and the evaluation reports where Articos loses as well as where it wins.
I built the rubric before the prompt. Optimize for fluency and the work fails on contact with someone who reads Baymard and NN/g. Six criteria, each scored against every draft before any model call shipped:
The rubric became an executable eval harness. Forty-six studies routed through it; 36 passed the citation-traceability check end to end; the rest exposed exactly the failures the rubric named. Evidence specificity was the hardest pass. First-pass synthesis produced confident-but-unsourced claims at 38% of the rate of the final pipeline.
Decision · build the rubric before the prompt. Without an external quality target, the model optimizes for fluency, which a senior researcher dismisses in the first paragraph. Tradeoff: two weeks delaying the demo. Stakeholders wanted a working pipeline. I shipped a rubric instead.
Validation harness. 46 studies routed through Articos across 9 domains; 36 passed end-to-end with citation traces verified and themes mapped to source paragraphs at the sentence level; the rest were caught on citation issues before any human saw the output. Mean Research Fidelity Index was 0.797 ± 0.050. On the 15-study controlled comparison, where all three conditions are scored by one fixed judge, the harness reaches 42% theme precision against 6.7% for bare prompting, an F1 of 0.537 against 0.123. Recall is nearly free, and slightly lower here than bare prompting’s: 75% against 85%. Precision is what the architecture buys. The eval data is publicly inspectable.
Trace. On the rubric, the weak sentence fails evidence specificity and traceability. There’s no source paragraph behind it, and nothing to check in two clicks.
The method is public
Articos publishes the science behind the product. The five failure modes of AI research it was built against: no scientific grounding, no memory or life history, confirmation bias, persona homogeneity, and cost. What each system does about them: hypothesis-blind generation with context isolation, cognitive memory, stance-diverse personas grounded in NEO-PI-R and ACT-R, six-stage synthesis, and an adversarial quality review that scores every study before it ships. The validation evidence is downloadable, including the pre-print, the datasets behind the precision and F1 figures, and a worked persona example.
Read the full methodology on articos.com →
The eval harness, confidence caps, and hypothesis-blind generation here are the applied form of Grounded Simulation, the architecture I introduced and wrote up on SSRN (it’s under peer review) for faithful synthetic UX research. For the formal paper and the rest of my auditable AI research, see the research page.
What it can’t do yet. Two things stay open. The product can’t yet grade source authority at the paragraph level for non-English corpora, and it can’t reliably catch a sophisticated false-authority pattern (a real-but-irrelevant citation). The methods-auditor lens gets the egregious cases; the subtle ones still pass.
What worked, what didn’t, what’s next
The four receipts. Each one names a failure we shipped, the reviewer feedback that caught it, and the fix now in production:
- Overconfident synthesis. A reviewer put it plainly: this sounds smart, but I cannot verify it. Fix: a 75% confidence cap when the sources agree unanimously, claim-level source links, and caveats inlined instead of pushed to a footer.
- Generic clichés. Fluent filler survived every early draft. Fix: a 13-item blocklist that refuses phrasings like “in today’s fast-paced digital world” at critique, and a repair pass that forces a named behavior and a concrete tradeoff in its place.
- Claim traceability. A reviewer could not verify a single claim in two clicks. Fix: inline links, a source-list sidebar, and an audit panel mapping every claim back to its source paragraph.
- The feature we killed. We built a one-click “generate full report” button, and the reports came out polished but undefendable. Fix: researchers asked for the explicit-checkpoint flow back, and got it. The telemetry that settled it is in Product.
FIG.12. One tile per receipt: the report as it ships now, where each fix shows. // click any receipt to enlarge
The live product. Over the same six months, articos.com grew roughly 10× in active users. It earned coverage from TLDR Design, Superhuman, and There’s An AI For That, plus inbound ChatGPT referrals, the assistant recommending the product as an answer. Users spread across the US, UK, Canada, Singapore, Germany, and Australia.
The detail sits behind a conversation. A growth multiple is the honest summary. The month-by-month shape is the part actually worth discussing, and Articos is a Disrupt Labs product, so I walk through it rather than post it. Active users across the six-month window, sessions, page views, engagement rate, average session length, referral volume by publication, and the geographic split. Ask me and I’ll take you through them.
How the six months ran
- December. Daily design syncs, then a seven-event handover sequence from December 12 to 19.
- February. Daily v2 catch-ups, and the A/B testing track began.
- Throughout. More than 100 working sessions, plus the website’s planning and iteration.
What’s next. (1) Claim-level evidence grading, with confidence derived from source strength, not model self-report. (2) Team review, where research leads overlay their critique on the system’s without rewriting the section. (3) The eval harness in production: if the system can’t defend a report by its own rubric, it refuses to export.
The product refuses to ship a report it cannot defend. The refusal is the feature.
Methodology & sources
Six months · ~37 LLM service files · 13 screens · 8-stage report pipeline · 46 studies through the validation harness (36 verified end-to-end). Grounded in literature including Weisz et al., Design Principles for Generative AI Applications (CHI 2024); De-skilling, Cognitive Offloading, and Misplaced Responsibilities (CHI 2025 EA); The trust crisis in artificial intelligence (Technology in Society, 2026); and HAX: Designing the Internet of Agents (arXiv 2025).