AHMAD BILAL
← AHMAD BILAL / WRITING AI & DESIGN
AI & DESIGNAGENTS UXPOV

Designing AI behavior, not AI screens

The screen is the easy part. The hard part is what the agent does: when it acts, when it holds back, when it admits it is not sure. That behaviour is the thing you actually design.

Ahmad Bilal2026~6 minAI & Design
FIG. 01 · ACT-HOLD-ASK Dots at a baseline in three groups: crossing it (act), stopped at it (hold), resting on it as open rings (ask).
Act, hold, ask. One agent, three behaviours, each decided in advance rather than left to the model.

I shipped a one-click "generate full report" button, and then I killed it.

The button went in because stakeholders wanted it for the demo. That is an honest reason. A researcher clicks once, a finished report appears, the room reacts. What it replaced demos badly: explicit checkpoints where the researcher approves the persona set, then the themes, then the synthesis, each one a small stop where the software waits for a person. On a stage that reads as a product asking permission to do its own job.

Then reports started going out. Every one the button produced got edited before anyone shared it. Not proofreading. Researchers were cutting themes, softening claims, deleting sections that read as confident and were not. The button had not removed the point where a person judges. It had moved that point to after the artifact looked finished, which is the most expensive place for it. A document that looks done carries authority it has not earned, and the person editing it is arguing with something that has already decided.

So the button came out. The checkpoints went back in.

01What the reversal cost

That was not free, and the bill is worth stating plainly.

I had to go back to the people who had asked for the button and tell them the demo was getting slower again. Time from starting a study to holding a report went up and stayed up. Some of the stops are tedious: approving a persona set is a real minute out of a real afternoon, several times per study, and nobody has ever told me they enjoy it. If you are weighing this trade in your own product, weigh it in the right units. You pay every day. The failure you are preventing only shows up sometimes, and when it does, it shows up as a document somebody already sent.

In the next internal survey, four of the five researchers using the tool said the added friction made them more confident in the output rather than less. Five people. That is a team, not a study, and I would not carry the ratio any further than the room it came from. What made it worth acting on was the direction, the opposite of my assumption. I had removed a stop because I thought stops were a tax, and the people paying the tax said the stops were what let them put their name on the result.

A checkpoint earns its place when it arrives while changing the answer is still cheap. That is the whole rule. Most of behaviour design is working out where that moment sits, study by study.

FIG. 02 · THE COST STAIR Four pipeline stations with a rising cost stair beneath, thin early correction and heavy hatched late one.
The same correction, priced at each station: a tick early, a hatched block after it looks finished.

02Left alone, the model has one default

That default is to sound confident. Designing an agent's behaviour means deciding in advance where the default is wrong and what should happen instead. Two of those decisions are worth your time, because both port into systems that have nothing to do with user research.

The first is a cap. When every simulated voice in a study agrees, the synthesis stage refuses to report confidence above 75%.

That looks backwards. Unanimity is the strongest signal a naive scoring function can receive. The problem is that these voices are not independent. They come from one model, one prompt family, one curated library of persona lenses, inside one run. When ten people in a room separately land on the same answer, that tells you something, because each of them could have disagreed for reasons of their own. When ten simulated voices agree, you have learned about your sampling before you have learned anything about the world.

Agreement among simulated voices tells you about your sampling long before it tells you anything about the users.

The commonest cause of a unanimous panel is that the panel collapsed. Personas differed in name and biography and not in the thing being asked about. Or the prompt leaked its preferred answer and every voice picked it up. Or the model's average register quietly swallowed the edge cases, which is what average registers do. Nuance disappearing and nuance being absent look identical in the output. A scorer cannot tell them apart, so it should not be allowed to grade them as though it could.

Hence the cap. The exact figure is chosen rather than derived. Argue me down to 70 and I would not fight you. That a ceiling exists at all, and that a reader can see which substage applied it, is the part worth keeping. If you are building anything that folds several model voices into one number, look at what your aggregation does when every voice agrees. Most treat it as the maximum. Ask whether yours can tell a settled question from a collapsed one, and if it cannot, cap it and say so in the output.

03The agent argues with itself before it speaks

The second decision is a loop. The model writes a section. A separate stage critiques that section against a cliché blocklist and a sycophancy check. A repair stage then rewrites the weak parts, and every source pin has to survive the rewrite. The researcher reads the repaired draft and never sees the first one.

No screen expresses that. You will not find it in a design file, there is no state for it in the prototype, and a user notices it only as the absence of a certain kind of embarrassing paragraph. It still shapes the product more than any panel does. There is a matching rule on the generation side, that the part writing personas and scripts never sees the hypothesis under test, and grounded simulation covers why that one matters more than it sounds.

FIG. 03 · ARGUED FIRST A hatched-out first draft passing through a slotted critique gate into a clean second draft, source pins running beneath both.
The first draft dies at the critique gate; the reader only ever sees the second, and the source pins never break.

04Behaviour you cannot repeat is luck

What turns any of this from a good day into a product is the structure the model runs inside. The report pipeline breaks into stages, and those break into more than a hundred substages, each one there to catch a specific failure: theme drift, citation hallucination, overconfident synthesis, repair drift.

That number is not a target and copying it would be a strange goal. It is the honest cost of putting inspection points where the failures actually happen. What it buys is repeatability: the same study run twice hits the same gates, and a reviewer can point at the exact substage that capped a score or rejected a theme. Building with LLMs goes properly inside that structure, and auditable AI research takes the reviewer's side of it.

Something to try this week, and it takes about twenty minutes. Find the last AI-generated artifact from your system that a person edited before sending. Work out where in the pipeline that edit could have been made instead, and what it would have cost there. If the answer is three stages earlier for a tenth of the effort, you have found a checkpoint worth building, and you have found out what your one-click button is really doing.

Which stop in your own system would your users defend if you tried to take it out?

05References

06Further study

Branching by what you are actually trying to do next.

If you are deciding where the human checkpoint goes.

If your scorer treats unanimity as the maximum.

If you are building the critique-and-repair loop.