How Stefan works with agents and judges results

This page describes how Stefan Preu prepares agents for work and how he judges what they produce. These are his reported practices and preferences, not universal rules or measured results.

Preparing an agent for a long run

Before a task likely to run for hours, Stefan asks the agent to get familiar with the project and its recorded context first. The agent should then bring the questions that genuinely need his input, near the start where possible, ideally with possible answers or options that make the decision easier. Settling important choices early avoids hours of work in the wrong direction.

What a smart question is

For Stefan, a smart question is one the agent has investigated first: is it already answered in the project material, or reasonably inferable from other context? He has seen agents resolve some of their initial questions this way, leaving fewer for him.

He rejects universal rules such as "always ask about X" or "never interrupt for Y". Questions later in the run are fine. When an agent should interrupt depends on the task.

Check the setup before investing hours

An early failure in a necessary tool or connection can matter for a long task. Stefan prefers to notice and fix such a problem before letting an agent invest hours in a compromised setup, for example by repairing the connection and starting the session again cleanly.

His concern that an agent may avoid a tool after it failed once comes from experience, not from a universal rule about agent behavior. Check the actual situation rather than assuming a failure has been recovered from.

"Was it tested end to end?"

When an agent reports that work is finished, Stefan often takes a quick look to understand the result, then asks something more specific than "is it done?": was it tested end to end, the way a real user would use it?

He wants to know what supports the claim that the result is good, not only that the agent says it is complete. The right evidence still depends on the task and its risk.

Define a good result before measuring effort

Asked about a run that took twice as many turns but still produced a "good result", Stefan first asks how "good" was established. The definition of quality and its evidence come before conclusions about efficiency.

He would investigate unusual effort, for instance hard-to-find data or a task that was difficult to understand. But a green dashboard or a confident agent summary is not a complete definition of success.

Bigger signals instead of micromanagement

Stefan prefers watching bigger signals to choosing every small edit himself. For example, he would ask an agent to investigate excessive CI time rather than pick individual tests to delete.

Signals he finds useful for his own workflow include host resource use, time spent building and testing, and the number of agent turns a task takes. They need comparable tasks and judgment: more turns might reflect a harder task, difficult data access, confusion or useful extra work. Turn count alone does not measure quality or explain a slowdown.

The waiting that matters

Stefan's concern is not whether a shell command takes two seconds or five. What matters is accumulated waiting for builds, test runs and CI pipelines that can each take around twenty to thirty minutes. Those waits add up to real development friction, so he asks where the overall workflow repeatedly waits for feedback.

The durations are his illustrative examples, not a benchmark.

When an agent struggles

Stefan watches where agents struggle, not only whether they finish. Confusing code, awkward data access, accumulated complexity or an unsuitable workflow can show up as agent friction, and the same friction often slows human developers too. He treats it as a lead for investigating the system around the agent.

A struggle is not automatically proof that the code is wrong or that the model is at fault. The next step is to look at the specific trace, result and task.

Agents reviewing agents

Stefan experiments with having another, sometimes larger, model review results or compare an agent's output with a developer's. The comparison can surface differences and questions worth checking.

He treats a reviewing model's verdict as one input to evaluation, not as independent proof. Evidence for the particular task is still needed.

Tests as models change

In Stefan's experience by October 2026, excessive or duplicated tests had become more of a problem than too few. He removed some older prompt instructions about test style and coverage thresholds as the models he used changed.

His aim is to cut redundant tests without losing meaningful coverage. This is a dated personal observation, not a claim that every model writes enough tests, that coverage targets are unnecessary, or that an agent's assurance is enough reason to delete a test.

Oversight depends on the risk

Stefan's preference for giving agents room and avoiding code-level micromanagement describes his own working style in his own kind of work. He explicitly does not extend it to aviation, medical or other safety-critical settings.

The necessary review, controls and evidence depend on the task and its consequences.

Data foundation and personal experimentation

Stefan separates two things that shape results with agents. One is the data foundation: whether relevant data exists, how good it is and whether an agent can reach it. The other is personal learning: trying agents on your own tasks, in your own style and workflows, to develop a feel for what they can and cannot do.

Both matter, and one does not replace the other. There is no universal prompting recipe that substitutes for either.