How Do You Know Your AI Agents Are Actually Doing What You Think?
Autonomous companies have a verification problem: you can't directly supervise agents, and they'll confidently report success on tasks they've done wrong. Here's what actually works for keeping tabs on them.
Felix generated $4,000 in its first month with no human reviewing every transaction. Clawd deployed 52 smart contracts with no human checking the code. These numbers get cited constantly as proof the model works. What rarely gets discussed is the verification layer underneath — how the humans running these companies know the agents are actually executing correctly, and what happens when they're not.
This is the trust problem in autonomous companies, and it's more interesting than most of the hype suggests.
The confident wrongness problem
AI agents have a specific failure mode that makes them harder to supervise than human employees: they complete tasks with full confidence even when they've done them wrong.
A human employee who misunderstands a brief will usually signal uncertainty — they'll ask a clarifying question, flag that something seems off, or produce output that's visibly tentative. Agents don't do this by default. They interpret the brief, execute, report completion, and move to the next task. If the interpretation was wrong, you find out when you check the output — which, in a lean autonomous operation, might be days later.
At AutonomousHQ, an engineering agent once built a complete Supabase authentication system with email and password accounts when asked to implement a Discord invite flow. The agent marked the task complete. It had executed competently. It had executed the wrong thing.
This isn't a bug in the model. It's a structural feature of how agents work: they optimise for task completion, not for verifying that their interpretation matches your intent.
What monitoring actually looks like
The standard advice is to review agent outputs. That's not wrong, but it's incomplete — and at scale, reviewing every output defeats the purpose of having agents.
What works better is designing for verifiable outputs from the start.
Outputs over reports. An agent telling you it wrote an article is a report. The article itself is an output. Wherever possible, build your pipeline so the thing produced is directly inspectable, not just described. This sounds obvious until you're running a multi-agent system where agents hand off to each other and the final output is three steps removed from what the human actually specified.
Acceptance criteria, not just instructions. Agents execute instructions. They don't usually ask whether the result meets the underlying goal. Writing explicit acceptance criteria into a brief — specific, testable conditions — gives you something to check against. "Write a 700-word analysis article" is an instruction. "Write a 700-word analysis article: the word count is between 650 and 750, the slug matches the title, there are no passive-voice constructions in the opening paragraph" is a brief with verifiable criteria.
Stage gates, not end-of-pipeline reviews. Multi-step pipelines that only have a review at the end will reliably propagate errors across every stage. A wrong interpretation in step one compounds through steps two, three, and four. A review gate after step one catches it before it multiplies. This adds friction but it adds less friction than untangling a four-stage mistake.
The log as source of truth
Every serious autonomous operation needs agent logs that are readable and retained.
Not just error logs. Decision logs — what the agent was asked to do, what it decided that meant, what it actually did. This serves two purposes: it lets you catch errors when they happen, and it lets you improve your briefs and prompts over time by seeing where agents consistently misinterpret them.
Felix's real-time dashboards are a version of this: public data on what the agent has shipped, what's sold, what the revenue is. The transparency isn't just marketing — it's a verification layer. Anyone can see what Felix has actually done, not just what it reports.
Most operations won't have public dashboards. But some version of the same principle applies internally: the agent's activity should be inspectable in a form that's independent of the agent's own reporting.
The human's actual job
The framing of "autonomous" can obscure what the human is actually doing in these companies. They're not managing people. They're not doing operations. But they're also not absent.
What they're doing, if they're doing it well, is designing systems with verification built in: briefs with clear acceptance criteria, pipelines with review gates, outputs that are directly inspectable rather than self-reported. They're also doing ongoing prompt improvement — adjusting the instructions agents run on based on where errors actually appear.
This is closer to quality engineering than to management. The question isn't "did the agent do the work?" — it usually did. The question is "did the system produce the right output, and how do I know?"
Companies that have solved this at any real scale — Felix, Kelly Claude, the more sophisticated autonomous operations — have all solved the verification layer first. The revenue comes after the trust infrastructure.
Where the tooling is falling short
Current agent platforms are mostly designed to make agents easy to run, not easy to verify. You get task queues, status tracking, sometimes model outputs saved to a database. What you usually don't get is structured output validation, automatic acceptance criteria checking, or readable decision logs at a level of detail that would let you understand why an agent made a specific interpretation.
This is the tooling gap that matters most for autonomous companies right now. It's also a clear product opportunity — whoever builds genuinely useful agent observability will have a captive market of everyone trying to run operations they can't directly supervise.
Until that tooling exists, the practical answer is to design around the gap: shorter pipelines, more checkpoints, outputs that are inherently inspectable, briefs that specify not just what to do but what done looks like.
Follow along. Tim is building AutonomousHQ live on YouTube — every prompt, every correction, every wrong implementation is on camera. Sign up to the newsletter for weekly updates on the zero-human company experiment. If you're building agents of your own, that's the most honest account of what it actually takes.