ACTIVE  ·  BUILDING  ·  v2026.11.20 BUILD 2026-11-20  ·  JL:IOTA:001
No. 112 · 2026-08-02

Twenty-Eight Percent

Johns Hopkins tested every leading AI agent on real clinical work. Only 28 percent of complex tasks finished on the first attempt.

DISPATCH  ·  LOGGED WITH MAI

Johns Hopkins ran frontier AI agents against real clinical workflows last week. Documentation chains, operational tasks, the work staff do every day. Twenty-eight percent of complex tasks completed on a first attempt. Fewer than 8 percent of agents stayed consistent across repeated runs.

Johns Hopkins changed the scope. Teams that broke multi-step processes into single, bounded tasks — one agent, one handoff, one check — got completion rates above 90 percent. The improvement came from narrowing the ask.

That math matters. An agent completing a simple task at 94 percent reliability across hundreds of daily cases produces more than one attempting a twelve-step process and failing seven times out of ten. Reliable completion compounds across the workload.

This tracks with a broader pattern. Ninety-seven percent of executives have deployed agents. Twenty-nine percent report ROI. The gap is scope. Most teams went from vendor demo to production and mistook deployment for results.

The fix is asking a smaller question. One that the agent can actually answer, every time, at production quality. Then asking the next one.

LOGGED WITH MAI  ·  2026-08-02  ·  No. 112
← All Dispatches