Anthropic and Accenture announced on September 18 that they will each invest at least $1 billion over five years to build embedded AI evaluation. Faculty, Accenture’s AI business, will work inside Anthropic to test models, review safeguards, and assess how systems behave before and during release. That sounds like a frontier lab story. It is also a business operations story.
The lesson is not that every company needs a billion-dollar safety program. The lesson is that AI oversight is moving closer to the work. If AI is making decisions, drafting customer responses, touching internal systems, or guiding employees through workflows, review cannot live in a policy document that nobody opens. It needs to be part of how the system runs.
The audit comes too late
Most companies still treat AI risk as something to review after the pilot. A team experiments with a tool. Someone writes acceptable-use rules. Legal checks the vendor terms. IT approves access. Then the system gets pushed into a workflow and everybody hopes the review process catches problems later.
That model was already weak for simple AI tools. It gets worse with agents.
An agent does not just produce a paragraph. It can take a goal, read context, call tools, update records, trigger messages, and hand work back to a person who may assume the output is ready. The risk is not only that it says something wrong. The risk is that it behaves plausibly enough that nobody slows down to inspect the path it took.
Anthropic described embedded evaluators as outside specialists with access closer to an employee’s. That matters. They can watch models take shape, follow the decisions around how systems are built and deployed, report incidents, and identify blind spots. In plain business language: they are not auditing the finished brochure. They are sitting inside the operating room.
That is the part leaders should pay attention to.
Independence is not a vendor checkbox
There is a fair criticism here. Accenture is not a nonprofit safety lab. Anthropic is funding the work directly because, as Anthropic said, there is no settled funding system yet for independent evaluation. TechCrunch noted that many AI watchers expected names like METR, Redwood Research, or Apollo Research before a consulting giant.
That tension is real. Independence is not magic because a third party has a logo on the slide.
But it would be a mistake to miss the operating pattern because the first version is imperfect. The useful idea is embedded scrutiny. Someone close enough to see the work, separate enough to challenge it, and responsible enough to report what they find. That is different from the way many organizations handle AI today, where the person deploying the workflow is also the person deciding whether the workflow is safe enough to use.
Small and mid-sized companies can copy the pattern without copying the structure. They do not need an embedded evaluator. They need an owner who is not the builder.
If marketing builds an AI workflow for outbound emails, marketing should not be the only reviewer. If operations deploys an agent to triage support tickets, operations should not be the only team deciding whether the agent is routing correctly. If finance uses AI to flag invoice anomalies, the person measuring accuracy should not be the same person who is trying to prove the tool was worth buying.
Separation of duties sounds boring. Boring is good here.
What this looks like without the billion-dollar budget
A practical version has four parts.
First, name the workflow. Not the tool. The workflow. “We use AI in customer support” is too vague. “AI drafts first-response emails for tier-one support tickets before human review” is specific enough to manage.
Second, assign two owners. One owns performance. One owns trust. Performance asks whether the workflow saves time, improves throughput, or reduces cost. Trust asks whether it introduces errors, exposes data, confuses employees, or creates customer risk. Those should be separate responsibilities, even if both people sit on the same team.
Third, review the path, not just the output. Did the AI use the right source? Did it skip a required check? Did it escalate the right cases? Did the human reviewer understand what they were approving? This is where most lightweight AI policies fail. They judge the answer, but the business risk often lives in how the answer was produced.
Fourth, create a stop rule. If error rates exceed a defined threshold, if a sensitive category appears, if employees start bypassing review, or if the workflow touches a system it was not approved to touch, the process pauses. Not forever. Long enough for someone accountable to inspect it.
That is not heavy governance. That is basic operational hygiene.
The real shift is where trust lives
The old AI adoption question was, “Can this tool do the task?” The better question now is, “Can we operate this safely when it becomes part of daily work?”
That question changes who needs to be in the room. Not only IT. Not only legal. Not only the business owner excited about the workflow. You need the person who understands the work, the person who understands the risk, and the person with authority to pause the system when the signal looks wrong.
MIT’s 2025 State of AI in Business research found that only about 5 percent of enterprise generative AI pilots delivered rapid revenue acceleration, while the rest produced little measurable financial impact. That failure rate is usually discussed as an ROI problem. It is also an ownership problem. Companies keep testing AI capability without designing the operating system around it.
Anthropic and Accenture are working at the frontier, but the business lesson is ordinary: AI does not become safer because a policy exists. It becomes safer when review is built into the workflow, close enough to see what is happening and independent enough to say no.
For most leaders, the first step is simple. Pick one AI workflow that already matters. Name the person responsible for performance. Name a different person responsible for trust. Decide what would make you pause it.
If you cannot answer those three things, you do not have an AI workflow yet. You have an experiment running inside the business.