The agent demo tells you what a system can do under favorable conditions. The second meeting should establish what it does when the instructions are confusing, the source is wrong, or a person tries to make it cross a boundary. That distinction is moving into procurement. In an October 19 announcement, the American Institute of CPAs (AICPA) and Artificial Intelligence Underwriting Company (AIUC) describe a joint review that pairs a SOC 2 report on organizational controls with tests of how an AI agent behaves. Buyers should not mistake the announcement for proof that any particular agent is safe. They can, however, start asking for the evidence behind the performance claims.
In brief: Ask a vendor to show what the agent was permitted to do, which failure cases were tested, who reviewed the results, and how a customer would catch a failure after deployment. A certification can help. You still have to decide what the agent may touch.
Ask for evidence of agent behavior
A familiar vendor review asks whether the company has policies for protecting data, controlling access, and responding to incidents. Pair those questions with tests of how the agent handles a misleading customer note or an outside document telling it to send data somewhere it should not.
The AICPA and AIUC announcement describes two different kinds of examination. SOC 2 looks at controls around a system, including areas such as security and confidentiality. AIUC-1 adds testing of the agent’s behavior under attempted manipulation, data leakage, and unreliable answers. AIUC says its standard was developed with more than 250 security leaders and is updated quarterly. Those figures describe how the standard is developed and maintained. A product’s effectiveness requires its own evidence.
The partners say a typical AIUC evaluation runs through thousands of scenarios. That sounds substantial. The buying question is still narrower: were the scenarios relevant to your work, your data, and the actions this agent would be allowed to take? A test of a customer-service assistant that only drafts internal replies does not settle the risk of letting the same assistant issue refunds.
Bring your own hard case
Suppose a vendor demonstrates an agent that resolves customer billing disputes. In the demo, it finds an invoice, explains a charge, and writes a polite response. Everyone sees the labor it might save. The buyer should bring an actual category of dispute from their own operation: an invoice that conflicts with a contract amendment, or two customers with similar names and different billing terms. Use a sanitized example if real customer records cannot be shared.
Ask the vendor to run the case while your billing lead watches. Can the agent say it cannot resolve the conflict? Does it show the records behind its answer? Can it draft a recommendation without changing the balance or contacting the customer? If a person rejects the recommendation, where is that correction recorded? The answers tell you more about how the agent would join your team than a smooth demonstration does.
Then ask for the written boundary. What information can it read? What can it write? Which actions require a human to approve them? Who can turn it off, and what happens to work already in progress? Staff need to understand those operating decisions in plain language so they know when to intervene.
At the trial, write down the refusals alongside the completed tasks. If the agent declines to guess at a disputed charge, that may be the most useful result in the room.
The gap will be in the buyer’s practice
The partners say several CPA firms are already delivering the combined service, with a fuller launch expected in early 2027, after initial engagements. Ask what has actually been examined and use the available evidence in your buying decision. Keep the scopes distinct: SOC 2 examines organizational controls, while agent tests examine specified behavior. Each connected system still needs review within the scope of your deployment.
One figure in the announcement needs similar care. The partners say more than 40,000 organizations obtained either a SOC 2 attestation or AIUC-1 certification last year. That figure counts organizations receiving either credential. The number of agents passing the new combined review remains unspecified, as does the status of the vendor in front of you. Use a logo on a slide as a prompt to ask about the scope and findings of that vendor’s review.
You can do this without a combined report. Bring one hard case, a permitted-action list, and a human handoff to each vendor trial. Compare the products against the same job. If you wait for a badge before asking those questions, the agent may already be working in more places than anyone planned.
I would put a second meeting on the calendar before approving a pilot. Bring the process owner, the person who checks the work today, and whoever can decide what records the agent may access. Ask the vendor to show a difficult case, describe the limits of any outside review, and leave you with an account of what happened that your team can inspect. Limit the pilot to preparation and recommendation; keep irreversible customer-facing actions outside its permissions. Expand only after ordinary staff can spot a bad answer and stop it in time.
That is the work behind the agentic organization: someone has to decide who catches the disputed invoice before a customer does.
Research and draft: Mai. Editorial direction: John Lipe.