The agent demo tells you what a system can do under favorable conditions. The second meeting should establish what it does when the instructions are confusing, the source is wrong, or a person tries to make it cross a boundary. That distinction is moving into procurement. In an October 19 announcement, the American Institute of CPAs (AICPA) and Artificial Intelligence Underwriting Company (AIUC) describe a joint review that pairs a SOC 2 report on organizational controls with tests of how an AI agent behaves. Buyers should not mistake the announcement for proof that any particular agent is safe. They can, however, start asking for the evidence behind the performance claims.

In brief: Ask a vendor to show what the agent was permitted to do, which failure cases were tested, who reviewed the results, and how a customer would catch a failure after deployment. A certification can help. You still have to decide what the agent may touch.

A security questionnaire cannot watch an agent work

A familiar vendor review asks whether the company has policies for protecting data, controlling access, and responding to incidents. Those questions matter. They do not tell you what an agent will do when it reads a misleading customer note and drafts a confident but false reply, or when an outside document tells it to send data somewhere it should not.

The AICPA and AIUC announcement describes two different kinds of examination. SOC 2 looks at controls around a system, including areas such as security and confidentiality. AIUC-1 adds testing of the agent’s behavior under attempted manipulation, data leakage, and unreliable answers. AIUC says its standard was developed with more than 250 security leaders and is updated quarterly. Those figures describe the standard’s development and maintenance, not the effectiveness of a product you might buy.

The partners say a typical AIUC evaluation runs through thousands of scenarios. That sounds substantial. The buying question is still narrower: were the scenarios relevant to your work, your data, and the actions this agent would be allowed to take? A test of a customer-service assistant that only drafts internal replies does not settle the risk of letting the same assistant issue refunds.

Bring your own hard case

Suppose a vendor demonstrates an agent that resolves customer billing disputes. In the demo, it finds an invoice, explains a charge, and writes a polite response. Everyone sees the labor it might save. The buyer should bring an actual category of dispute from their own operation: an invoice that conflicts with a contract amendment, or two customers with similar names and different billing terms. Use a sanitized example if real customer records cannot be shared.

Ask the vendor to run the case while your billing lead watches. Can the agent say it cannot resolve the conflict? Does it show the records behind its answer? Can it draft a recommendation without changing the balance or contacting the customer? If a person rejects the recommendation, where is that correction recorded? The answers tell you more about how the agent would join your team than a smooth demonstration does.

Then ask for the written boundary. What information can it read? What can it write? Which actions require a human to approve them? Who can turn it off, and what happens to work already in progress? These are operating decisions, not technical trivia. If your staff cannot describe the boundary in plain language, they will not know when to intervene.

At the trial, write down the refusals alongside the completed tasks. If the agent declines to guess at a disputed charge, that may be the most useful result in the room.

The gap will be in the buyer’s practice

The new offering does not make certification an instant prerequisite for buying AI. The partners say several CPA firms are already delivering the combined service, with a fuller launch expected in early 2027, after initial engagements. Ask what has actually been examined rather than treating a future certification as present evidence. A SOC 2 report by itself is not a behavioral test of an agent; a successful agent test by itself is not a review of every system it can reach.

One figure in the announcement needs similar care. The partners say more than 40,000 organizations obtained either a SOC 2 attestation or AIUC-1 certification last year. That is not a count of agents that passed the new combined review. It says nothing about the vendor in front of you. A logo on a slide is a starting point for a scope question, not a conclusion.

You can do this without a combined report. Bring one hard case, a permitted-action list, and a human handoff to each vendor trial. Compare the products against the same job. If you wait for a badge before asking those questions, the agent may already be working in more places than anyone planned.

I would put a second meeting on the calendar before approving a pilot. Bring the process owner, the person who checks the work today, and whoever can decide what records the agent may access. Ask the vendor to show a difficult case, describe the limits of any outside review, and leave you with an account of what happened that your team can inspect. Start the pilot with preparation and recommendation, not irreversible customer-facing action. Expand only after ordinary staff can spot a bad answer and stop it in time.

That is the work behind the agentic organization: someone has to decide who catches the disputed invoice before a customer does.

Research and draft: Mai. Editorial direction: John Lipe.