TL;DR: Most agentic AI vendors sell autonomy, not an exit. Forrester's 2026 research found that 22% of production AI agents deliver negative ROI after twelve months, and 41% of those failures trace back to one root cause: nobody defined success before the contract got signed. Before you sign anything, run the vendor through the same test I ran systems through in the engine room: what happens when it fails, who gets called, and how fast can you roll back. This piece hands you the full checklist, built on what I call the Sovereignty Stack.
- Forrester: 22% of production AI agents show negative ROI at 12 months, and 41% of those failures trace to undefined success criteria before launch.
- Gartner projects more than 40% of agentic AI projects will be canceled by 2027, and more than half of enterprise generative AI projects never reach production.
- ISO 42001 is becoming a procurement checkpoint for AI vendors, and certification runs EUR 20,000 to 50,000 for a small or mid-sized company.
- The Sovereignty Stack gives you four questions to ask before you sign: who owns the data, who can explain the process, how fast can you exit, and what does year two actually cost.
The CASREP Test
In the Navy, we never deployed a new system without a casualty report procedure attached to it. A CASREP is not paperwork. It is the answer to three questions asked before the system goes live: what happens when it fails, who gets called, and how fast can you roll back.
I run every AI vendor pitch through the same three questions now. If the vendor cannot answer all three without hedging, they are not ready for my production floor. Most cannot. That gap is exactly where the failure statistics below come from.
I sat through a vendor pitch not long ago for an agentic tool built to run outbound sales sequences. The founder walked through the model, the roadmap, and the integrations in detail. When I asked what happens if the agent sends the wrong offer to five hundred leads overnight, he paused. That pause told me everything the deck could not.
Why the Fleet Is Taking Casualties
The market is moving fast enough that due diligence keeps getting skipped. Tracxn counts 1,426 agentic AI companies globally, with a record 267 founded in 2024 alone, and $8.64 billion in funding flowing into the category in 2026 year to date. That is a lot of vendors pitching autonomy with very little operating history behind the pitch. Speed to market is not the same thing as readiness for your production floor.
The failure data backs up the caution. Gartner projects that more than 40% of agentic AI projects will be canceled by 2027, and more than half of enterprise generative AI projects already fail to reach production. A 2026 WRITER survey found 97% of companies had deployed AI agents, but only 23% reported significant ROI. Proving the return, not deploying the agent, is the hard part now.
Forty-one percent of the negative-ROI cases Forrester tracked came down to one failure: nobody wrote down what success looked like before the system went live. Another share failed because the agent never got real access to the tools and data it needed to do the job. Both are diligence failures, not technology failures. You cannot patch a contract after the fact.
None of this means agentic AI is a bad bet. It means the category is early, the way any market with 267 new entrants in a single year is early. Early markets reward operators who do the diligence work nobody else wants to do. That work is boring, and boring work is exactly where the edge lives.
The Sovereignty Stack
I built the Sovereignty Stack after watching too many owner-operators sign vendor contracts that traded control for convenience. It has four layers, and you check all four before you check a single feature. Skip a layer and you find out the hard way, usually during a renewal negotiation with no use.
1. Data Sovereignty
Who owns the training data, the interaction logs, and the outputs the agent generates? If the vendor's contract is silent on this, assume they own it and you rent it back. Ask for the data export format in writing before you ask about pricing.
I have seen a vendor's terms of service quietly claim ownership of all customer interaction data generated on the platform. A founder discovered it only when he tried to export a full customer list during a migration. The export button existed. The full dataset did not.
2. Process Sovereignty
Can you document exactly how the agent makes a decision, or is it a black box you are trusting on faith? A vendor that cannot produce a decision log is asking you to operate blind. That is not automation. That is delegation without oversight.
A watchstander who cannot explain a casualty to the officer of the deck gets relieved of the watch. The same standard should apply here. If your AI agent cannot explain itself to you in plain language, it should not be standing watch over your revenue.
3. Exit Sovereignty
How long does it take to migrate off this platform if the relationship ends? Get the number in writing, then double it. Every vendor underestimates this, and most owner-operators never ask the question until they need the answer.
One founder I know budgeted two weeks for a platform migration. It took four months, because the vendor's export tool broke on datasets over a certain size and support tickets sat unanswered for a week at a stretch. Budget for the vendor's worst case, not their best one.
4. Cost Sovereignty
What happens to the price after year one, and what happens if usage triples? Renewal pricing is where vendors make their real margin. Model it before you sign, not after the invoice arrives.
Usage-based pricing is the newest version of an old trick. The demo runs on a small dataset at a small price. Production runs on your full customer base, and the invoice arrives with a multiplier nobody mentioned on the sales call.
The Checklist
Run every vendor through this before a dollar changes hands. Print it. Bring it to the sales call.
- Ask for the CASREP answer: what happens when the agent fails, who gets called, and how fast can you roll back.
- Ask for the decision log. If the agent cannot produce a plain-language explanation of a decision it made, treat that as a process sovereignty gap.
- Ask for the exit clause in writing: data export timeline, export format, and any cost attached to leaving.
- Ask how success will be measured, in writing, before the contract is signed. This one step would have prevented 41% of the negative-ROI cases Forrester tracked.
- Ask whether the vendor holds or is pursuing ISO 42001 certification, and watch how confidently they answer.
- Ask for three reference customers past the 12-month mark, not the 90-day honeymoon mark.
The Sales Call Script
Diligence only works if you actually say the words out loud on the call, not just think them. Open with this: what happens the first time this agent makes a decision that costs us money, who finds out, and how fast do we know. Watch the response time, not just the content.
A vendor with a real answer will name a specific monitoring system, a specific escalation path, and a specific rollback window, usually in one breath. A vendor without one will pivot to a feature you did not ask about. That pivot is data. Write it down and compare notes after every call, because the pattern across five vendor calls tells you more than any single call ever will.
After You Sign: Watchstanding
Diligence does not end at signature. On a ship, watchstanding never stops, and neither should your oversight of an agent making decisions on your behalf. Set a recurring review, weekly for the first month and monthly after that, where you check the agent's decision log against what actually happened.
This is not paranoia. It is the same discipline that keeps a watch team sharp after months without an incident. The moment you stop checking the log is the moment the failure you did not plan for finds you first. Thirty minutes a month is a small premium against a bad quarter.
ISO 42001 and the New Procurement Perimeter
ISO 42001 is the AI management system standard, and it is turning into the procurement gate that SOC 2 became for software a decade ago. Certification runs EUR 20,000 to 50,000 for a small or mid-sized vendor, which is real money for an early-stage company. Enterprise buyers are starting to treat it as a hard requirement in 2026, not a nice-to-have. If your vendor balks at the question, that tells you where they rank the standard internally.
You do not need to gatekeep on certification alone if you are a five-person shop buying a $200-a-month tool. But you should ask the question and watch the answer. A vendor who has never heard of ISO 42001 is a vendor who has not thought about governance at all. That is useful information, and it belongs in your risk assessment, not just your budget.
Doctrine Connection: Due diligence is non-negotiable. Every operator I respect runs the same discipline on a vendor contract that they would run on a hire, a lease, or a capital raise. Agentic AI does not get a pass because it is new and the pitch decks are polished. You are not buying software anymore: you are onboarding a decision-maker into your operation, and decision-makers get vetted before they get access.
Frequently Asked Questions
What is the single biggest AI vendor red flag?
A vendor who cannot answer the CASREP question in one sentence. If they need a demo to explain their own failure mode, they have not thought about it enough, and neither has their engineering team.
Do I need ISO 42001 certification from every AI vendor I use?
Not necessarily, especially for a small tool with low blast radius. But you should ask whether the vendor holds it or is pursuing it, and treat a confident, specific answer as a positive signal about their governance maturity.
How is the Sovereignty Stack different from a standard vendor scorecard?
A standard scorecard grades features and price. The Sovereignty Stack grades control: who owns your data, who can explain the agent's decisions, how fast you can leave, and what the real cost looks like in year two.
How long should exit planning take before I sign a contract?
Get the vendor's stated migration timeline in writing, then plan your own diligence around double that number. Nearly every vendor underestimates exit time, and you want that buffer built in before you ever need it.
Jeff Barnes is the founder of Digital Evolution Marketing Group and Angel Investors Network. DEMG provides marketing systems and AI operations consulting for owner-operators. This article reflects operational experience and publicly available data. It is not financial, legal, or investment advice. Tools and platforms mentioned are not sponsored endorsements. Verify all claims, run your own numbers, and consult qualified professionals before acting.
