AI Assurance: How to Vet an AI Tool Before You Buy

Every week someone is sold an AI tool. A chatbot, an agent, a voice assistant, some clever automation. The pitch is always strong, and fairly so, because the seller is showing you the capability, the vision, the cutting-edge best case of what the thing can do. That is their job. The problem is what happens next. The dream gets bought before anyone checks whether it survives contact with your actual business. AI assurance is the quiet, unglamorous step in between: making sure the tool is what it says it is, and that you will actually get the value you are being promised, before you sign.

What AI assurance means

Assurance is not a new idea. You already apply it, informally, to most big decisions. You would not hire someone off a great CV alone, or sign a lease without reading it. AI assurance is the same instinct pointed at an AI purchase, and it comes down to two honest questions. First, is the technology sound and does it work the way it is being described? Second, will you realistically capture the value being promised, or is that value mostly in the pitch? Everything below is a way of answering those two questions before the money leaves your account, rather than six months after.

Look under the bonnet

Start with what the tool is actually built on. Plenty of AI products are a clean interface and a well-written instruction sitting on top of a foundation model you could already use yourself. That is not automatically bad, but you should know when you are paying a monthly fee for something your existing chatbot largely already does. Where the tool is genuinely more complex, the questions get more important, not less.

Ask what model sits underneath it and how it has been customised for a business like yours. Ask how it is hosted and where the server actually is. Ask who owns the stack, so you know what you walk away with if you ever leave. And if it is a subscription, get specific on the limits: what exactly is included, what counts as an extra, and what happens to your setup if the underlying AI model changes next quarter. A confident seller answers these plainly. A vague answer is itself an answer.

Where your data actually goes

This one deserves its own space, because it is the question most likely to cause you a real problem later. A large share of AI models are processed on servers in the United States. If you are going to put personal or client data through a tool, you need to know where that data goes, who processes it, and for how long. The right questions are simple: what does the data processing agreement say, what are the retention rules, and where does processing physically happen?

There is a clear way to keep control. Strip the data back so the tool only ever sees what it genuinely needs. Use the enterprise or business tiers that come with no-training terms and a proper data processing agreement rather than the consumer version. Where the data is sensitive, host an open-source model yourself so nothing leaves your control, and pin processing to UK or EU data residency. None of that is exotic. It is the difference between a tool you can defend to your own clients and one you cannot.

Will you actually use it

Here is the harder question, and the one almost nobody asks in the room. Assume the tool works exactly as demonstrated. Will you, realistically, get the value out of it? Look at how people use the AI subscriptions they already pay for. Plenty are using a fraction of what they have, not because the tool is weak, but because nobody ever showed them how to get more. A new tool can go the same way, and an automation that only handles one slice of a process might be capturing a small fraction of the benefit that was actually on the table.

There is a second half to this. Does the thing being sold match what your customers actually want, or what a salesperson has convinced you they want? Website chatbots are the classic case. A lot of people simply do not warm to them, especially for customer service, yet the appetite gets assumed. In a 2025 survey of just over two thousand people, 79 per cent said they would rather deal with a human than an AI agent, and most were comfortable letting a bot handle only simple, routine tasks. It is a United States survey, but the direction of travel is hard to ignore. And watch for the polished wrapper dressed as a bargain: an app promising big-consultancy-grade reports for a small monthly fee, when the report is really a prompt over a model your own subscription already runs. The cost is not the headline price. It is paying for a shiny cover over something you already own.

Two checks you can run today

You do not need to be technical to make a serious start. The first check is a classic audit habit: run the tool through who, what, when, where, why and how. It sounds basic, and that is the point. It pulls you off the shiny front cover and forces you to look at each part of the product or service in its own right.

The second check uses AI against the pitch. Take the chatbot you already use, point it at the seller’s website, tell it plainly what you are being sold and what you want it to do for your business, and ask it what questions you should be putting to yourself and to the seller. It will surface angles you had not thought of. We keep a ready-made version of this as the AI Purchase Assurance Questions prompt, and a companion Wrapper Test prompt for working out whether a tool is a genuine product or just a smart interface.

Why an independent view matters

There is a limit to how far a busy owner can take this alone, and it is not a failing. Think of an aeroplane. Hand most people an airliner and ask them to fly it to Spain and they cannot, not without a great deal of training, manuals and practice. Hand it to a pilot and they are away. AI and automation are genuinely technical, and with automation especially the hard part is not the tool but the process sitting behind it: whether you are automating the right thing, in the right way, to get the right outcome. That is the same reason the audit comes before the build.

An independent view matters because the independent person asks the right questions and has no tool to sell you. My whole career has been built on asking businesses the right questions to make sure the proper risk management controls are in place. That is the value of a second pair of eyes on your side of the table rather than the seller’s: it is the difference between the demo and the reality.

If you only do one thing before you sign, book a short strategy session and bring the pitch with you. Even if all you want is a steer on what to ask the seller, that is a fine use of the time, no strings attached. You can get 30 minutes in the diary here, or email me directly at [email protected].