AI proof of concept: how to scope one that ends in a decision
A seven-step method for scoping an AI proof of concept that ends in a clear go, change course, or stop decision.
BY STEFAN MARITZ··7 MIN READ
TL;DR
- An AI proof of concept is a short, contained test of whether an AI approach can do one specific job well enough to justify a pilot. Its output is a decision: go, change course, or stop.
- Many proofs of concept end without that decision. At least 30% of generative AI projects were expected to be abandoned after the proof of concept stage, mostly over data quality, risk controls, cost, or unclear value.
- Write the decision criteria before you start: the business question, the quality or time threshold that means "go", the cost per task you can afford at scale, and the date you will decide.
- Test on real data with the people who will use it. A demo on clean sample data proves that the model works, and very little about whether your business can use it.
- A successful proof of concept is the start of the adoption work. Name the owner who will run the pilot before the proof of concept begins.
Most organisations that have tried AI have at least one proof of concept that "went well" and then went nowhere. The demo impressed the exec team, everyone agreed it was promising, and six months later nobody can say what was decided.
This post explains what an AI proof of concept is and how it differs from a pilot, why so many stall, and how to scope one so that it ends in a clear decision. It is written for the leader who has to approve the next one and wants it to be the last one that drifts.
What is an AI proof of concept?
An AI proof of concept is a small, time-boxed test that answers one question: can this AI approach do this specific job well enough, at an acceptable cost, to justify testing it with real users? It is deliberately narrow. It tests feasibility and leaves adoption for the next stage.
The terms get used loosely, so it helps to separate the three stages most projects pass through.
| Stage | Question it answers | Who uses it | What success looks like |
|---|---|---|---|
| Proof of concept | Can AI do this job well enough, at a cost we can carry? | The project team and two or three expert users | Pre-agreed quality and cost thresholds met on real data |
| Pilot | Will people use it in their daily work, and does it change the numbers? | One team, doing their normal work | The workflow has changed and the business measure has moved |
| Production | Can we run it reliably for everyone it applies to? | Everyone in scope | Stable, supported, monitored, and part of how the work is done |
Why do so many AI proofs of concept go nowhere?
At least 30% of generative AI projects were expected to be abandoned after proof of concept by the end of 2025, because of poor data quality, inadequate risk controls, escalating costs, or unclear business value. Only 22% of companies have moved beyond the proof-of-concept stage at all. The pattern is repeating with AI agents, systems that take actions on their own across several steps: over 40% of agentic AI projects are expected to be cancelled by the end of 2027.
Every one of those causes can be checked before the proof of concept starts. Unclear value means nobody wrote down what success was worth. Escalating cost means nobody estimated cost per task at full volume. Data quality problems show up the moment you use real data instead of a tidy sample. The wider pattern behind why AI pilots fail is the same, and most of it is visible at the scoping stage.
The other cause is softer. A proof of concept with no named owner has no one whose job it is to make the decision, so it defaults to "promising" and drifts.
How to scope an AI proof of concept
Scope it the way an engineer scopes a load test on a new component: define the load, the pass mark, and what happens if it fails, all before the test runs.
- Write one business question. "Can we draft first-response emails for the 12 most common customer queries to a standard our senior agent would send with minor edits?" is testable. "Can AI help customer service?" is not.
- Set the pass mark in advance. Agree the quality threshold (for example, the share of outputs an expert accepts with light edits), the time saved per task, and the error types that are unacceptable at any rate.
- Estimate cost at full volume. Price the model usage, integration, and human review per task at the volume you would run in production. A proof of concept that only works at demo scale has answered the wrong question.
- Use real data, including the messy cases. Pull a sample of actual inputs from the last few months, with the awkward ones left in. The edge cases are where the decision gets made.
- Involve two or three people who do the work. They judge quality far better than the project team, and their early involvement is the first step of adoption.
- Agree the three possible outcomes. Go to pilot, change course (a different approach, narrower scope, or better data first), or stop. All three are useful results.
- Name the pilot owner and the decision date. The person who will run the pilot if the answer is "go" should help design the proof of concept. Put the decision meeting in the diary on day one.
What should an AI proof of concept measure?
Measure performance on the job itself. Model benchmarks tell you how a system performs on someone else's test, and your proof of concept should tell you how it performs on your work.
- Acceptance rate: the share of outputs a qualified person would use with light or no edits.
- Time per task: measured against how long the same task takes today, timed rather than estimated.
- Serious error rate: outputs that would cause harm if they went out unchecked, counted separately from minor flaws.
- Cost per task: all-in, including the human review step.
Accuracy on its own can mislead. A system that is right 90% of the time may still be a poor fit if checking every output takes as long as doing the work, which is why time and review cost belong in the same table. These measures also feed straight into an honest view of AI return on investment later.
What happens after a successful proof of concept?
A pilot, run by the owner you named at the start, with one team doing its normal work. This is where most of the effort moves from technology to people: changing the process, training people on their own tasks, and deciding what they stop doing. It is the part of AI change management that decides whether the result survives.
If the answer is "change course", write down why in one paragraph and decide the next test. If the answer is "stop", say so plainly and record what you learnt. Stopping a proof of concept that was set up properly is a good outcome, because it cost weeks instead of a year.
Frequently asked questions
What is an AI proof of concept?
An AI proof of concept is a short, contained test of whether an AI approach can do one specific job well enough, at an acceptable cost, to justify a pilot. It should end in a clear decision to go, change course, or stop.
How long should an AI proof of concept take?
In our view, a few weeks is usually enough when the question is narrow and the data is available. If it needs months, the scope is too broad or the data work has not been done yet.
What is the difference between a proof of concept and a pilot?
A proof of concept tests whether the AI can do the job. A pilot tests whether people will use it in their real work and whether the business measure moves.
Why do AI proofs of concept fail?
The common causes are poor data quality, weak risk controls, rising costs, and unclear business value. Most of these can be spotted before the work starts by writing down the success criteria and cost per task in advance.
Should we build the proof of concept ourselves or use a vendor?
Either can work. What matters is that your team owns the business question, the pass mark, and the data, so the decision stays yours whoever builds it.
If the right answer turns out to be a tool built around your workflow rather than an off-the-shelf product, that is the work our custom-built AI-native systems service does.
WORK WITH PRAXES
Ready to move from reading about AI to getting your organisation ready for it? Start with the diagnostic.
Take the AI Readiness Diagnostic