Three questions before you greenlight an AI project

There is no shortage of AI project ideas. Every team has a backlog of things that “could be AI-powered.” The problem is not generating ideas. The problem is picking the ones that will actually ship, actually work, and actually matter — versus the ones that will consume six months of engineering time and produce a demo that nobody uses.

After watching dozens of AI projects succeed and fail, we’ve landed on three questions that predict the outcome. Not perfectly — nothing predicts perfectly — but reliably enough that we won’t greenlight a project until the team can answer all three.

If you can answer them, build. If you can’t, stop and figure out why before you spend the money.

Question 1: Can you measure the baseline today?

This is the question that kills the most projects — not because the answer is no, but because nobody bothers to ask.

The baseline is the current performance of the process you’re trying to improve. How long does it take? How much does it cost? What’s the error rate? How many units flow through per week? You need these numbers before you build anything, because without them you cannot prove that AI made it better.

This sounds obvious. It is not practiced.

Here’s what happens when teams skip it. They build an AI system. It works. Someone asks “how much did this improve things?” The team says “it’s faster.” The executive asks “how much faster?” The team says “significantly faster.” The executive asks “compared to what?” And nobody has an answer, because nobody measured the old process before they replaced it.

Now you’ve built something that might be brilliant and you can’t prove it. You can’t calculate ROI. You can’t justify the ongoing cost. You can’t make the case for expanding it to other use cases. You have an AI system running in production and a gut feeling that it’s helping.

Gut feelings don’t survive budget season.

The pattern when this goes wrong. A fintech company we spoke with built an AI system to review loan applications. It worked well — the team was confident it was faster and more consistent than the manual process. But they hadn’t measured the manual process before they built the AI. When the CFO asked for the ROI, they had to estimate the old baseline from memory. Their estimate was contested. The project lost its budget expansion because the numbers weren’t defensible. The AI worked. The business case didn’t.

What to do. Before you write a line of code, spend one week measuring the current process. Time it. Count the errors. Calculate the cost per unit. Document it. This is boring work. It is also the work that makes everything else possible. A week of measurement pays for itself a hundred times over when someone asks “was this worth it?”

If you genuinely cannot measure the current process — if there’s no way to quantify what you’re improving — that’s a signal. It means either the process is too informal to measure (fix that first) or the value of AI is too diffuse to capture (pick a different project). Either way, you’re not ready to build.

Question 2: Who owns this in production?

This is the question that determines whether your project ships or becomes a permanent prototype.

When we ask this question, the most common answer is “the AI team.” This is the wrong answer. The AI team builds AI systems. They do not — and should not — operate every AI system in the company. If the AI team owns every production AI system, you’ve created a bottleneck that doesn’t scale and a team that spends all its time on operations instead of building new things.

The right owner is the team that owns the process the AI is improving. If AI is classifying support tickets, the support operations team owns it. If AI is extracting data from invoices, the finance operations team owns it. The AI team builds it, trains the operations team, hands it over, and moves on.

This sounds simple. It is organizationally hard. The operations team didn’t ask for an AI system. They don’t understand how it works. They don’t know how to debug it when it fails. Handing them a model and saying “you own this now” is a recipe for failure.

The pattern when this goes wrong. A logistics company built an AI system to optimize route planning. The AI team built it, tuned it, and got great results. Then they tried to hand it off to the dispatch team. The dispatch team had no idea how the model made decisions. When the model suggested a route that seemed wrong, they had no way to evaluate whether the model was right and their intuition was wrong, or vice versa. They stopped using it within a month. The AI team had to take it back and operate it themselves — while also trying to build the next project. Within six months, the AI team was spending 60% of its time operating old projects and 40% building new ones. Velocity collapsed.

What to do. Name the production owner before the project starts. Involve them from day one — not as a stakeholder who gets updates, but as a team member who sees how the system is built, understands the failure modes, and helps define the monitoring and escalation procedures. Build the runbook together. When the system goes live, the production owner should feel like they helped build it — because they did.

If no team is willing to own the system in production, that’s a signal. It means either the system isn’t valuable enough for anyone to care about, or the organizational structure doesn’t support AI adoption. Both are worth knowing before you spend the engineering time.

Question 3: What happens when you turn it off?

This is the question that separates real value from theater.

Imagine the AI system has been running for three months. Now imagine you turn it off. What happens?

If the answer is “nothing” — nobody notices, no process breaks, no metric changes — the system wasn’t creating value. It was creating activity. Activity feels productive. It is not the same as value.

If the answer is “people notice within a day, and the team has to scramble to cover the gap” — that’s real value. The system is doing something that matters. It’s embedded in a workflow. People depend on it.

This question works as a filter at every stage. Before you build: “If we built this and then turned it off, would anyone care?” During the pilot: “If we stopped the pilot tomorrow, would the pilot users fight to get it back?” After launch: “If this went down for 24 hours, what’s the impact?”

The pattern when this goes wrong. A media company built an AI system to generate content summaries. The system worked. It produced summaries. The summaries were fine. But when we asked “what happens if you turn it off,” the answer was revealing: “The editors would just write the summaries themselves. They did it before and it took about 5 minutes each.” The AI system was saving 5 minutes per article on a task that produced 3 articles per day. That’s 15 minutes of daily savings. The system cost $2,400/month in compute and API fees, plus engineering time for maintenance. The math didn’t work. The system was technically successful and economically pointless.

What to do. Ask this question honestly before you start. Not “would it be nice to have this?” — everyone says yes to that. Ask “if we built it and then removed it, what would break?” If nothing breaks, the use case is too thin. If the answer is “we’d need to hire two people to cover the gap” — now you have a business case.

The turn-it-off test also reveals dependency risk. If the answer is “everything breaks and we have no fallback,” you need to build with more redundancy. The ideal answer is “things would get worse in a measurable way, and we have a manual fallback that’s painful but functional.” That’s a system creating real value with a manageable risk profile.

Using the three questions together

The three questions work as a filter. Run every proposed AI project through them.

Can you measure the baseline? If no, measure first or pick a different project.

Who owns this in production? If nobody, solve the ownership problem first or pick a different project.

What happens when you turn it off? If nothing, pick a different project.

Projects that pass all three tend to ship, tend to work, and tend to justify their cost. Not because the questions are magic — but because they force the team to answer the hard organizational questions before they start writing code.

Most AI projects don’t fail because the technology doesn’t work. They fail because nobody measured the baseline, nobody owned the production system, or the thing they built didn’t matter enough for anyone to miss it. These are all knowable in advance. You just have to ask.

tl;dr

The pattern. Teams greenlight AI projects without answering basic questions about measurement, ownership, and value — then wonder why the projects stall or get cut. The fix. Before committing resources, confirm you can measure the baseline, name a production owner, and articulate what breaks if the system is turned off. The outcome. You only build AI projects that can prove their impact, ship to production, and survive budget season.