Scale, fix or retire your AI agents: how to justify continued AI investment

It's budget season, and the AI team has twelve agents in production across HR, IT, sales and legal. This is an illustrative example, not a client. Usage is up and, on average, the program looks healthy. Then finance asks for a 20% cut, and the team realizes it can't say which agents produce the value.

That's the question this article answers: how to decide, agent by agent, where AI budget should go next.

How do enterprises justify continued investment in AI agents?

One agent at a time. For each agent, compare the value it produces with what it costs, and check whether people actually use it. That gives a clear verdict for every agent: scale it, fix it or retire it. Finance can support a set of evidenced decisions far more easily than one blended ROI figure for the whole program.

Why a program-wide ROI number stops being enough

A single figure for the whole program averages strong agents and weak ones together. When budgets tighten, an average gives you nothing to protect. A flat cut lands on every agent, including the ones producing most of the value.

Per-agent evidence changes the conversation. Instead of defending the program, the AI team proposes where money should move.

What metrics show an AI agent is delivering value?

Four signals, collected the same way for every agent:

• Demand. How many of the intended users come back to the agent each month. Repeat use is the measure, because logins and seat counts overstate it.

• Task success. The share of conversations where the agent actually completed the job. Implicit feedback is the best guide here: behavior such as rephrasing, correcting or abandoning that shows how an interaction went without the user rating it.

• Value. For employee agents, time saved on successful tasks, converted at net hourly cost (the employee's hourly cost, net of what the AI costs to run). Customer-facing agents need their own value rule agreed with finance, because revenue influenced is not margin and can't be set against cost directly.

• Cost of ownership. What the agent costs to own each month beyond running it: licenses, upkeep, and the people who support it. Running cost is already netted out of value, so it isn't counted again here.

How to decide: scale, fix or retire

The decision rests on two questions. Does the agent return more than it costs? And do people want it?

Demand is what separates fix from retire. A weak agent that people keep trying to use is telling you there is a real need, and showing you where it falls short. A weak agent nobody uses offers neither.

One case sits outside the table: an agent that returns well above its cost but serves a small specialist team. Low usage doesn't make it a bad investment. Keep it, and judge it on value per user rather than total usage.

There are no universal benchmarks, so set your own

No industry standard says an agent should reach a certain success rate or return a certain multiple of its cost. Use cases differ too much for that. So the rules have to rest on lines you can defend without a benchmark.

Two lines do that. Break-even is the first: an agent either returns more than it costs to own, or it doesn't. The second is a demand target you set when the agent launches, expressed as the share of its intended users you expect to come back each month. Checked in this order, they give every agent exactly one verdict:

1. Fix when value is below cost, or has fallen for two quarters, and demand is at or above target.

2. Retire when value is below cost, demand is below target, and one round of fixes hasn't moved either. If no fix has been tried yet, treat it as fix.

3. Scale when value has been above cost and rising for two consecutive quarters, with demand at or above target.

4. Hold in every other case where value is above cost.

Write the demand targets down at launch, agree the rules with finance before the first review, and apply them the same way to every agent.

Give new agents a realistic ramp

An agent's first months rarely reflect what it will return. Gartner's September 2026 survey of 160 finance leaders found that even simple finance AI use cases, such as data extraction, accounts payable and receivable automation, and report creation, generally deliver returns within nine to 10 months. More complex use cases take longer.

Agents still inside their ramp aren't eligible for retire. Until they have had that time, the question is whether demand and task success are moving in the right direction, not whether value already covers cost.

An example portfolio review

The figures below are illustrative. Value is time saved multiplied by a net hourly cost of $75. Each agent launched with a demand target of 30% of its intended users returning each month.

IT triage and proposal drafting both return several times their cost, have grown for two quarters, and beat their demand targets, so they get more budget. The HR assistant has the most returning users but returns less than it costs. Retiring it would waste clear demand, and its low success rate points to the fix: the policy content it answers from. The legal clause checker is past its ramp, has already had a round of fixes, and still has neither demand nor value, so its budget moves to ticket triage.

Who owns the review, and how often

• The AI leader owns the portfolio and proposes each verdict.

• Finance agrees the thresholds and the value assumptions, ideally before the first review.

• Use case owners explain the context behind each agent's numbers and own the fix plans.

Run the review every quarter. Monthly is too noisy for most agents, and once a year is too slow to move money to where it works.

How to prove your AI investment is working in the budget review

Proof is evidence finance can check, agent by agent. Bring:

1. One line per agent with its verdict.

2. The four signals behind each verdict: demand, task success, value and cost of ownership.

3. The thresholds and assumptions you agreed in advance.

4. The trend for each agent across the last few quarters.

5. The reallocation: what moves from retired agents to the ones you are scaling.

This turns a defensive conversation about the whole program into decisions finance can support, agent by agent. For how to lay the numbers out, see building an AI agents value dashboard.

Evidence for every agent

A scale, fix or retire decision is only as good as the evidence behind it. Nebuly gives AI leaders that evidence for every agent, from the conversations themselves: who keeps coming back, which tasks get completed, and what each agent returns in time saved or revenue influenced. The portfolio review becomes a set of decisions you can defend, one agent at a time.

Know which agents to scale, fix, or retire before your next budget review: book a demo.