McKinsey estimates the long-term productivity potential of AI at $4.4 trillion annually across enterprise use cases. The gap between that potential and what most organizations are actually realizing is not a technology problem.

According to McKinsey's 2025 State of AI survey, 88% of organizations use AI in at least one business function. Only around one-third report scaling AI across the organization. Only 6% qualify as AI high performers, defining that as attributing more than 5% of EBIT to AI use. The difference between the 88% and the 6% is not which models they chose or how much they spent. It is how systematically they measure outcomes rather than activity.

This article sets out a practical measurement framework for enterprise leaders. Not a theoretical model, but the specific metrics that connect AI activity to business results, organized by use case type, with guidance on what to measure first and how to build the measurement infrastructure that makes ROI visible over time.

Why activity metrics fail as a measurement approach

Session counts, prompt volume, and active user rates are easy to collect and easy to report. They are not measures of business value. They measure whether the tool is being used, not whether it is delivering anything.

An internal AI agent can show 10,000 sessions per month while employees use it only for drafting emails and avoid it for any substantive work. A customer-facing AI agent can achieve a 90% containment rate while failing to resolve most of the queries it contains. Both look healthy on an activity dashboard. Neither is delivering the business value that justified the investment.

The shift from activity measurement to outcome measurement requires defining what you are actually trying to achieve with each AI deployment, and then tracking whether you are achieving it. That definition varies by use case, and using the wrong metrics for a given use case produces a misleading picture of performance.

Measuring internal productivity AI agents

Internal AI agents, those used by employees for productivity support, knowledge retrieval, content drafting, and workflow assistance, should be measured on the productivity outcomes they produce.

Hours saved per department. The foundational metric for internal AI ROI. Establishing a pre-deployment baseline is essential: how long do the target tasks currently take without AI assistance? After deployment, tracking how long those same tasks take with AI assistance, and multiplying the difference by the number of employees using the agent, produces the hours saved figure. This should be tracked by department and role, not averaged across the organization, because gains are rarely uniform.

The McKinsey customer operations study found that AI agents increased issue resolution rates by 14% per hour and reduced handling time by 9%. These gains were visible because the teams measured against pre-deployment baselines. Without a baseline, the same improvement would produce no measurable number.

Task completion rate. The proportion of employee AI interactions that result in the task being accomplished without abandoning the session, escalating to a colleague, or reverting to a manual alternative. A high task completion rate indicates the agent is genuinely useful for the workflows it targets. A low rate indicates a mismatch between what the agent does well and what employees actually need from it.

Return rate and session depth. Whether employees come back after their first interaction, and whether the complexity of their usage increases over time. Employees who find genuine value in an AI agent use it for progressively more demanding work. Employees who do not return, or whose usage stays shallow, are showing disengagement signals that precede the silent adoption failure we have described elsewhere.

AI proficiency by team. Whether employees are applying the AI agent to high-value tasks over time, or limiting it to low-stakes use. Proficiency growth indicates the agent is genuinely changing how work gets done. Flat or declining proficiency indicates nominal adoption without real embedding.

Override rate by task category. How often employees edit, reject, or ignore the agent's output before acting on it. A high override rate in a specific task category is a direct signal that the agent is underperforming on that task for that group of users, pointing precisely to where improvement effort should focus.

Measuring customer-facing AI agents

Customer-facing AI agents, those that interact with customers through support, sales, and service channels, require a commercial measurement framework that connects AI interactions to revenue outcomes.

Intent resolution rate. Whether the AI agent addressed the customer's actual goal, not just whether it produced a response. This is the closest available metric to "did this interaction deliver value for the customer?" A customer who asked about a billing dispute and received a policy explanation rather than a resolution encountered a technically successful interaction and an unresolved business outcome. Tracking intent resolution by interaction category reveals where the agent genuinely serves customers and where it produces volume without value.

Churn signals by interaction category. Competitive mentions, repeated unresolved frustrations, price or value challenges, and mid-conversation escalation requests are business-level signals that appear in AI agent conversations before they appear in CRM data or renewal pipelines. Organizations that track these patterns can identify at-risk customers weeks before traditional retention signals appear.

Revenue influence. The proportion of customer interactions that contain upsell or expansion signals, and whether those signals are handled in ways that support commercial outcomes. Customers who ask about higher-tier features, express frustration at plan limitations, or compare current pricing to alternatives inside an AI conversation are expressing commercial intent. Whether that intent is addressed or deflected is measurable and has direct revenue implications.

Escalation timing, not just escalation rate. The point in a conversation at which customers request human assistance matters as much as how often they do. Late escalation, after multiple failed AI interactions, consistently damages customer trust more than early escalation that routes customers appropriately from the start. Tracking when escalations occur, and what preceded them, reveals whether the agent is functioning as an effective triage layer or as a barrier between customers and resolution.

Containment vs. resolution. Containment rate measures interactions that stayed within the AI agent without escalating. Resolution rate measures whether the customer accomplished what they came for. A customer who abandons a conversation mid-session is contained but not served. High containment rates with low resolution rates indicate a deployment that is reducing support costs without improving customer outcomes, which is a risk to retention that cost metrics will not surface.

Connecting both layers to business outcomes

The most useful ROI measurement connects specific AI agent metrics to the business outcomes they influence.

For internal agents, the connection is direct: hours saved translates to productivity gain, which translates to cost reduction or capacity increase. An organization that saves 2,000 hours per month across a department is either reducing labor costs, increasing output with the same headcount, or enabling employees to focus on higher-value work. All three are quantifiable.

For customer-facing agents, the connection runs through retention and revenue. Reduced escalation rates reduce support costs. Higher intent resolution rates reduce churn risk. Identified and well-handled upsell signals increase expansion revenue. Each of these connections can be quantified by correlating AI agent interaction patterns with downstream business data: CRM outcomes, renewal rates, support cost per interaction, and revenue per customer segment.

The Iveco case study is instructive here. When Iveco deployed an internal AI copilot to help employees retrieve procedures and documentation, the ROI was not visible in session counts. It became visible when Nebuly's analytics produced more than 100 times more feedback data than manual review had generated, making it possible to see precisely where the agent was saving time, where it was not, and what that meant for the productivity baseline they had established before deployment.

Building the measurement infrastructure

Measurement infrastructure needs to be established before deployment, not retrofitted after it.

Three steps make this practical. First, define the baseline: for internal agents, how long do the target tasks currently take? For customer-facing agents, what are the current escalation rates, resolution rates, and customer satisfaction scores? Without baselines, post-deployment improvement is visible only in relative terms, which makes ROI reporting to leadership approximate rather than evidenced.

Second, instrument for behavioral signals from day one. Session counts are captured automatically by most platforms. Behavioral signals, task completion, return rate, session depth, override rate, intent resolution, escalation timing, require deliberate instrumentation. The organizations that have this data at 90 days are the ones that can make evidence-based investment decisions. Those that do not are still relying on impressions.

Third, connect AI metrics to business data on a regular cadence. A monthly review that compares AI performance metrics to CRM outcomes, support cost data, and productivity baselines is what keeps measurement operational rather than theoretical. This is the practice that separates the 6% of organizations seeing real EBIT impact from the majority that are still in the deployment stage without a clear picture of what the investment is worth.

Nebuly

Nebuly is the ROI platform for enterprise AI. It connects to the AI agents your business runs on, the assistants your customers interact with, and the tools your employees use every day, including Claude, ChatGPT, and Copilot, and translates that activity into business value. How much time is being saved across teams. What revenue your AI is influencing. What adoption and AI proficiency look like in practice, across departments and geographies. All aggregated at the organizational level, never tied to individuals.

If you need clarity on what your AI investment is actually delivering, book a demo.