AI Adoption Stories #1: What user analytics revealed about a GenAI copilot rollout
AI Adoption Stories #1: What user analytics revealed about a GenAI copilot rollout
TLDR
→ A global agricultural machinery enterprise deployed an AI copilot to support its dealer network, targeting technical maintenance information and regional adoption. → Within 40 days, one dealer group had more than 400 unique users — but adoption varied significantly across branches within that group. Filtering by branch revealed exactly where adoption had stalled and where proactive follow-up was needed. → Error rates were higher in Latin America and Europe than in North America. Filtering conversation data by language revealed the cause: non-English queries had significantly higher error rates. The issue was language performance, not regional engagement. → These insights came from a few quick filters on conversation data — no dashboards reporting only technical uptime, no waiting weeks for a manual review. The team could act the same day. → The case illustrates what behavioral visibility makes possible: not just knowing the system is running, but understanding who is using it, how, and where it is falling short.
One of our clients, a global enterprise in the agricultural machinery sector rolled out a AI copilot to support its dealer network. The goal was to improve access to technical maintenance information and boost AI adoption across regions.
After launch, one internal lead wanted to know more.
“How much is this one dealer actually using it?”
Using Nebuly’s AI user analytics platform, the team filtered usage by dealer group name.
One stood out. In just 40 days, that dealer had more than 400 unique users, a strong signal of AI adoption.
But usage wasn’t consistent across every location.
Some branches were more active than others.
So the team looked deeper. A custom field let them filter usage by individual branch. This helped them spot where adoption was high and where it had stalled.
They could now follow up directly with quiet branches to understand what was blocking usage.
Higher error rates in some regions
A separate question came after they noticed that the assistant’s error rate was higher in Latin America and Europe than in North America.
Using Nebuly, the team filtered performance data by language. It was clear that non-English queries had much higher error rates.
That insight shifted their priorities, and they could focus on improving multilingual performance.
What they learned
- One dealer group had very strong adoption
- Some branches weren’t using the assistant at all
- Higher error rates were tied to language, not location
These insights came from just a few quick filters. 5 minutes. No guesswork. No waiting on dashboards that only show technical uptime.
The team left with a clear view of where adoption was strong, where it had stalled, and how language impacted performance. That visibility helped shape what came next, both in product and rollout.
Nebuly
Nebuly is the ROI platform for enterprise AI. It connects to the AI agents your business runs on, the assistants your customers interact with, and the tools your employees use every day, including Claude, ChatGPT, and Copilot, and translates that activity into business value. How much time is being saved across teams. What revenue your AI is influencing. What adoption and AI proficiency look like in practice, across departments and geographies. All aggregated at the organizational level, never tied to individuals.
If you need clarity on what your AI investment is actually delivering, book a demo.
FAQs
How does filtering conversation data by branch or language reveal adoption patterns?
Filtering conversation data by organizational dimension — branch, region, language, department — transforms aggregate usage figures into specific, actionable signals. An overall adoption rate tells you whether people are using the tool. Branch-level filtering tells you which specific groups have adopted it and which have not, so follow-up and support can be targeted precisely. Language filtering reveals whether performance differences are driven by the model's multilingual capability rather than user behavior, which points to a very different kind of fix.
Why is error rate analysis by language important for global AI deployments?
AI agents trained primarily on English data often perform less accurately on queries in other languages. This is not immediately obvious from overall error rate metrics, which average across all languages and can mask poor performance in specific linguistic contexts. When a global enterprise sees higher error rates in certain regions, the intuitive assumption is regional engagement or workflow differences. Language-level analysis often reveals that the real driver is model performance on non-English queries — a technical issue with a specific, solvable fix.
What does this case show about the difference between system monitoring and conversation analytics?
System monitoring tells you whether the AI is running. Conversation analytics tells you how people are using it and where it is failing them. In this case, the system was running correctly in every region. Error rates, adoption variation by branch, and language performance gaps were all invisible to infrastructure monitoring. They became visible in minutes through filtering conversation data — enabling immediate, targeted action rather than weeks of manual investigation.
Stay up to date on what we're learning, building, and seeing as enterprise teams deploy and measure AI agents in production.