Generative AI has captured boardroom excitement and billion-dollar investments, but behind the excitement lies a stark GenAI divide. According to a recent MIT Media Lab report, after $30–40 billion poured into enterprise AI, 95% of organizations are seeing no measurable ROI. In fact, just 5% of AI initiatives are delivering real business value, while the vast majority remain stuck in pilot mode.

It’s a classic case of hype vs. reality: adoption of tools like ChatGPT is widespread, yet genuine transformation is rare.

This article dives into why most generative AI projects struggle to move the needle – and how a new approach focusing on user feedback and analytics can help companies cross that chasm.

The “GenAI divide”: haves vs. have-nots

The term “GenAI divide” comes from how unevenly the benefits of generative AI are spread.

On one side are the haves: the ~5% of organizations extracting significant value, sometimes millions in savings or new revenue.

On the other side are the have-nots: the 95% whose AI pilots haven’t translated into business impact.

Interestingly, overall adoption is high: over 80% of companies have experimented with tools like ChatGPT or GitHub Copilot, and nearly 40% have some system in deployment.

But these are mostly surface-level wins (think small productivity boosts for individuals) rather than enterprise-level transformations.

The MIT study noted that only two sectors (Tech and Media) showed clear disruption from GenAI, while at least seven other industries are “widespread experimentation without transformation.” In other words, plenty of proof-of-concepts, very few proofs-of-value.

What does this divide look like in practice? One mid-market COO in the MIT Media Lab report put it bluntly:

“The hype on LinkedIn says everything has changed, but in our operations, nothing fundamental has shifted… we’re processing some contracts faster, but that’s about it.”

For most companies, GenAI hasn’t yet reconfigured cost structures or customer experiences – it’s made some tasks a bit quicker, but the core business runs the same as before.

The high curiosity, low conversion pattern is evident: lots of pilot projects, very few scaled solutions. Enterprises actually ran more AI pilots than smaller firms, yet they convert the fewest into production systems, often spending 9+ months in pilot purgatory. This is the heart of the GenAI divide: a few leaders forging ahead, and a long tail of organizations still figuring out how to make AI stick.

Why internal GenAI adoption is so hard

If the value is there for the taking, why aren’t more companies crossing over?

The root causes turn out to be less about the AI models themselves and more about how humans and AI interact.

The MIT researchers pinpoint a fundamental learning gap as the main barrier. In traditional software, an application can improve through updates once developers see how people use it. In generative AI, many systems today “do not retain feedback, adapt to context, or improve over time.”

Each user session is like Groundhog Day – the AI doesn’t remember the last conversation or learn from its mistakes.

Imagine a human employee who forgot every customer interaction immediately; they wouldn’t last long in a job!

Yet many GenAI tools have exactly that issue: short-term memory and no long-term learning.

This lack of adaptability leads to poor user experience, especially for internal AI copilots meant to assist employees. Early enthusiasm can quickly turn into frustration. For example, an employee might love a GenAI assistant for drafting an email or pulling a quick report – until they realize it can’t learn their preferences or context from one request to the next.

As one user cited in the MIT report explained,

“It’s excellent for brainstorming and first drafts, but it doesn’t retain knowledge of client preferences or learn from previous edits… For high‑stakes work, I need a system that improves over time.”

Without memory or continuous learning, AI assistants end up repeating the same mistakes and requiring users to re-explain context constantly. For busy professionals, that’s a deal-breaker. They’ll abandon the tool after a few tries, deciding it’s easier to do the task manually or stick with familiar (non-AI) software.

This dynamic is a big reason internal AI initiatives struggle with adoption.

Another issue is integration into real workflows.

Many pilots are developed as isolated demos or “wrappers” around an LLM with limited integration into the company’s processes and data. They impress in a demo but break down in day-to-day use – workflows are brittle, and the AI suggestions often miss the mark because the system isn’t tuned to actual on-the-ground needs.

There’s also the classic change management challenge: employees can be resistant to new tools (“Why should I trust this AI?”), especially if the output quality is inconsistent.

In fact, in a survey of enterprise leaders, resistance to adopting new AI tools was ranked as the top challenge, followed closely by output quality and UX problems, many of which trace back to the AI’s missing memory and poor workflow fit.

In short, internal GenAI adoption is hard because people won’t fully embrace a tool that doesn’t feel reliable or integrated into their real work.

One interesting finding is how employees are responding: when the official AI tools don’t meet their needs, they find workarounds:

Over 90% of companies reported their staff use personal AI tools (like free ChatGPT accounts or other unofficial apps) to help with work, even if only 40% of those companies have an “approved” AI tool internally.

This “shadow AI” trend means individuals are crossing the AI divide on their own, getting value in small ways (like drafting a memo or analyzing data with a personal GPT tool), while the larger organization’s GenAI program remains stuck.

It’s a clear signal that the demand for AI assistance is there, employees do want AI to make their jobs easier, but the supply (the enterprise’s official AI solution) isn’t delivering.

When a sanctioned AI copilot is launched without careful thought to user experience, it risks becoming yet another unused corporate tool. In fact, we’ve heard heads of product voice exactly this worry: “What if we roll out this AI assistant and nobody ends up using it?” If you can’t tell whether an internal AI is actually helping or just gathering dust, it’s nearly impossible to justify scaling it up or investing more resources. This uncertainty around usage and value is a huge adoption killer.

Closing the feedback loop with user analytics

How can companies break out of this pattern and move the needle on GenAI adoption? The emerging consensus is that organizations need to start treating GenAI projects less like magic boxes and more like normal products – which means instrumenting them with user analytics and feedback loops.

In other words, to overcome the learning gap, your AI has to learn from its users. Every interaction – every prompt, every response, every time a user rephrases a question or gives up on a query – is telling you something. It’s effectively feedback on where the AI is useful and where it’s failing.

Think about how we improve a customer-facing app or a website: we collect data on what users do, where they click, where they get confused or drop off.

We wouldn’t dream of launching a new web product and then not track anything but server uptime, or launching a website and only look at server logs, ignoring Google Analytics on user behavior. Yet, that’s exactly what many teams are doing with their AI tools.

For GenAI, “observability ≠ adoption.” Low-level metrics like API latency or token counts might tell you that the model is running, but they won’t tell you if your employees are actually using it effectively.

To measure success, we need metrics more akin to user engagement and satisfaction: How many queries come in each day? What types of questions are people asking? Where does the AI’s answer lead to frustration or confusion (e.g. the user rephrases the question, or abandons the AI and calls support)? Which teams or departments have embraced the tool, and which barely touch it?

These are the kinds of insights that LLM user analytics provides, and they form the feedback loop for improvement.

By analyzing real AI usage insights (not just whether the system is up, but how it’s being used), companies can iterate their GenAI deployments much faster.

For instance, if you discover that 30% of users who try your customer support chatbot end up requesting a human agent within two interactions, that’s a red flag about the bot’s helpfulness.

Maybe it’s failing on certain query types – something you can fix with better training data or a tweak in the prompt.

Or perhaps you notice that one department in your company has sky-high usage of an internal AI assistant, while another department with the same tool hardly uses it at all.

Digging into why (maybe the latter department’s workflow is slightly different, making the tool less relevant) can inform how you adjust the deployment or provide training.

User analytics creates a virtuous cycle: deploy, measure, learn, improve. Then deploy a better version. This kind of feedback loop is exactly what’s missing in the 95% of stalled projects. Without it, teams are essentially flying blind. With it, they gain the vision to make data-driven enhancements.

In fact, organizations that establish structured feedback loops for their LLM products can outpace competitors by 10× in how quickly they improve their AI capabilities. Speed of learning becomes a real competitive advantage.

Crucially, closing the loop also helps address the human side of adoption.

When users see their feedback reflected in a rapidly improving AI system. For example, the assistant that failed to answer compliance questions last month is giving much better answers this month, their trust grows. They become more willing to rely on it, which in turn boosts adoption.

It’s similar to agile software development with tight user feedback cycles, applied to AI: you roll out small improvements continuously based on what the users are actually doing and needing. Over time, this can turn a skeptical audience into avid adopters.

Bridging the gap: where Nebuly fits in

This shift toward user-centric, analytics-driven AI is exactly the movement we at Nebuly are focused on.

We often describe our platform as the “user intelligence layer” for GenAI products, essentially, providing that Google-Analytics-for-AI capability that has been missing.

Our perspective is that every user prompt is feedback, and capturing those signals at scale is the key to bridging the GenAI divide. By aggregating and analyzing how people interact with conversational AI systems, we aim to turn the lights on in what has been a black box of user behavior.

What does this look like in practice?

Imagine you’ve launched an internal AI copilot for your sales team. Without user analytics, you might only know things like how many queries the model handled, or at best the accuracy of responses in a test scenario. With a user analytics approach, you can see that, say, 50 sales reps tried the assistant this week, but many got stuck when asking for pricing recommendations. You might discover through the data that whenever a user’s query contains a specific product code, the AI’s answer is too generic – leading reps to disengage.

These insights are gold. They tell the product and AI teams exactly where to refine the assistant (perhaps by integrating it with your pricing database or adding training examples for those scenarios). In one real-world example, a Fortune 500 company rolled out several internal AI assistants across departments – but initially had no visibility into how they were used. Some assistants, it turned out, were barely used at all until the company identified why (e.g. one department’s tool was missing integration with a key internal system, so it often gave incomplete answers). By instrumenting their GenAI apps with Nebuly’s analytics, they pinpointed such issues and fixed them, turning once-struggling pilots into widely adopted solutions.

Stakeholders went from asking “Is anyone even using this thing?” to having a clear dashboard of usage patterns, user feedback, and improvement over time. That transparency not only helped justify the investment, it actively reduced internal resistance – teams could see the AI getting better and bringing tangible benefits, like cutting document processing time by 50% or automating answers to common IT support questions.

Nebuly’s broader mission is aligned with what the GenAI Divide report calls out as the path forward: systems that persist memory and learn from feedback.

We believe that understanding your users is how AI delivers business impact. It’s not enough to deploy a great model; success in the AI era will come from continuously learning how real people are engaging (or not engaging) with that model.

By closing the loop – connecting user behavior back into model and product improvements – companies can transform GenAI from a flashy demo into a dependable workhorse.

In practical terms, that means product managers and AI teams need to track new kinds of metrics. Instead of just asking “What’s our model’s accuracy or uptime?”, you start asking “Where are users getting frustrated? What percentage of sessions lead to a successful outcome? Which requests are we failing to handle?” This user-centric mindset is how the GenAI have-nots become GenAI haves.

Conclusion:

The generative AI landscape in 2025 may look like a story of 5% success and 95% frustration, but the next chapter isn’t written yet. Every company still has the opportunity to cross the GenAI divide by learning from those early missteps.

The key is to stop treating GenAI like a one-off experiment and start treating it like any other product that needs nurturing. That means instrumenting it, monitoring it, and iterating on it with real-world usage data. The organizations that embrace this approach – building continuous feedback loops, focusing on user experience, and empowering their “prousers” (those employees already finding ways to leverage AI) – are turning the promise of AI into practice.

They’re moving from pilots to full production, from isolated wins to scalable ROI.

In the end, bridging the GenAI divide comes down to a simple but profound shift: listening to your users.

Generative AI is powerful, but it will only drive transformation if it’s aligned with how people actually work and what they need. By using LLM user analytics to illuminate the blind spots, companies can finally see why an AI tool isn’t catching on and take action to fix it. It’s a feedback-driven revolution – one that turns adoption from a black box into a well-lit loop.

As we like to remind our clients: understanding your users is how AI delivers business impact. The companies that take that to heart will be the ones to close the gap, turning today’s AI have-nots into tomorrow’s success stories.