COMPARE
Microsoft Viva Insights measures adoption and impact of Microsoft 365 Copilot through its Copilot Dashboard, Microsoft's Copilot analytics surface, and reports a high-level ROI estimate for the agents it covers. Nebuly is a vendor-agnostic ROI platform for enterprise AI that measures return from real interactions across every AI tool and agent in use, including tools built or bought outside Microsoft.
Book a demo
IN SHORT
Microsoft Viva Insights measures adoption of Microsoft 365 Copilot and estimates ROI from fixed activity multipliers applied at one tenant-wide rate. Nebuly measures ROI from real conversations across every AI tool, estimating time saved per task in your own cost structure, and it also analyzes what users ask, where agents fail, and how well employees prompt. This page compares the two on ROI methodology, cost analysis, and what each product can see.
What is the difference between Microsoft Viva Insights and Nebuly?
1. Nebuly analyzes the conversations, not just the clicks. The Copilot Dashboard counts deliberate actions. There is no topic analysis, no content analysis, no error rate, no failure metric. Nebuly breaks down every conversation by topic, surfaces unmet needs and demand the agent isn't serving, and gives you error rates per topic and per use case with failure clustering, drillable to the individual interaction. Plus implicit feedback: frustration, retries, abandonment, negative sentiment read from the conversation itself, no survey needed.
2. ROI computed from your real interactions, in your own cost structure. Seat-side, Microsoft applies three assistance factors: 6 minutes per search or summary action, 6 minutes per creation action, and the duration of any meeting summarized or recapped. One tenant-wide hourly rate, $72 by default from US BLS data. Microsoft itself describes these as broad approximations rather than precise calculations. Nebuly estimates time saved per individual task based on what the task actually was, and values it at the real cost of whoever performed it. Same output metric, very different credibility in front of a CFO.
3. Nebuly doesn't just meter spend, it reduces it. Copilot Credits tell you what you spent, in tokens and credits. Nebuly shows average cost per task performed, so you know what a business process actually costs to run, and ships an out-of-the-box analysis of your current model mix plus a recommended mix that holds task quality comparable at lower cost. Metering measures. Nebuly optimizes.
And the frame that holds all three: one place for every agent. Microsoft's scope is M365 Copilot and the agents it measures. The moment you add Claude, ChatGPT, a third-party vendor or something built in-house, that usage leaves the radar. Nebuly is vendor-agnostic, so the view stays single as your stack diversifies.
Microsoft Viva Insights vs Nebuly at a glance
How does each platform measure AI ROI?
Microsoft's model
The Copilot Dashboard reports Copilot assisted hours as actions multiplied by fixed research-based multipliers: 6 minutes per search or summary action, 6 minutes per creation action, and full meeting duration for summaries. Assisted value is assisted hours multiplied by an hourly rate that defaults to $72 (US BLS), set once for the whole tenant and not saved between sessions. Microsoft's own caveat is that these are broad approximations rather than precise calculations, and the Dashboard warns that Copilot is not likely solely responsible for the differences shown. Coverage is a trailing 28-day window with up to 6 days of delay.
Nebuly's model
Nebuly estimates ROI from the conversations themselves rather than from an activity count. For every successful conversation it calculates time saved (the time a person would have spent completing the same task manually) and value saved. Two mechanics drive the accuracy:
Time is estimated per task, not per action. Nebuly classifies each interaction into your taxonomy (Topic and sub-topic) and applies a manual-time estimate for that specific task type, overridable at any level of the taxonomy.
Value is set in your own cost structure and net of AI cost. The time saved is valued at the real cost of the person or function performing the task, and the AI execution (token) cost of running the task is subtracted to give the net value.
Only conversations that meet success criteria count toward ROI, so failed or abandoned interactions do not inflate the result. The headline relationship is ROI = Time Saved × Net Hourly Cost, reported as total net value (USD), hours returned to the team, and net value per successful task, broken down by user action.
How does Nebuly analyze conversations, failures, and proficiency?
Conversation and topic analysis
Nebuly classifies every conversation by topic and sub-topic through your taxonomy, surfacing what users ask, unmet needs, and demand the agent is not serving, drillable to the individual anonymized interaction.
Failure detection
Nebuly defines an error from the user's perspective, not the model's: a conversation can be technically valid, with no exception or timeout, and still count as a failure if the user did not get what they came for. To detect this it combines explicit feedback (thumbs down, negative ratings), implicit behavioral signals (rephrasing, retries, abandonment, frustration), and analysis of the AI's own reasoning and responses. The result is error rates per topic and per use case, with failure clustering that is filterable and drillable to the individual interaction.
AI proficiency
Nebuly assesses AI proficiency (also known as AI fluency), how effectively employees prompt, by analyzing the quality of their prompts, for example how clearly the task is stated, how much context and output detail is given, and how efficiently the conversation reaches a result. Each employee is rated from 0 to 100 and grouped into one of four levels, Expert, Practitioner, Beginner, or Novice, and Nebuly shows where they are competent versus struggling, with behavioral gap detection per person, team and function, and surfaces which trainings would close each gap.
Microsoft Copilot Dashboard (in Viva Insights)
Free with any M365/O365 business or enterprise subscription plus active Exchange Online
No paid Viva Insights license, no Copilot license needed to view
4 pages: Readiness, Adoption, Impact, Sentiment
Data is a trailing 28 days, up to 6 days delay, plus 7 days for newly licensed users
Below minimum group size (default 10), values show as "--"
Readiness page
Progress on activating Copilot for users
Recommendation cards: Copilot not pinned, wrong update channel, org-built agents blocked, outstanding license requests
License candidates: high Copilot Chat or M365 users flagged as prime for a license
Config hygiene only, not skill
Adoption page
Copilot licensed employees, active Copilot users (1+ action in 28 days)
Adoption by group, 6-month trendline, weekly or monthly
Usage levels, rolling 12 weeks: Power (15+ actions per week, active 9 of 12 weeks), Habitual (1 to 14 per week, active 9 of 12), Novice, Other
Drivers of power usage: frequency and consistency
Returning users: active in current and prior period
App breakdown: Teams, Outlook, Word, Excel, PowerPoint, Copilot Chat, plus OneNote, Forms, Loop, Edge, Copilot app
Benchmarks: internal cohort (job function, region, manager) and external ("companies like yours," 20+ companies, randomized for privacy)
Filters: Scope, Organization, Job function (Job function only if admin uploads it)
Impact page
Copilot actions taken, total across M365 apps
Copilot assisted hours: actions multiplied by multipliers from Microsoft's own research
Multipliers: 6 min per search or summary action, 6 min per creation action, full meeting duration for summaries
Copilot assisted value: assisted hours multiplied by hourly rate, default $72 (US BLS)
Hourly rate is tenant-wide, and does not save between sessions
Microsoft's own caveat: "broad approximations based on the best available research rather than precise calculations"
Requires optional diagnostic data on for full coverage
Explicit warning that Copilot is not likely solely responsible for the differences shown
Satisfaction rate
Only native quality signal
Thumbs up as a percentage of all thumbs up plus down, rolling 28 days
Hidden unless 30+ responses from 5+ unique users
No error rate, no hallucination, no failure detection
Sentiment section
Survey data, not telemetry
Sources: Viva Glint, Viva Pulse, or admin CSV upload
4 fixed questions: quality, mental effort, speed, productivity
Normalized to a 100-point scale, heat map by attribute
Suppressed below minimum group size
Agent Dashboard / agent insights
Active agent users, agents with 1+ active user, agent responses, top 3 agents
Copilot Credits usage if pay-as-you-go enabled
Public preview
Needs 50+ Copilot or 50+ Viva Insights licenses
Adoption only, no value or quality
Analyst workbench
Custom person queries, Copilot metrics added manually
Power BI templates
Values can differ from the Dashboard due to different filters and date ranges
License tiers
1 to 49 Copilot licenses: Readiness, Adoption, Impact, Copilot Chat insights
50+ Copilot or 50+ Viva Insights: adds agents, benchmarks, intelligent summaries, survey sentiment, week and month trendlines, manager group views, delegation
The full 50+ tier is included at no extra cost
Not present anywhere
Topics or what people actually ask
Conversation content or transcripts (Purview and eDiscovery only, for legal and security)
Error rates, failures, hallucinations
Implicit frustration, retries, abandonment
Per-task time saved or per-task cost
Per-person or per-function cost modeling
Cost per task, model mix analysis, cost optimization
Any non-Microsoft AI tool
Copilot Studio agents report (Power BI template, analyst workbench)
Not the Agent Dashboard. This one has value and quality metrics
Prerequisite: 50+ Copilot licenses, one published agent with 4+ weeks of data
Scope: agents built in Copilot Studio only (websites, Teams and M365 Copilot, mobile, Facebook, Azure Bot channels). Excludes declarative agents, Agent Builder agents, and anything built outside Copilot Studio
Metrics: active agents, engaged sessions, satisfaction score, resolution rate, sessions by topic and trigger, Agent Assisted Hours, Agent Assisted Impact
Conversational:
(KS references + weighted sessions) x 6 min. Weights: resolved 1.0, escalated 0.7, abandoned 0.7Autonomous:
[(KS refs x 6 min) + Σ(action count x TimeSavings_Action(a)) + (sessions w/o actions x 2.8 min)] / 60TimeSavings_Action(a)is a distinct multiplier per action type, from "existing research studies, large language model (LLM)-based estimation techniques, and analysis of product telemetry"Multiplier and hourly rate editable per agent, not tenant-wide. Impact = AAH x $72/hr default
The following are not present in the Microsoft Copilot analytics surface: topics or what people actually ask; conversation content or transcripts (available only through Purview and eDiscovery, for legal and security); error rates, failures, or hallucinations; implicit frustration, retries, and abandonment; per-task time saved or per-task cost; per-person or per-function cost modeling; cost per task, model mix analysis, and cost optimization; and any non-Microsoft AI tool.
What is the difference between Microsoft Viva Insights and Nebuly?
Microsoft Viva Insights measures adoption of Microsoft 365 Copilot through its Copilot Dashboard and reports a high-level, survey-and-research-based ROI estimate for the agents it covers. Nebuly is vendor-agnostic and measures ROI from real interactions in your own cost structure, adding conversation topic analysis, error and failure detection, implicit feedback, and AI proficiency across every AI tool in use.
Does Microsoft Viva Insights measure AI ROI accurately?
It produces a high-level estimate using fixed assistance factors (6 minutes per action, full meeting duration for summaries) valued at one tenant-wide default hourly rate of $72. Microsoft describes these as broad approximations rather than precise calculations. Nebuly instead estimates time saved per task and values it at the real cost of whoever did the work, net of AI execution cost.
How does Nebuly measure AI ROI?
Nebuly analyzes each successful conversation and estimates the time a person would have spent doing the same task manually, using per-task manual-time estimates from your taxonomy. It values that time at the real cost of the person or function performing the task and subtracts the AI execution (token) cost, so ROI equals time saved multiplied by net hourly cost. Only conversations that meet success criteria are counted..
Can Viva Insights analyze what employees actually ask Copilot?
No. The Copilot analytics surface counts deliberate actions and does not include topic or content analysis. Nebuly provides a topic breakdown of every conversation, with drill-down to individual interactions and detection of unmet needs.
Does Microsoft Viva Insights detect errors, failures, or hallucinations?
No. There is no error rate, hallucination, or task-failure metric. The only native quality signal is an aggregate thumbs up and down satisfaction rate. Nebuly reports error rates per topic and per use case, with failure clustering that is filterable and drillable.
How does Nebuly detect AI errors and failures?
Nebuly defines an error from the user's perspective rather than the model's, so a technically valid conversation can still count as a failure if the user did not get what they needed. It combines explicit feedback, implicit signals such as rephrasing and abandonment, and analysis of the AI's own reasoning, producing error rates per topic and per use case with failure clustering.
Does Nebuly measure how well employees use AI?
Yes. Nebuly measures AI proficiency (also known as AI fluency) by analyzing how effectively employees prompt, rating each employee from 0 to 100 and grouping them into Expert, Practitioner, Beginner, and Novice levels, showing where they are competent versus struggling, with behavioral gap detection per person, team and function, and surfacing which trainings would close each gap.
Can Microsoft Viva Insights measure non-Microsoft AI tools such as ChatGPT or Claude?
No. Scope is limited to Microsoft 365 Copilot and the agents Microsoft measures. Usage of Claude, ChatGPT, third-party vendors, or in-house builds is out of scope. Nebuly is vendor-agnostic and keeps a single view across the whole stack.
Can Nebuly measure agents built outside Copilot Studio or Microsoft 365?
Yes. Nebuly is vendor-agnostic and covers agents regardless of where they are built, including third-party and in-house agents, which fall outside Microsoft's measured scope.
Is the Microsoft Copilot Dashboard free?
The Copilot Dashboard in Viva Insights is free with any M365 or O365 business or enterprise subscription plus active Exchange Online. No paid Viva Insights license and no Copilot license is needed to view it, though richer tiers unlock with 50 or more Copilot or Viva Insights licenses.
What time window does Microsoft Copilot analytics use?
The Copilot Dashboard shows a trailing 28-day window with up to 6 days of delay (up to 72 hours in the admin center), plus 7 days for newly licensed users. Nebuly uses client-controlled ingestion frequency with no limit on the time window.
Does Microsoft measure agents built in Copilot Studio?
Yes, through a separate Copilot Studio agents report (a Power BI template) that includes value and quality metrics, but only for agents built in Copilot Studio. It excludes declarative agents, Agent Builder agents, and anything built outside Copilot Studio, and it requires 50 or more Copilot licenses and one published agent with at least 4 weeks of data.
