Summary
A leading health and wellness technology company combines real-time behavioral analytics with privacy-first design to understand how people interact with its GenAI assistant. Nebuly enables this company’s teams to scale insights, improve user experience, and maintain trust at every level.
Sector
Health & Wellness
Use case
Conversational GenAI assistant for personal health insights
Launch
March 2025
nebuly deployment
Self-hosted on private AWS cloud
The company chose Nebuly for its ability to process conversational health data safely at scale.
A global health and wellness technology company partnered with Nebuly to better understand how people use its GenAI assistant.
At launch, the company previewed the assistant to 1.5–2.0 million eligible users, with Nebuly analytics designed to support up to 500,000 daily interactions at full scale.
GenAI Assistant – User Analytics
Nebuly provides analytics for the company’s customer-facing GenAI assistant.
The platform analyzes chat interactions to uncover user intents, satisfaction levels, friction points, and risk signals, giving teams visibility into how people actually experience and engage with the assistant.
This helps identify when users are frustrated, confused, or disengaged, enabling continuous improvement of conversation quality and user satisfaction.
GenAI Assistant – Behavioral Analysis
Nebuly also enables behavioral segmentation to understand how different types of users interact with the assistant over time.
By grouping users based on their engagement patterns, for example:
teams can uncover usage habits, engagement depth, and behavioral trends across cohorts (e.g. morning-only users vs. power users).
This helps product and data teams profile user behavior, identify what drives adoption or drop-off, and design more personalized and effective chatbot experiences.
Limited visibility on user behavior
The company could not systematically identify what people discuss with the GenAI assistant or where it performed below expectations. Earlier analysis relied on manual CSV exports and spreadsheets, missing quiet but important user signals.
Measurement challenges
Users often struggled with frustrating chatbot experiences, receiving irrelevant, incomplete, or incorrect answers. The company lacked visibility into when and why users were dissatisfied, since existing tools couldn’t measure LLM-specific issues like poor response quality or confusing follow-ups.
Without clear user-centric metrics such as user satisfaction, error rate, or perceived risk, the company couldn’t identify what made conversations fail or improve them over time.
Compliance and privacy needs
Because the product handled sensitive health topics, privacy and compliance were non-negotiable. Teams required a solution that controlled access, kept data local, and excluded user-specific or regulated content.
“This is a true unlock.
”
Data integration and privacy-first setup
A self-hosted Nebuly instance on AWS brought together assistant and experience data. The solution provided unified analytics across business units and supports custom tagging, clustering, and demographic analysis.
Defined KPIs and behavioral signals
Nebuly introduced consistent metrics such as user satisfaction, error rate, and engagement, tracked by topic and user cohort. Dashboards now track risk and satisfaction trends in real time.
Privacy, compliance, and auditability
Granular permissions, retention policies, and user-level deletion ensure compliance. The self-hosted option gives internal stakeholders full control over healthcare-related data.
Self-service insights
The platform enables product managers, analysts, and behavioral scientists to access information directly. This replaced manual analysis and improved collaboration across functions.
“Nebuly has been a huge help to understand what members are talking with the assistant, or identifying areas where it fell short.
”
Teams now use satisfaction and risk KPIs to guide product decisions, refine features for localization, and focus on key audience segments.
“There are very rarely situations where we get to work with a company like Nebuly. The team has been very reactive, and proactive, and it’s been fantastic.
”