User behavior

Autocapture the behavior of your LLM users

Empower your team to gain instant visibility into massive amounts of LLM user data.

User intent

Explore tasks, questions and actions users are doing

The best way to easily understand what your users are doing with your LLMs is using the User Flow chart. This chart identifies the most frequent User Actions (tasks or questions) done by your users. You can use User Flow to understand how your users sequentially ask questions to your LLM-product and analyze drop-offs or unsuccessful behavior.

Discussed topics

Understand the most discussed topics by your users

Topic reveals the subjects users frequently discuss in your LLM product. By identifying these popular topics, you can enhance your LLM product by enriching content in areas most discussed by users. For clear visualization, use the Analytics Chart, filtering Users or Interactions by Topic. To track trends over time, utilize the Line Chart or Vertical Bar Chart.

Causes of frustration

Understand why LLM users are dissatisfied

Nebuly allows you to automatically understand why some of your users are not satisfied with the answers they are getting from your LLMs. This can help you understand where to improve your LLMs.

Frustration reasons

Explore why some LLM answers are insufficient according to your users. To track the causes of frustration, use the Analytics chart, filtering Users or Interactions by Frustration Reason. To track trends over time, utilize the Line Chart or Vertical Bar Chart.

Challenging questions

Pinpoint user actions and questions that lead to dissatisfaction. Use the Analytics chart, break down Users or Interactions by Action and filter by all Frustration Reason. To track trends over time, utilize the Line Chart or Vertical Bar Chart.

Topic deficiencies

Determine topics where LLMs' answers are inadequate for your users. Use the Analytics chart, break down Users or Interactions by Topic and filter by all Frustration Reason. To track trends over time, utilize the Line Chart or Vertical Bar Chart.

RAG effectiveness

Evaluate the efficacy of RAG data sources for your users. Use the Analytics chart, break down Interactions or Users by Data Source and filter by all Frustration Reason. To track trends over time, utilize the Line Chart or Vertical Bar Chart.

Dissatisfied user behavior

Analyze user behavior after (or before) receiving an unsatisfactory answer. Use the User Flow chart and filter by all Frustration Reason.

Time to value

Understand how long it takes your users to get the answers they were looking for. Select two actions in the Time To Value chart to learn how long it took users to complete the second action after performing the first.

Data sources

Understand RAG data sources commonly utilized by your users

Data Sources show the most accessed RAG sources by your users. To view this clearly, utilize the Analytics chart and filter by Data Source for Users or Interactions. To track trends over time, utilize the Line Chart or Vertical Bar Chart.

Cohorts

Understand user cohorts of your LLM products

Visualized how satisfied, neutral and dissatisfied users interact differently with your LLM product.

Power users

Identify the most engaged LLM users

Nebuly helps you highlight the users who find the LLM's answers useful.

Review satisfied user profiles.
Track satisfaction trends over time.
Monitor changes in their interaction frequency.

Dissatisfied users

Identify the users that are likely to churn

Nebuly helps pinpoint which are the users dissatisfied with your LLMs' answers, enabling timely action.

You can gain insights into these user groups by:

Examining neutral and dissatisfied user profiles.
Tracking users' frustration trends over time.
Observing changes in their interaction frequency.

Cost tracking

Track the cost of your LLMs

Understand how cost correlates with user engagement and satisfaction to measure ROI.

Unit cost by user

Track how much each LLM user is costing you

Nebuly allows you to link user satisfaction levels to LLM costs.

For a clear cost-benefit analysis, it addresses:

The cost of serving dissatisfied users.
The cost of serving satisfied users.

Security & compliance

Enterprise-grade security

At Nebuly we take security and compliance extremely seriously. Whether you’re a fast-growing startup or one of the largest companies in the world, your data is in good hands with us.

Strong regulatory compliance

We maintain strong regulatory compliance for data protection

We adhere to protocols that meet the certification standards required to comply with key data protection legislation. We are actively working towards obtaining SOC2 Type 2 and ISO 27001 certifications and undergoing audits.

PII removal

We ensure PII removal for enhanced data privacy

We automatically identify all data fields that contain personally identifiable information (PII) and replace all PIIs with pseudo values or codes that do not reveal any personal information about the individual.

Secure encryption

We employ high-level encryption for maximum data security

Secure your data with the same encryption that banks use and ensure that no authorized parties can view information. All data in transit is encrypted using TLS/SSL.

Identity and access management

We offer robust identity and access management for secure data control

Create granular permissions that dictate who can access and work with your data.

Support for role-based permissions
Logs can be deleted upon requests

Infrastructure security

We prioritize infrastructure security for data protection and scalability

We take steps to ensure that our infrastructure is both secure and scalable.

Private endpoints enforced across system infrastructure
IP whitelisting and private VPC enforced

Detailed awareness training

We provide thorough security training to ensure compliance and data protection

All Nebuly staff undergo comprehensive security training when they join and periodically thereafter to maintain compliance standards.

Craft bespoke LLMs for your users

Request a demo and you'll get an overview of Nebuly's capabilities and how it can help understanding your users.

Book a demo