Summary
D-ID is a leading AI video creation platform, known for enabling brands to generate photorealistic videos from text using generative AI.
Sector
Generative AI
Use case
Customer-facing AI agents
Launch
April 2024
nebuly deployment
Self-hosted on private AWS cloud
D-ID’s platform powers millions of AI-human conversations through custom AI Agents that combine video, voice, and natural language. With diverse use cases spanning customer support, training, and content creation, the team needed a robust solution to help clients understand and optimize how their AI Agents performed across varied, often complex, interactions, ultimately improving AI Customer Experience.
By embedding Nebuly’s leading LLM analytics engine, D-ID brought real-time LLM user analytics directly into its product. This gave end users visibility into how their AI Agents were being used, enabling continuous improvement and measurable performance at scale, directly contributing to effective customer experience and conversational AI engagement.
“We evaluated various LLM insights tools and chose Nebuly for their outstanding responsiveness and ability to customize insights quickly for our needs. The performance of their LLM detection for user intents and negative feedback exceeded expectations with great accuracy.
”
D-ID needed to help customers understand how their custom AI Agents were performing. However, with each agent generating unique LLM interactions, manual review wasn’t scalable, and off-the-shelf analytics tools couldn’t keep up with the complexity of real-world conversational AI analytics.
After evaluating more than 25 solutions, D-ID selected Nebuly to power a built-in insights engine that could detect intent, highlight friction, and surface trends securely and at scale, addressing critical LLM evaluation needs.
D-ID partnered with Nebuly to embed a white-labeled insights engine into its platform, giving customers a powerful set of tools to:
01. Create highly interactive, custom-built AI Agents.
02. Gain real-time visibility into how users are interacting with these agents.
03. Continuously improve AI agent performance with actionable feedback, driving LLM user retention.
“A really powerful application of Nebuly for D-ID Agents is to detect and fill in gaps in the knowledge of agents. Nebuly provides actionable insights when an agent can’t answer a question, and enables D-ID customers to update the agent’s knowledge base to ensure that end users will get the right response going forward.
”
By surfacing real usage data, D-ID customers now get:
01. Visibility into the most common user-agent conversation topics.
02. Feedback that highlights sentiment and friction points, supporting comprehensive LLM feedback.
03. AI analytics for user volume, topic frequency, and agent performance.
Before choosing Nebuly, D-ID carried out an in-depth evaluation of leading LLM analytics solutions, focusing on flexibility, detection accuracy, and integration speed. Nebuly stood out for its ability to tailor AI insights quickly and embed them seamlessly into the platform, showcasing its robust LLM evaluation framework.
The rollout followed a clear path:
→ Step 01: Vendor landscape review to benchmark capabilities, including LLM evaluation benchmarks.
→ Step 02: Proof of concept using a sample dataset to assess performance and validate LLM evaluation metrics.
→ Step 03: Live integration of Nebuly’s dashboard for selected customers, ensuring smooth LLM ops.
Now embedded with Nebuly, D-ID can grow its AI Agent analytics capabilities to serve more customers without extra effort, gaining deeper insights into their LLMs and consistently improving AI customer experience.