Design a voice analytics framework that tracks conversation quality, user satisfaction, and bot performance metrics.
## CONTEXT Most voice application teams operate blind — they know how many conversations happened but not why 30% of users dropped off at turn 3, which intents have the highest confusion rates, or whether last week's prompt change improved or degraded the experience. Without voice-specific analytics, teams waste development cycles guessing what to fix. A purpose-built voice analytics framework that tracks conversation quality, user behavior patterns, and NLU performance gives teams the data they need to systematically improve every interaction. ## ROLE You are a voice analytics specialist with 10 years of experience designing measurement frameworks for conversational AI products at scale. You built the analytics platform for a voice-first company whose assistant handles 8 million conversations monthly, and your dashboards are credited with driving a 22-point improvement in task completion rate over 12 months by surfacing the specific dialog paths, intents, and error patterns responsible for user drop-off. Your methodology connects every voice metric to a business outcome and prioritizes actionable insights over vanity metrics. ## RESPONSE GUIDELINES - Define every metric with a precise calculation formula, not just a description - Include specific threshold values for what constitutes good, acceptable, and poor performance - Design the analytics to surface actionable insights that map directly to development priorities - Specify how to segment metrics by user type, intent category, and conversation path - Do NOT create a dashboard that only shows aggregate numbers — drill-down capability by conversation, intent, and time period is essential - Do NOT track metrics without defining what action the team should take when each metric moves ## TASK CRITERIA 1. **Core KPI Definitions** — Define the primary voice metrics for [INSERT VOICE APPLICATION] with exact calculation formulas: task completion rate (successful completions / total attempts), average turns to resolution, fallback trigger rate, user drop-off rate by conversation turn, first-turn recognition accuracy, and customer satisfaction score. 2. **Conversation Funnel Analysis** — Map the full conversation funnel from invocation through task completion. Define funnel stages specific to [INSERT VOICE APPLICATION], calculate conversion rates between stages, and identify the top 3 drop-off points with hypotheses for why users abandon at each stage. 3. **Intent Performance Analytics** — Track metrics per intent: recognition accuracy, average confidence score, confusion matrix showing which intents get misclassified as which, unrecognized utterance logs grouped by frequency, and intent-level completion rates. Rank intents by improvement priority using a composite score of volume times failure rate. 4. **NLU Health Monitoring** — Design monitoring for the natural language understanding layer: confidence score distributions over time, entity extraction accuracy rates, slot fill rates per intent, and utterance coverage metrics that show what percentage of user inputs the system handles confidently vs. with low confidence. 5. **User Behavior Patterns** — Define behavioral analytics: session frequency and recency per user, feature adoption rates across capabilities, common conversation paths visualized as flow diagrams, repeat callers analysis (same user, same issue), and power user vs. one-time user segmentation. 6. **Conversation Quality Scoring** — Build a per-conversation quality score (0-100) combining: NLU average confidence (30% weight), task resolution status (30% weight), number of error recoveries needed (20% weight), and conversation duration vs. benchmark (20% weight). Define quality tiers and their business implications. 7. **Sentiment & Satisfaction Tracking** — Specify how to measure user satisfaction: post-conversation surveys (CSAT, NPS), in-conversation sentiment detection from tone and word choice, implicit satisfaction signals (task completed, user said thank you), and dissatisfaction signals (user said "agent," raised voice, abandoned). 8. **Alerting & Anomaly Detection** — Define alert rules tied to [INSERT BUSINESS GOALS]: task completion rate drops below threshold (alert within 1 hour), unrecognized utterance spike (alert within 30 minutes), average confidence score degradation trend (daily alert), and new error pattern emergence detection. 9. **Reporting Cadence & Views** — Design 3 reporting views: real-time operational dashboard (current hour metrics, active issues), weekly performance review (trend charts, top issues, improvement recommendations), and monthly strategic report (business impact analysis, roadmap alignment, capacity planning). 10. **Improvement Prioritization Framework** — Create a system that automatically ranks improvement opportunities by: estimated user impact (affected conversations per week), estimated effort (simple prompt fix vs. model retrain vs. new capability), business goal alignment with [INSERT BUSINESS GOALS], and expected lift in task completion rate. ## INFORMATION ABOUT ME - My voice application: [INSERT VOICE APPLICATION — e.g., customer support voice bot, smart speaker skill, IVR system, in-app voice assistant] - My business goals: [INSERT BUSINESS GOALS — e.g., reduce call center volume by 30%, increase self-service completion to 80%, improve CSAT to 4.5] - My current pain points: [INSERT PAIN POINTS — e.g., high drop-off rate, unknown failure causes, no visibility into NLU accuracy] - My analytics tools: [INSERT TOOLS — e.g., Mixpanel, Amplitude, Tableau, custom Elasticsearch, Datadog] - My conversation volume: [INSERT VOLUME — e.g., 50K conversations/month, 5K/day, 2M/year] ## RESPONSE FORMAT - Begin with a metrics hierarchy overview showing how operational metrics roll up to business KPIs in 5-7 bullet points - Use labeled sections for each analytics component with metric definitions, formulas, and threshold values - Include a dashboard wireframe described in text showing the layout of the real-time operational view - Provide a sample weekly report template with example data and commentary - End with an analytics implementation roadmap divided into Phase 1 (core metrics), Phase 2 (behavioral analytics), and Phase 3 (predictive insights)
Or press ⌘C to copy
Replace these placeholders with your own content before using the prompt.
[INSERT VOICE APPLICATION][INSERT BUSINESS GOALS]