Are you using an AI chatbot for customer support? While these bots are powerful, they're not perfect. Enterpret can help you understand exactly how your chatbot is performing and identify clear opportunities for improvement.
Before creating your analysis dashboard, two important setup considerations:
Identify Your Chatbot: Most AI chatbots operate as a seat within your helpdesk. Look for identifiers like agent email or specific metadata fields that distinguish bot from human interactions.
Track Performance Tags: Most probably you already have a set of tags that either the AI bot or your human agents are adding to the tickets. If you don’t we recommend following the structure given below
Recommended Performance Tags:
Rick - Knowledge Gap ❓: Shows when bot couldn't answer due to missing information in its knowledge base
Rick - Ignored ⛔: Indicates when bot's didn’t respond to the customer's query
Rick - Handed Off 🔄: Tracks when bot appropriately transferred conversation to a human agent
Rick - Resolved ✅ : Marks conversations where bot successfully addressed the customer's need
Rick - Out of Scope 🎯: Identifies queries that fall outside the bot's intended capabilities
Rick - Replied 💬: Shows when bot provided an initial response to customer query
Rick - Escalated ⚡: Flags when human intervention was needed after bot attempted to help
Rick - Incorrect Path 🚫: Indicates when bot took wrong conversational direction with customer
Rick - Pending Resolution ⏳: Tracks conversations requiring additional follow-up or resolution
Rick - Needs Improvement ⚙️: Marks responses that were relevant but could be enhanced
Rick - Training Required 📚: Highlights areas where bot needs additional pattern recognition or responses
Rick - Perfect Response ⭐: Identifies ideal bot interactions to use as best practice examples
Pro Tip: Replace "Rick" with your chatbot's name in these tags and add them to your helpdesk if you don’t have similar tags already.
Now, let's build a dashboard that helps us
track:
Overall bot effectiveness
Knowledge gaps
Resolution rates
Areas needing improvement
Training opportunities
Step by Step Instructions
Create a Dashboard
Let’s create a dashboard called “Rick ( Replace this with your own AI bot’s name) Performance Analysis”
Start adding the quantify analysis listed below to put together a report. Use text block to add the section titles
Adding Quantify Charts
Click on Add Quantify on the Dashboard
One by One do the analysis mentioned below and add it to the dashboard
Quantify Analysis to Add
1. Bot Activity Overview
Let’s start by putting together a chart to tracks overall bot usage. This will also help spot any system issues or unexpected downtime.
Total Conversation Volume
Plot: Source
Filter: Agent Email = [Bot Email]
View: Trend + Anomalies
Rick Tags All Time
Plot: Tags ( from your helpdesk metadata)
Filter: Agent Email = [Bot Email]
View: Bar chart
Rick Top Reasons
Plot: Reasons
Filter: Agent Email = [Bot Email]
View: Bar chart
Rick vs Humans
Plot: Reasons
Use Compare
Filter A: Agent Email IS ANY OF [Bot Email]
Filter B: Agent Email ISN'T ANY OF [Bot email]
View: Bar chart
2. Resolution Success
Resolved vs Handed-Off Patterns
Shows which topics your bot handles well vs needs human help with.
Plot: Reasons
Use Compare
Filter A: Agent Email = [Bot Email] AND Tag = "Resolved"
Filter B: Agent Email = [Bot Email] AND Tag = "Handed Off"
View: Bar Chart
3. Knowledge Gap
Topics Needing Training
Identifies areas where bot needs knowledge base updates.
Plot: Reasons
Filter: Agent Email = [Bot Email] AND (Tag = "Knowledge Gap" OR Tag = "Training Required")
View: Bar Chart
4. Escalation Analysis
Incorrect Handling
Shows where bot is misunderstanding user intent or needs better routing.
Plot: Reasons
Use Compare
Filter A: Tag = "Incorrect Path"
Filter B: Tag = "Escalated"
View: Bar Chart
5. Quality Tracking
Helps track if your improvements are increasing quality responses.
Perfect vs Needs Improvement
Plot: Reasons
Use Compare Filter A: Tag = "Perfect Response"
Filter B: Tag = "Needs Improvement"
Duration: Last 3 months
View: Trend Chart Switch to table view on Dashboard
6. Scope Analysis
Topic Coverage
Shows which new topics you might want to train your bot on.
Plot: Reasons
Use Compare
Filter A: Tag = "Out of Scope"
Filter B: Tag = "Resolved"
View: Bar Chart
7. Response Issues
Ignored vs Replied
Identifies potential bot responsiveness issues.
Plot: Reasons
Use Compare
Filter A: Tag = "Ignored"
Filter B: Tag = "Replied"
View: Trend
Pro Tip: Set up regular dashboard subscriptions to track these metrics weekly. Look for topics that move from "Knowledge Gap" to "Resolved" to validate your improvements.