AI Impact dashboard provides a comprehensive view of how AI impacts customer support operations. It helps you measure AI engagement and resolution, compare AI and staff performance, understand customer intents and sentiment, track AI-generated labels, and monitor staff adoption of AI-powered features.
Navigating to AI Impact
Navigate to Analytics from the main navigation menu.
Choose Helpdesk.
Click on AI Impact to open the dashboard.
Key Metrics
The AI Impact dashboard provides the following key metrics:
AI Engaged: Total number of conversations in which the AI agent engaged with the shopper.
AI Resolution Rate: Percentage of conversations resolved by the AI agent.
Staff AI Usage: Total number of times staff members used AI-powered features to assist with their work.
Time Saved by AI: Estimated amount of staff time saved through the use of AI assistants.
Conversation Journey
The Conversation Journey section shows how conversations progress from creation through AI engagement and resolution. It includes two views: Funnel and Outcome.
Funnel
The Funnel view shows the number of conversations progressing through each stage:
Conversations Created: Total number of conversations created during the selected period.
AI Engaged: Number of conversations in which the AI agent engaged.
AI Resolved: Number of AI-engaged conversations resolved by the AI agent.
Handed Over to Staff: Number of AI-engaged conversations handed over to staff.
The percentage displayed at each stage represents the proportion of conversations that reached that stage relative to the relevant preceding stage.
Use the Funnel view to understand conversation volume at each stage and identify where conversations drop off or require staff intervention.
Outcome
The Outcome view shows how AI-engaged conversations were ultimately resolved:
AI Resolved: Number and percentage of conversations resolved by AI.
Staff Resolved: Number and percentage of conversations resolved by staff.
Use the Outcome view to compare AI-led and staff-led resolutions and understand the contribution of AI to overall resolution.
Support Performance
The Support Performance section compares AI and staff performance across Handling Time, First Response Time, and Resolution.
Handling Time
Handling Time shows the average time taken to handle a conversation:
AI Agent AHT: Average handling time for conversations handled by the AI agent.
Staff AHT: Average handling time for conversations handled by staff.
AHT with AI Assistance: Average handling time for conversations where staff used AI Agents.
Use these metrics to compare handling efficiency across AI, staff, and AI-assisted conversations.
First Response Time
First Response Time shows the average time taken to provide the first response:
AI Agent FRT: Average first response time when the AI agent provides the response.
Staff FRT: Average first response time when staff provides the response.
FRT with AI Assistance: Average first response time for conversations where staff used AI Agents.
Users can enable or disable any metric on the graph to compare only the metrics they want to view.
For example, users can disable AI Agent FRT and view only Staff FRT and FRT with AI Assistance.
Use these metrics to compare response speed across AI, staff, and AI-assisted conversations.
Resolution
Resolution compares the average time taken to resolve conversations:
AI Agent Resolution: Average resolution time for conversations resolved by the AI agent.
Staff Resolution: Average resolution time for conversations resolved by staff.
Use these metrics to compare resolution efficiency between AI and staff.
Intents, Resolution & Sentiment
This section provides insights into the top 5 customer intents and their corresponding performance metrics.
Metric | Description |
Intent | The top 5 intents identified from customer conversations. |
Volume | Number of conversations associated with each intent. |
Resolution | Number and percentage of conversations for the intent that were resolved. |
Handover to Staff | Percentage of conversations for the intent that were handed over to staff. |
Sentiment – AI Handled | Sentiment of conversations handled by the AI agent for each intent. |
AI Labeling
The AI labeling section provides an overview of labels applied by AI and the coverage of AI labeling across tickets.
Total AI Labels Applied: Total number of labels applied by AI.
Tickets with AI Labels Applied: Number of tickets where AI applied at least one label.
Coverage: Percentage of total labels applied by AI, calculated as AI labels applied ÷ total labels applied × 100.
Top 10 Labels Applied by AI
Displays the top 10 labels applied by AI in a graph.
Label Name: Name of the label applied.
Number of Labels Applied: Total number of times the label was applied by AI.
Conversation Sentiment
Shows the distribution of Positive, Negative, and Neutral sentiments generated by AI, along with the count and percentage of conversations for each sentiment.
AI Adoption
The AI Adoption section shows how staff use AI-powered features and the overall spam rate.
Rephrase: Rephrase feature usage by staff.
Help Me Write: Help Me Write feature usage by staff.
Product Recommendation: Product Recommendation feature usage by staff.
Spam Rate: Percentage of conversations marked as spam by the AI agent.
Staff Performance
The Staff Performance section provides performance and AI usage metrics for individual staff members.
The following metrics are displayed by default:
Metric | Description |
Staffs | Name of the staff member. |
Tickets Handled | Number of tickets handled by the staff member. |
AI Usage | Number of times the staff member used AI features. |
AHT | Average Handling Time for the staff member. |
FRT | Average First Response Time for the staff member. |
SLA Breached | Number of tickets where the SLA was breached. |
Additional metrics can be viewed by clicking Manage Columns and enabling the required columns:
Metric | Description |
CSAT – Positive | Number of positive CSAT responses received. |
CSAT – Neutral | Number of neutral CSAT responses received. |
CSAT – Negative | Number of negative CSAT responses received. |
Rephrase Usage | Number of times the staff member used the Rephrase feature. |
Help Me Write Usage | Number of times the staff member used the Help Me Write feature. |
Product Recommendation | Number of times the staff member uses Product Recommendation |
Filters & Reports
AI Analytics provides filters and downloadable reports to help analyze data based on the required date range and store.
Filters
Date Filter: Allows users to select the required date range. The analytics data and downloaded reports reflect the selected date range.
Channel: Allows users to filter analytics data by the selected support channel.
Reports
The following reports are available for download:
Support Performance Report: Contains support performance metrics.
AI Performance Report: Contains AI performance and usage metrics.
Staff Performance Report: Contains staff-level performance and AI usage metrics.
Downloaded reports reflect the selected date range and store filters.
FAQs
What does AI Engaged mean?
AI Engaged represents the total number of conversations in which the AI agent engaged with the shopper
What does Time Saved by AI represent?
It represents the estimated amount of staff time saved through the use of AI Agents.
What is the difference between AI Resolved and Staff Resolved?
AI Resolved represents conversations resolved by the AI agent, while Staff Resolved represents conversations resolved by staff.
Can I download AI Analytics reports?
Yes. You can download the Support Performance Report, AI Performance Report, and Staff Performance Report.
Can I customize the First Response Time graph?
Yes. The First Response Time graph is dynamic, allowing you to enable or disable individual metrics.
For further assistance or to raise feature requests related to Helpdesk Analytics, please contact [email protected].