How to analyze Aura AI Assistant performance in the Insights section

The Insights section in Customerly is your control center for monitoring the efficiency of your customer support—especially the performance of your AI teammate, Aura. This revamped area includes several new metrics designed specifically to help you understand and improve how Aura interacts with your customers.

You can access the Insights section by clicking on it in the left-side navigation menu, just under the Help Center. Once inside, you’ll find multiple panels, but for this article, we’ll focus on the Aura AI Assistant section.

This section is dedicated to tracking how well Aura handles your customer conversations. You’ll find key performance indicators (KPIs) and detailed metrics that reflect Aura's ability to autonomously resolve, assist, or escalate conversations.

Aura resolution rate

At the top of the dashboard, you'll see the Aura Resolution Rate—the main KPI. It’s calculated based on:

• Conversations fully resolved by Aura: These are completed without any human involvement.

• Assumed resolved conversations: Aura handled these, but a human teammate closed them, assuming they were successfully resolved.

• Escalated conversations: These were passed to a human due to various reasons such as low confidence, missing information, or explicit customer request.

Use the date range filter to view performance over specific time periods (daily, weekly, or monthly).

Conversation activity breakdown

Scrolling further down reveals a breakdown of Aura’s activity:

• Conversations resolved by Aura

• Assumed resolved conversations

• Escalated conversations

• AI involved conversations (Aura contributed at least one message)

• AI not involved conversations (fully managed by human agents)

This helps you assess how engaged Aura is in your overall support flow.

Escalated conversations

Understanding why Aura escalates conversations is crucial for improvement. Here are some common escalation reasons:

• Low confidence in the available answer

• Repetition loops where Aura recognized too many new conversations coming from the same customer.

• Incomplete customer information, such as vague questions or just a screenshot

These insights give you direct pointers on where to improve Aura’s sources (Help Center articles, canned responses, etc.).

Track efficiency by conversation topic

Customerly also includes Conversation Topics, which cluster similar conversations together. This allows you to correlate Aura’s performance with specific topics:

• Which topics get escalated the most?

• Which are resolved effectively by Aura?

• Are there recurring issues like bugs or technical problems?

For example, Aura may excel at answering simple account questions but struggle with complex billing inquiries. This insight helps you refine support documentation or add new intents and canned responses.

How to act on the insights

Once you’ve reviewed the Aura metrics, you can take action:

1. Improve content sources: Add or update Help Center articles and canned responses.

2. Refine intents: Create or adjust intents in Chatflows to better capture customer needs.

3. Monitor and adapt: Use weekly or monthly filters to observe trends and track progress over time.

By treating Aura as a real teammate, regularly reviewing her performance, and making data-backed improvements, you can deliver a better support experience for your customers and lighten your human agents' workload.

Need help improving Aura’s results?
Contact us or explore related articles on creating intents, refining chatflows, and using AI Missions to automate complex tasks.