Session metrics and definitions

Definition

The Agent Success metric evaluates whether an interaction successfully achieved its business objective. Each session is assigned one of three success codes based on the outcome. Yes-A (Value: 1) represents an ideal scenario where the agent flawlessly completes its primary task without any issues. Yes-B (Value: 1) indicates a fallback success, meaning the agent encountered a problem but successfully utilised a backup plan to save the customer lead. Finally, No (Value: 0) denotes a failure where a technical or logical error resulted in a completely lost opportunity.

Examples: Home services (plumbing or HVAC)

Agent Success = "Yes-A" (Flawless Execution): The system logs a "Yes-A" when the AI perfectly follows its primary instructions from start to finish. There are no technical hiccups, errors, or conversational missteps. Even if the caller ultimately decides not to book a service, the AI is considered fully successful because it executed its duties exactly as intended.

• Example: A caller requests an emergency plumbing repair. The AI seamlessly collects their contact details, explains the standard diagnostic fee, and locks in an appointment on the calendar without missing a beat.

Agent Success = "Yes-B" (Recovered Success): A "Yes-B" rating applies when the AI hits a roadblock—like a software glitch or a conversational misunderstanding—but manages to pivot to a backup plan. By correcting itself or using an alternative method, the AI successfully rescues the interaction and secures the lead.

• Example 1: The AI attempts to book a furnace tune-up, but the scheduling software temporarily goes offline. Rather than hanging up, the AI apologises for the system delay, takes down the customer's information manually, and instantly alerts the human dispatcher to call them right back.
• Example 2: A caller asks for a Sunday appointment. The AI initially states the business is closed, but immediately catches its mistake, checks the weekend on-call roster, and successfully books the emergency technician.
• Example 3: After agreeing on a time for a roof inspection, the system fails to send the standard confirmation email. Noticing the error, the AI automatically triggers a text message confirmation to the customer's phone instead, ensuring the appointment is locked in.

Agent Success = "No" (Critical Failure): The "No" status is given when the AI suffers a fatal error, either technical or conversational, that it cannot recover from. This results in a dead end for the customer and a lost business opportunity.

• Example 1: A new customer calls about a burst pipe. The AI gets stuck in a loop repeatedly asking for an existing account number. Unable to bypass the prompt, it drops the call without offering any real help.
• Example 2: The AI confidently tells a caller, "I've booked your HVAC repair for 2 PM today," and ends the conversation. However, a logic error caused it to skip asking for the customer's physical address, rendering the appointment useless and completely losing the job.

Definition

The Conversation Score measures the overall quality of your AI Receptionist’s interactions by averaging the Conversation Success Score (CSS) across a customer's rated sessions. To generate this metric, individual interactions are evaluated using a 1-5 star widget on the Sessions page. Each star rating is then translated into a CSS value, with every star representing 20% (CSS = 0.2 × Star Rating).

When calculating the final score (the sum of all CSS ratings divided by the number of assessed sessions) unassessed and test sessions are completely excluded so they do not skew your data. Ultimately, a higher score (approaching 1.0 or 100%) indicates highly successful outcomes, while a lower score flags that your AI Receptionist’s conversational logic or training may need some adjustments.

Example: Mixed performance

An AI Receptionist has completed 30 total sessions, of which 25 have been assessed with star ratings:

• 10 sessions rated 5 stars (CSS = 1.0 each)
• 5 sessions rated 4 stars (CSS = 0.8 each)
• 5 sessions rated 3 stars (CSS = 0.6 each)
• 5 sessions rated 2 stars (CSS = 0.4 each)
• 5 sessions unassessed (excluded)

Sum CSS values: (10 × 1.0) + (5 × 0.8) + (5 × 0.6) + (5 × 0.4) = 10 + 4 + 3 + 2 = 19.0

Divide by assessed sessions: 19.0 ÷ 25 = 0.76

Result: The Conversation Score is 76%. This indicates the agent is performing adequately but is delivering inconsistent experiences, highlighting an opportunity to review the lower-rated sessions for logic adjustments.

Definition

Recognition determines exactly how much credit the AI Receptionist gets for deals generated during business hours, representing the percentage of wins that would have been lost without it. How this credit is calculated depends on how your call flow is set up in the "ASR - Redirection Type (Working Hours)" attribute.

If your business uses an Overflow setup (where humans try to answer first and the AI steps in for missed calls), the AI receives 100% Recognition because every deal it closes is one your team couldn't get to. On the other hand, if you use a Non-overflow setup (where the AI answers first and humans act as a fallback), the Recognition value is set to your historical missed-call percentage. Because your human staff would have naturally answered a portion of those calls, the AI is only credited for the specific share of deals it rescued from slipping through the cracks.

Example: A busy dental clinic

For example, out of 500 incoming calls, your front desk staff managed to answer 400 (i.e., 20% of calls were missed). After implementing an AI Receptionist, your clinic stopped losing those 20% of calls. Therefore, 20% of all appointments generated by the AI Receptionist are considered exclusively attributable to it. In this example, the Recognition attribute is 20.