Introduction to Evaluations & Guardrails

Evaluations and guardrails are the core of Tuner’s monitoring capabilities. This guide explains what they are, how they work, and how to use them to improve your voice agent’s quality.

What Are Evaluations?

An evaluation is the process of analyzing a call against a set of criteria. Think of it as an automated report card for every conversation. Instead of manually listening to calls, you define what matters and let Tuner do the analysis at scale.

Why Evaluations Matter: Without evaluations, you’re flying blind. You might know that a call “failed,” but you won’t know why. Evaluations give you the “why” by checking for specific behaviors, metrics, and outcomes.

What Are Guardrails?

A guardrail is a set of rules that define what “bad” looks like for your agent. They allow you to catch issues early by automatically flagging calls that exhibit problematic behavior. When a guardrail is breached, it can trigger a red flag or an alert.

Why Guardrails Matter: Guardrails are your safety net. They ensure that even if you’re not actively monitoring, you’ll be notified when something goes wrong.

How They Work Together

1. Evaluations Analyze

   The evaluation pipeline analyzes every call based on your criteria.

2. Guardrails Monitor

   Guardrails monitor the results, looking for breaches.

3. Red Flags & Alerts Notify

   When a guardrail is breached, a red flag is created and an alert is sent.

Types of Evaluations in Tuner

Behavior Checks

Behavior checks are custom evaluations that verify whether your agent performed specific actions or followed certain guidelines during a call. You define the criteria, and Tuner automatically checks every call against them.

Example: Create a behavior check called “Appointment Confirmed” that verifies the agent explicitly confirmed the date, time, and location of an appointment before ending the call.

See Creating Custom Evaluations.

Voice Metrics

Voice metrics are pre-defined, quantitative measurements that Tuner computes automatically for every call. These metrics give you objective data about the conversation without requiring any setup.

Example: The “Talk-to-Listen Ratio” metric shows how much your agent spoke compared to the user. A ratio that’s too high might indicate the agent is dominating the conversation and not letting the user speak.

See Introduction to Pre-defined Metrics.

Labels & Red Flags

The Rule Builder allows you to create custom labels and red flags based on a combination of behavior checks, voice metrics, and other conditions. This gives you fine-grained control over how calls are categorized and flagged.

Example: Create a red flag called “High-Risk Call” that triggers when the call duration exceeds 10 minutes AND the user sentiment is negative AND the “Identity Verified” behavior check fails.

See How to Use Red Flags for Issue Triage.

Common Use Cases

Detecting Hallucinations

Create a guardrail that flags calls where the agent provides incorrect information.

Ensuring Compliance

Create a behavior check that verifies the agent always reads a required disclaimer.

Monitoring Sentiment

Create a guardrail that alerts you when user sentiment is consistently negative.

---

Next Steps

Learn about the pre-defined metrics that Tuner computes for every call.

Introduction to Pre-defined Metrics

An explanation of all out-of-the-box metrics and what they measure.