Introducing Watchtower: Every Call. Scored.
Watchtower scores every AI-handled call on five quality dimensions and rolls them into a single health score. Know your agent is performing — without reading a single transcript.
When you hand your phone lines to an AI agent, one question keeps coming back: how do I know it's actually doing a good job?
You could listen to recordings. You could read transcripts. You could spot-check a handful of calls and hope the rest are fine. That's what traditional QA looks like — expensive, slow, and incomplete.
Watchtower is the opposite.
What Watchtower Does
Every call your Allison agent handles is automatically scored on five quality dimensions the moment it ends. No delay, no sampling, no human reviewers. Every single call.
The five dimensions:
Resolution (30% weight) — Did the caller get what they needed? If someone called to book an appointment, was it booked? If they had a billing question, was it answered? Resolution is the most heavily weighted dimension because it's the one that matters most to your callers.
Accuracy (25%) — Was the information correct? This is where hallucination detection lives. If your knowledge base says you're open until 6 PM, did the agent say 6 PM — or did it guess 5? Accuracy scoring catches the moments when an AI sounds confident but says something wrong.
Professionalism (20%) — Natural conversation flow, clear communication, appropriate greeting, smooth handoffs. This measures whether the call felt right. A caller who gets the right answer in the wrong tone is still a dissatisfied caller.
Sentiment (15%) — How did the caller feel? Not at one moment, but across the arc of the conversation. Did they start frustrated and end satisfied? Did the agent make things better or worse? Sentiment tracks the emotional journey of every call.
Escalation Handling (10%) — When a call needs a human, does the handoff happen correctly? When it doesn't need one, does the agent resolve it without unnecessary transfers? Both matter.
These five scores are combined into a single number from 0 to 100: your Agent Health Score.
The Health Score
Think of it like a credit score for your AI agent.
- 90-100 (Excellent): Your agent is performing at expert level. Most calls resolved cleanly, callers are happy, information is accurate.
- 75-89 (Good): Performing well with minor areas for improvement. This is where most well-configured agents land.
- 60-74 (Needs Attention): Consistent issues detected. Time to review flagged calls and update your knowledge base.
- Below 60 (Critical): Significant problems that need immediate attention. Watchtower will alert you before you get here.
The health score is a 30-day rolling average, weighted toward recent performance. If you have a bad week but fix the issues, your score recovers within days — not a month.
Zero Effort Monitoring
Here's the part that makes Watchtower different from "go check a dashboard every day":
Weekly digest emails. Every Monday morning, you get an email with your health score, dimension breakdown, flagged calls, and areas for improvement. If everything's fine, it takes 10 seconds to scan and move on. If something needs attention, you know exactly what.
Automatic alerts. If your health score drops below 70, you get an immediate alert — not Monday's summary. Same if your score declines for three consecutive periods.
Flagged calls. Watchtower automatically flags any call where a dimension scores 3 or below, the agent gave clearly wrong information, or a caller was left unresolved. These calls surface in your dashboard and weekly digest with specific rationales for why they were flagged.
You don't have to monitor anything. Watchtower watches, scores, and tells you when something matters.
How It Works (The Short Version)
There's no setup. There's no separate tool. There's no additional cost.
When a call ends, quality scoring happens automatically as part of the post-call analysis. Scores appear on your call history immediately.
The Watchtower dashboard in your account shows everything: health score with trend, dimension breakdown with averages, flagged calls with expandable rationales, and score distribution across all your calls.
Here's exactly what it looks like:
Watchtower
Agent quality monitoring — every call scored automatically
Agent provided incorrect office hours; caller became frustrated and ended call
Failed to escalate urgent dental emergency to on-call dentist
Missed appointment scheduling; caller asked twice with no resolution
Caller scheduled a cleaning for next Tuesday. Confirmed time and sent SMS reminder.
Caller's primary need (scheduling a cleaning) was fully resolved. Appointment booked, time confirmed, SMS reminder sent.
All information provided was correct — available time slots matched the schedule, provider name was accurate.
Natural greeting, appropriate pacing, clear confirmation of details. Caller experience was seamless.
Caller started neutral, ended pleased. Expressed satisfaction with the quick booking process.
No escalation needed and none was attempted. Correct handling of a routine call.
Caller asked about Saturday hours. Agent provided correct hours and offered to book.
Caller asked if Delta Dental is accepted. Agent confirmed and explained coverage.
Caller rescheduled Thursday appointment to Friday. Agent confirmed the change.
Caller reported severe tooth pain. Agent failed to transfer to emergency line.
New patient called to register. Agent collected info and booked initial exam.
Caller asked about outstanding balance. Agent provided correct amount.
Post-procedure follow-up call. Agent confirmed recovery instructions.
1–8 of 247
Why 100%, Not 5%
Traditional call QA works by sampling. A quality team listens to a random selection of calls — usually somewhere between 2% and 10% — and extrapolates from there.
If 3% of your calls have an accuracy issue and you're only sampling 5%, you might catch one. Or none. And you won't find out until a customer complains.
Watchtower scores every call because it can. There's no human listener bottleneck, and scoring happens within seconds of the call ending — not days or weeks later.
This means patterns surface immediately. If a new FAQ is causing wrong answers, you'll see accuracy scores drop in the next digest. If your escalation rules are misfiring, the escalation dimension will flag it. You don't wait for complaints to tell you something's broken.
What This Means for You
If you're running an Allison agent, Watchtower answers the trust question.
You don't have to wonder if calls are going well. You don't have to listen to recordings to find problems. You don't have to hire QA staff or build monitoring tools.
You get a score that tells you at a glance — and the dimension breakdown that tells you exactly where to look if something needs fixing.
Included on Every Plan
Watchtower is not a premium add-on. It's not a higher tier feature. Every Allison Voice plan — Starter, Professional, and Business — includes full Watchtower scoring, the dashboard, weekly digests, and alerts.
Quality monitoring is foundational, not optional. If you're trusting an AI with your phone calls, you should be able to see how it's performing. That shouldn't cost extra.
Watchtower is live for all Allison Voice customers. Explore how it works or call Allison to set up your agent and start seeing scores from day one.
Ready to try Allison?
Set up your AI voice agent in minutes. Start answering calls 24/7.