CanaryTel - Independent QA, compliance, and monitoring for answering services

Don’t say your agents follow the script.
Prove it.

Independent QA, compliance, and monitoring for answering services.

The problem

Why this matters

Script drift

Hundreds of client accounts. Scripts change, agents turn over, steps get skipped.

Unaudited overnight shifts

The calls that matter most happen when nobody is listening.

AI agents regress silently

A prompt or model change breaks a failure path. No one notices until a caller does.

One bad call, one lost client

Clients rarely complain first. They leave, or they escalate after the damage.

2:14 AM

A real kind of failure

Caller reports a child with a 104°F fever and trouble breathing.
Script: page the on-call pediatrician within 3 minutes.

02:14Call answered. Symptoms captured.

02:33Page sent. 19 minutes late. Wrong doctor.

Next dayNobody knew until the client complained.

QA today

Supervisor spot-checks

A handful

of calls reviewed by hand.
The rest, nobody hears.

Manual listening. Sampled, not systematic.

Happy paths only. Failure paths are rarely exercised.

No evidence trail you can hand to a client.

Introducing CanaryTel

The secret shopper, automated

We call your lines
before your callers do.

Scheduled, randomized, scored against each client’s script. Independent of your platform and your staff.

Who relies on the evidence

Nine parties, one method

Answering service operators

Clients of answering services

AI voice agent providers

In-house teams, contact centers, BPOs

Healthcare compliance, risk, and liability insurers

Compliance hotlines

Government agencies and public programs

Investors and acquirers

Franchisors and multi-location brands

CanaryTel sits between parties who each need to prove or verify something about a phone line. Operators prove script adherence to clients. Clients verify the vendor they pay. AI voice agent providers regression-test failure paths and show customers third-party evidence. In-house teams, contact centers and BPOs need the same as operators. Compliance, risk and liability insurers want evidence that after-hours escalation works. Corporate compliance and ethics hotlines need proof the hotline works from the first ring to a closed test report. Agencies and public programs sometimes require an independent secret-shopper. Investors and acquirers want a quality baseline that is not the seller’s own number. Franchisors want one standard across locations. Canary Calls apply to all nine; Canary Watch needs the data owner’s cooperation.

One test, every party

Neutrality

One scored call.
Evidence everyone can use.

No one grades their own homework

The operator, the AI vendor, and the client each have a stake. CanaryTel has none.

Contracts can name the evidence

Agree the measure before a dispute starts.

Fewer disputes

Same timestamps, audio, and transcript for every party. The argument moves to what to fix.

Tier 1: Canary Calls

End-to-end campaign testing

Happy and failure paths

Routine message taken correctly
Emergency escalation
Wrong number, hang-up mid-script
After-hours rollover, on-call not answering

Scored step by step

Every call is recorded, transcribed, and scored against the client’s script. Output: pass or fail per scenario and a script adherence score.

Delivery verified

Was the message dispatched, to the right on-call, within the SLA? Each test account’s roster points to a CanaryTel-owned test number, so latency and recipient are captured. Real clinicians are never contacted.

What a Canary Call result looks like

CALL cc_7f3a91 | Scenario: Pediatric urgent escalation | 02:14:07

✓greetingCorrect client greeting and recording notice00:04

✓captureName, callback number, symptoms captured00:41

✓triageFever and breathing classified URGENT01:12

✗pageExpected: Dr. Patel paged by 02:17. Actual: Dr. Okafor at 02:33+19:02

!closeReadback offered, callback ETA not given02:05

Script adherence 80%Escalation SLO FAILFinding F-2291 opened

Tier 2: Canary Watch

Monitoring · Early access

Real calls, scored

AI voice agents: live via webhook. Amtelco and Startel: via scheduled export or API, pilot-dependent. Handling time, disclosures, and script adherence against a rubric agreed per campaign; escalation correctness where dispatch logs exist.

SRE-style alerts

SLO burn and anomalies at the cadence of your data feed. Regression detection after a prompt or model change for AI agents that send deploy events. Per-client timeline and audit reports.

Tier 2 is Canary Watch, currently in early access. Where Canary Calls are scheduled probes, Canary Watch looks at real calls. Be precise about integration: AI voice agents connect live by webhook today; for Amtelco and Startel we work from scheduled exports or the platform API, and the exact path is settled during the pilot. Scoring runs against a rubric we agree per campaign, with sampling and human review of failures. Escalation correctness needs dispatch logs, so we score it where those are available. Alerts work like site reliability engineering: if the share of on-call escalations dispatched inside five minutes burns through its SLO, you hear about it that night, not at the quarterly review. For AI agents that send us version or deploy events, we flag regressions after a prompt or model change. Any plan that touches patient health information includes a BAA.

Uptime for every call path

SRE metrics

Escalation SLO over 30 days, dropping below the 99 percent target after a prompt change on day 19

Escalation SLO: urgent calls dispatched in under 5 minutes. Illustrative data.

Escalation SLO time-to-dispatch on urgent calls

Path coverage % of call paths exercised

Script adherence % of required steps done correctly

Built for the calls that matter

Verticals

Medical

On-call physician escalation. Urgent versus routine triage. Built for HIPAA-sensitive workflows. A missed page is a patient safety event.

Legal intake

Conflict-sensitive intake. Accurate capture of lead details. Speed to callback.

Property and HVAC dispatch

Correct emergency classification: no heat, gas leak, flood. Right technician paged. On-call rotation honored.

Works alongside your platform

Amtelco, Startel, and any answering

Amtelco

Genesis, Infinity, Intelligent Series call and message records.

Startel

CMC and Secure Messaging Plus call and message records.

AI or human answering

AI voice agents, BPOs, in-house after-hours teams. Anything that takes a call.

No rip and replace. We test from the outside, like a caller, and read what your systems already log, where you share it.

How it works

Architecture. A private evidence warehouse.

Test call runnerCanary Calls

Live logsAmtelco, Startel, AI

→

Transcriptsand logs

→

Timelinesper account, private evidence warehouse

→

Scoring agents

→

Findings

Alerts

Audit reports

Every call, dispatch event, and finding is stored on a per-account timeline. Rules and agents write findings back to the same record.

Tiers

Named tiers. Talk to us.

Canary Calls

Scheduled and randomized test calls
Happy and failure paths
Scored, with evidence

Canary Watch

Everything in Canary Calls
Real-call scoring (early access)
SLO alerts and audit reports

Enterprise

Custom scenario libraries
Client report portals, SSO
Custom integrations, regional test numbers

Start with a free week

The offer

1

Free one-week audit, one campaign

Seven days of Canary Calls: scheduled and randomized, happy and failure paths
Scored report with timestamps, audio, transcript
No integration, no cost, no commitment

2

Then a pilot

3 campaigns, 8 scenarios each
Two happy paths, six failure paths, agreed in advance
Baseline audit report to share with clients

Fictional callers and a CanaryTel-owned test on-call number. You must own the line or have the operator’s agreement.

Why independent matters

Evidence you can hand over

An internal check says
“we think so.”
An independent record says “here is the call.”

No referral fees or commissions from platforms or answering services.
Not owned by an answering service, platform, or AI vendor.
Results never edited at anyone’s request. Disputed findings get a documented re-test.
Methods disclosed to the paying party.

Industry awards already use mystery-caller test calls. Tools that test AI voice agents exist. We found no scored secret-shopper QA for both human and AI answering services.

Why independence matters. When a client asks whether their script is being followed, an internal answer is an opinion. An independent record, with audio, transcript, timestamps and a finding history per account, is evidence. Industry programs such as the ATSI Award of Excellence already use mystery-caller test calls, which shows the value of secret-shopper QA. Other tools we found (Hamming, Cekura, Coval) test AI voice agents only; we found none that does scored secret-shopper QA across human and AI answering. Phrase this carefully. The independence policy, also at /why-canarytel/: no referral fees or commissions from platforms or answering services; not owned by an answering service, platform or AI vendor; results are never edited at anyone’s request, and a disputed finding gets a documented re-test; methods are disclosed to the paying party. Operators can hand audit reports to clients as a trust and retention tool, and hand them to auditors as a compliance trail.

Independent testing is already expected

Precedent for an outside tester

Regulators require third-party phone testing in some programs.
Prosecutors ask whether companies test their hotline.
Auditors test phone lines with real calls.
The answering-service industry grades with mystery calls. We make it continuous.
Answer-time and accessibility targets are written into rules.

CanaryTel evidence supports these measures; it does not certify compliance.

Five points of precedent. One: CMS requires ACA Marketplace plans on the federal exchange to use a survey vendor separate and distinct from the issuer for appointment wait-time secret shopper surveys (https://www.cms.gov/files/document/awt-sss-tech-guide-qhp-ffe-508.pdf), and the Medicaid managed care rule requires states to use an independent entity for secret shopper surveys starting in 2028 (https://www.ecfr.gov/current/title-42/chapter-IV/subchapter-C/part-438/subpart-B/section-438.68). These programs test provider offices; we cite them as precedent for independence, not as services we perform. Two: the DOJ Criminal Division’s Evaluation of Corporate Compliance Programs, updated September 2024, asks “Does the company periodically test the effectiveness of the hotline, for example by tracking a report from start to finish?” (https://www.justice.gov/criminal/criminal-fraud/page/file/937501/dl). It is guidance for prosecutors, not a statutory mandate. Three: Canada’s Auditor General used test calls in its 2025 audit of Canada Revenue Agency contact centres (https://www.canada.ca/en/auditor-general/our-work/audit-reports/canada-revenue-agency-contact-centres.html). Four: industry awards for answering services score test calls placed by independent judges; CanaryTel makes that continuous. Five: Medicare Advantage plans must connect callers who need an interpreter within 8 minutes, and TTY callers within 7 minutes, on 80 percent of calls (https://www.law.cornell.edu/cfr/text/42/422.111); telecommunications relay services must answer 85 percent of calls within 10 seconds (https://www.law.cornell.edu/cfr/text/47/64.604). CanaryTel evidence supports these measures; it does not certify compliance.

Reports shaped by your obligations

Evidence toward what applies to you

Obligation source	What we test	Evidence
Client contract SLA	Escalation SLO, dispatch latency	Timestamped dispatch capture
Client script and escalation rules	Step-by-step scenario scoring	Transcript excerpts, pass or fail per step
Healthcare privacy (HIPAA-sensitive)	Fictional data in tests, access limits	Data-handling statement, BAA if PHI
Public program or agency contract	Secret-shopper scenarios from the contract	Per-scenario results
Insurer or risk requirements	Failure-path coverage	Path coverage and findings

A report is evidence toward an obligation, not a certification. Customers and their counsel decide what is required.

Request the free audit

One campaign. One week. Free.

canarytel.pages.dev → Request the free audit

Use the contact form on the site.

Amtelco and Startel are trademarks of their respective owners. CanaryTel is an independent service and is not affiliated with or endorsed by Amtelco or Startel.