Call QA platforms demo well. Every one of them will show you a dashboard with rising scores and a transcript with sentiment highlighted. The differences that matter are the ones that surface three months after rollout, when you try to change a scorecard, or your calls turn out to be in two languages, or finance asks what a call actually costs to analyse.
These are the twelve questions we would ask. They are deliberately answerable — each has a concrete right answer for your situation, and a vendor either can or cannot do it.
Scorecard and scoring
1. Can we change the scoring criteria ourselves, without a support ticket?
Some platforms ship a fixed model. That is fine until your sales floor and your support desk need different standards. Ask to see someone add a parameter live in the demo.
2. Can different teams have different scorecards?
A collections team, a technical support queue and an outbound sales floor do not share success criteria. If one scorecard applies to the whole account, one of those teams is being measured on the wrong things.
3. Can we set the weightings?
Equal weighting across parameters implies every behaviour matters equally, which is never true. Ask whether weights are editable and whether they must total a fixed number.
4. Can a supervisor override an AI score, and is the original kept?
This is the question that decides whether your team trusts the system. AI scoring gets calls wrong. If a supervisor cannot correct it, agents stop believing the numbers. If a correction erases the original, you lose the audit trail. You want both: override, and the original still retrievable.
Language and audio
5. What happens on a call that switches language halfway through?
Very common in multilingual markets and rarely handled well. Many platforms translate to English first, which loses tone and politeness registers — the things you are trying to measure. Ask specifically about code-switching, not just "do you support language X".
6. Is the audio analysed directly, or transcribed then scored?
Transcription discards delivery. Two agents can say identical words with opposite effect. If tone matters to you, ask what the model actually receives.
Getting calls in
7. How do recordings reach the platform, and what does that require from our phone system?
There are three realistic answers: your system pushes to their API, they pull from an endpoint you expose, or someone uploads files. Pull usually requires the least change on your side. Be wary of "we have a connector for that" without a description of what the connector needs.
8. Can we limit how many calls get analysed?
If billing is per minute and ingestion is automatic, an unusually busy month is an unusually large invoice. Ask for a daily cap, a percentage sample, and the ability to filter on your own metadata so only the calls you care about are analysed.
Governance
9. How long are recordings kept, and who controls that?
Retention is a compliance question in most regulated industries. You want a configurable window and confirmation that expiry means deletion from storage, not merely hidden from the interface.
10. Can we restrict what each user sees?
A team lead should see their department. An agent should see their own calls. Ask whether scoping is enforced when the data is fetched or filtered in the interface afterwards — the second is not a security control.
11. Can it be deployed where our recordings already live?
For some organisations call recordings cannot leave their own infrastructure. If that is you, ask about on-premise and hybrid deployment early, because it eliminates a large part of the market.
Cost
12. What does one call actually cost, all in?
Per-minute pricing is easy to compare until extras appear: custom parameters that change the rate, modules sold separately, minimums, minutes that expire monthly. Ask for the cost of a specific realistic month, with your call volume and your parameter list.
| What to ask for | Why it matters |
|---|---|
| Cost of 10,000 minutes with our scorecard | Reveals whether custom parameters change the rate |
| Do unused minutes expire? | Monthly expiry punishes seasonal volume |
| Which features are separate modules? | Dashboards and assistants are often priced apart |
| Is there a minimum commitment? | Determines whether you can start small |
| Per-call cost visibility | Finance will eventually ask for this |
A note on comparing vendors
Be sceptical of comparison tables published by vendors — including ours. Any table where one column has ticks all the way down was written to produce that result. The twelve questions above are more useful precisely because they do not have a universal right answer: on-premise deployment is essential for some buyers and irrelevant to most, and a platform optimised for one is a poor fit for the other.
Ask the questions that matter for your situation, ask for a demo using your own recordings rather than the vendor's, and treat anything that cannot be shown live as not yet existing.
The best test of a call QA platform is not the demo. It is a week of your own recordings, your own scorecard, and your own multilingual, badly-recorded, interrupted-by-hold-music reality.
Any vendor confident in their product will let you do that. Xperia includes $3 of free credit for exactly this reason — enough to score real calls from your own floor before you talk to anyone.