Transcription accuracy is not a fixed number attached to a tool. The same service can produce a near-perfect transcript of a quiet one-on-one interview and mangle a lecture recorded from the back of a hall, because error rates shift with the audio, the speakers, and the vocabulary — not just the software. No tool on the market hits a single documented accuracy figure across all conditions, and any vendor quote that omits the audio conditions is selling you the best case.
That has a practical consequence. Before you pick between Whisper, Otter, Rev, or a human service, the honest question is not "which is most accurate?" but "which holds up on my audio?" — your meeting room, your colleagues' accents, your field's jargon. This piece explains how transcription error is measured, what the documented evidence says about the conditions that break transcripts, and how to run a fair check on your own recordings without trusting anyone's marketing. This connects to our earlier piece, Which Claude tier for which work? A decision guide from Anthropic's own pricing page.
How is transcription accuracy actually measured?
The standard yardstick is word error rate, usually shortened to WER. It is the percentage of words the transcript gets wrong, counting substitutions (the wrong word), deletions (a word dropped), and insertions (a word added). A WER of 10 percent means roughly one word in ten is off. Researchers and vendors both use it, which makes it comparable — with one large caveat.
The caveat is that WER depends entirely on the test set. A benchmark built from clean, read-aloud audiobook recordings will flatter any system. A benchmark built from noisy conference calls will punish it. When a vendor publishes an accuracy claim, the useful follow-up question is always: measured on what recordings, by whom? Without that, the number describes their demo, not your desk.
There is a second gap worth naming: WER treats every word equally, but transcripts are not read that way. A mistyped filler word costs nothing. A misheard drug name, statute number, or student's name can cost a correction cycle — or worse. A transcript can post a respectable WER and still fail at the words you actually needed.
Which audio conditions break a transcript first?
The failure modes are consistent across automatic systems, and they are worth listing because they predict where your own transcripts will go wrong:
- Background noise and distance. Speech recorded far from the microphone, or over room echo, loses the acoustic detail models rely on. Cafes, hallways, and lecture halls are the classic killers.
- Overlapping speakers. When two people talk at once, every system has to guess who said what. Speaker-labeling features help with attribution, but they do not recover words that were never cleanly recorded.
- Accents and dialects. Speech recognition systems are trained on data, and training data is not distributed evenly across accents. Documented evaluations of speech systems have repeatedly shown performance varying by speaker group, which is why a fair test uses recordings from your real speakers, not a studio narrator.
- Technical and domain jargon. Names of medications, legal terms, product codenames, and student names are the words models see least in training. They are also the words you care about most. Proper nouns are the usual casualties.
- Audio compression. Calls routed through conferencing software arrive pre-compressed, and compression strips exactly the frequencies that help disambiguate similar-sounding words.
None of this is a secret. It falls out of how these systems work: a speech model maps sound to likely words, and every condition above makes more than one word plausible. The model picks the likeliest. Sometimes it picks wrong.
What this means for Whisper, Otter, and Rev
The three sit in genuinely different positions, and the differences matter more than any headline accuracy figure.
Whisper is OpenAI's open-weights speech recognition family, with several model sizes you can run locally or through an API. The upside of open weights is verifiability: the model card is public, the versions are named, and independent researchers test it on public benchmark sets rather than the vendor's own. The trade-off is that you get a raw transcript engine. There is no built-in speaker labeling, no editing interface, no meeting integration — you assemble the workflow yourself, or run it inside another tool. For someone comfortable with a terminal, that control is worth a lot; for everyone else it is homework. Our earlier comparison of open-source Whisper against the bundled convenience tools covers that split in detail. Readers following this should also see AI transcription, compared: open-source Whisper versus the bundled convenience tools.
Otter is a meeting-focused service: it joins calls, produces a live transcript, labels speakers, and generates summaries. Its accuracy on clean, well-miked meeting audio is generally good, and its convenience is real. The limits are the limits of any closed, cloud-only service: you cannot inspect the model version handling your audio the way you can with Whisper, you depend on the vendor's pricing and data-use terms, and noisy or heavily accented audio still produces the errors described above. The summary features are a convenience layer, not a verification layer — a confident summary can be built on a misheard word.
Rev is the interesting case because it sells both an automatic product and human transcription, which makes the machine-versus-human trade explicit. Human transcription is slower and costs more, but a trained transcriber catches things machines still drop — especially jargon, names, and overlapping speech — and human services typically run their work past editors before delivery. The freelance platform GoTranscript describes exactly this quality loop on its hiring page: applicants take a transcription test, editors review samples of completed work "for accuracy, formatting, timestamps, and style," and maintaining a high quality score unlocks better-paid assignments. That is the process a human transcript buys you, and it is the reason human services remain the fallback when a machine transcript is not good enough.
The honest summary: for quick meeting notes where you were in the room and can correct from memory, an automatic service is usually fine. For interviews you will quote, research calls, legal or medical content, or anything with heavy jargon, budget for either a human pass or your own careful review against the audio.
How to test a tool on your own audio
Since no published benchmark matches your recordings, run a small one yourself. It takes under an hour:
- Pick two or three real recordings that represent your worst case, not your best — a noisy room, an accented speaker, a jargon-heavy call.
- Transcribe a two- to three-minute slice with each tool, noting the model version or service plan and the date. Versions change monthly, and last quarter's result may not describe this quarter's model.
- Check the transcript against the audio at the spots that matter: names, numbers, technical terms. Those are where errors do damage.
- Count errors on one fixed slice if you want a rough comparable number — the same slice, the same rules, for every tool. It will not be a laboratory WER, but it will be your WER.
- Note what fixing the errors cost you. A transcript that needs thirty minutes of correction is not free, whatever the subscription says.
One more lever people forget: the recording. Moving the microphone closer, or asking for a quiet room, improves every downstream tool more than switching tools does. Audio quality is upstream of everything.
What the evidence does not settle
Published head-to-head error rates for these specific products, across controlled audio conditions, are thin and often vendor-adjacent. Independent benchmark sets exist for speech recognition generally, but they age quickly as models update, and they rarely include the meeting-room and classroom audio most readers actually have. So treat any single accuracy figure — ours included, since we have cited none without conditions — as one data point, and your own two-minute test as the more honest one.
The working takeaway: match the tool to the stakes. Automatic transcription for notes and drafts; human review or human transcription when the words will be quoted, filed, or graded. And when a vendor quotes an accuracy number, ask what audio it was measured on before you let it make the decision.
