The Local
TTS Shootout

№ 01The Voices

Every model reads the same four passages: a casual conversational line, an exclamation-heavy expressive piece, a numbers-and-dates gauntlet ("$24.99 to $9.50 per million characters"), and a 90-word long-form narration that punishes models with weak pacing. All clips are loudness-normalized to −18 LUFS so your ears judge timbre and prosody, not gain. Ranking is my read of the evidence — arena Elo data, expressiveness, license, speed. Your ears get the final vote.

№ 02The Eval Report

Protocol

Each model read the same four passages, chosen to stress different failure modes: conversational (casual register, 27 words), expressive (exclamations, an ellipsis, repeated emphasis — 31 words), technical (dates, dollar amounts, decimals, an acronym — 36 words), and long-form (a 90-word narration testing pacing and end-of-speech discipline). One run per passage per model, no cherry-picking: the first completed generation is the published sample.

Conditions held constant: same Apple M5 / 32 GB machine, no other GPU load, machine kept awake with caffeinate, every model on its default/recommended preset voice (Kokoro af_heart, F5's built-in reference; MisoTTS, Chatterbox, and Dia on their defaults) at out-of-the-box settings. Dia received its required [S1] speaker tag. Wall-clock time was recorded per clip from inside each driver; real-time factor is generation seconds divided by seconds of audio produced — below 1.0 means faster than playback. Raw WAVs were timed and archived; only the published MP3s are loudness-normalized (−18 LUFS).

Quantitative findings

Measured real-time factor per passage (lower is better):

¹ first call includes pipeline warm-up; Kokoro's steady state is ~0.09–0.10.

Three quantitative signals beyond speed:

• Pacing, via long-form duration. The same 90 words became 45.9 s of audio from MisoTTS, 40.2 s from Kokoro, 37.2 s from F5, 31.8 s from Chatterbox — and 19.6 s from Dia, which logged EOS detected at 55% of its token budget. Anything under ~30 s for this passage means rushing or truncation; Dia did both, Chatterbox is brisk but complete.
• Output consistency, via RMS level. MisoTTS produced three clips at RMS ≈0.02 and one at ≈0.21 — a 10× gain spread at identical settings. Every other model stayed within ±5% of its own average. Harmless after normalization, but it means MisoTTS needs a loudnorm pass in any pipeline.
• Footprint. Disk: 0.3 GB (Kokoro) → 1.5 GB (Chatterbox, F5) → 3.2 GB (Dia) → 12 GB (MisoTTS 8-bit). Peak memory during generation tracked download size; everything fit comfortably in 32 GB with the 8B model running alone.

Qualitative findings

Limits of this eval

Read the findings with their boundaries: one preset voice per model (cloning-capable models were not tested on cloned voices), one generation per passage (no seed averaging), no word-error rate scoring, and no human listening panel — the ranking triangulates measured speed and stability with public blind-vote arena Elo (Kokoro ~1056, Chatterbox ~1006, vs xAI cloud ~1197), and those arena figures are single-sourced because the verification stage of the research run was rate-limited. MisoTTS is too new to appear in arena data at all; its rank-one position here rests on direct listening against the others, which is exactly what the players above let you re-do yourself.

№ 03The Models

The field splits into three architectural camps. Kokoro is a classical parametric pipeline — grapheme-to-phoneme front end feeding a small StyleTTS2-derived decoder — which is why 82M parameters punch so far above their weight and why it never mispronounces a date. MisoTTS and Dia are autoregressive LLM-style token models: a language-model backbone predicts discrete audio codec tokens, which buys expressiveness and costs speed plus occasional discipline problems (Dia's early end-of-speech). F5-TTS is flow-matching diffusion — iterative denoising, great cloning fidelity, inherently slow. Chatterbox sits in between: token-based but distilled and tight.

Why not the others? Fish Audio S2 Pro tops the open-weights arena (Elo ~1118) but wants a serious GPU. XTTS-v2 is two generations stale (Elo ~886) and non-commercially licensed. Orpheus and Maya1 are CUDA-first with no maintained MLX port at test time. Cloud reference: blind-arena data puts xAI's TTS (the service this exercise replaces) around Elo 1197 — the gap to the best local options is real but has collapsed from "obvious" to "minutes-of-listening" in about a year.

¹ free; requires "Miso Labs" UI credit only above 50M MAU or $10M monthly revenue.  ² weights non-commercial (Emilia dataset); code is MIT.  ³ real-time factor: generation seconds per second of audio, lower is better, measured on an Apple M5 / 32 GB with the machine awake (see Field Notes).

№ 04The Pipeline

Every model lives in its own throwaway uv virtualenv — these stacks have mutually hostile dependency trees, and isolation is cheaper than debugging them. Three runtimes cover all five models; mlx-audio alone covers three of them on Apple Silicon.

# 1 · one venv per model family
uv venv envs/mlx --python 3.12
uv pip install --python envs/mlx/bin/python mlx-audio soundfile

# 2 · same four passages through every model (mlx-audio family)
envs/mlx/bin/python eval/gen_mlx.py mlx-community/MisoLabs-MisoTTS-8bit misotts
envs/mlx/bin/python eval/gen_mlx.py mlx-community/chatterbox-fp16        chatterbox
envs/mlx/bin/python eval/gen_mlx.py mlx-community/Dia-1.6B-fp16          dia --dia-tags

# 3 · the two non-mlx-audio stacks
envs/kokoro/bin/python eval/gen_kokoro.py     # kokoro + misaki, PyTorch/MPS
envs/f5/bin/python     eval/gen_f5.py         # f5-tts-mlx

# 4 · fair-listening pass: loudness-normalize every clip to -18 LUFS
ffmpeg -i in.wav -af "loudnorm=I=-18:TP=-1.5:LRA=11" -b:a 192k out.mp3

# 5 · keep the Mac honest during overnight runs
caffeinate -dims

Each run records wall-clock per clip into timings.json — the RTF numbers in the table come straight from those files, not from papers.

№ 05For PMs

Cost structure flips at zero.

Cloud TTS bills per character; a local model bills once in disk space. For an app speaking a million characters a month, the difference is a recurring four-figure annual line item versus a one-time 12 GB download. The catch is floor quality: you're shipping last year's cloud voice, not this year's.

Latency is now a locality question.

Kokoro starts speaking in well under a second on a laptop — faster than most API round-trips. For voice UI, "local" beats "better" whenever the better voice is 400 ms of network away.

The quality envelope collapsed this year.

Blind-arena spread between xAI's cloud TTS and the best free local model is now ~80–140 Elo — audible side-by-side, invisible in isolation for most content. MisoTTS shipped days before this test; the envelope is still shrinking.

Licenses are the real differentiator.

Four of five models here are genuinely commercial-safe. The trap is the plausible-looking exception: F5's code is MIT but its weights are non-commercial. Audit the weights license, not the repo badge.

Small models are distribution-grade.

Kokoro is 300 MB and Apache-licensed — small enough to embed inside an app binary or run in the browser via WebGPU. "TTS as a bundled asset" is now a real product shape.

№ 06Field Notes

• spaCy's downloader silently murders Kokoro. Kokoro's G2P needs en_core_web_sm; spaCy's auto-downloader detects uv on PATH and runs uv pip install outside the venv, which fails and kills the process with exit 2 — no traceback. Fix: pre-install the model wheel from the spacy-models GitHub release.
• macOS sleep poisons benchmarks and downloads. One F5 clip "took" 49 minutes of wall clock because the lid effectively closed; the MisoTTS download stalled at 3.9 GB with the process at 0% CPU on a dead TCP connection. caffeinate -dims before any long run, and treat a frozen byte-count as "kill and resume," not "wait."
• huggingface-cli is dead, long live hf. The old binary now prints help text and exits 1. Resumable downloads: hf download <repo>.
• Dia needs stage directions. Text must be wrapped in [S1] speaker tags, and even then it treats a 90-word monologue as optional — EOS fired at 55% on the long passage. A dialogue model is not a narrator.
• MisoTTS gain wobbles between generations. Three clips landed at RMS ~0.02 and one at ~0.21 — a 10× spread from the same model and settings. Loudness-normalize anything you publish.
• "8-bit 8B" still means 12 GB. The quantized MisoTTS repo is 12 GB on disk (codec, conditionals, and config ride along). Budget disk and download time accordingly.
• One venv per model, no exceptions. The five stacks pin conflicting torch, numpy, and transformers versions. uv venv makes isolation a two-second decision.