What is speaker diarization?
The "who spoke when" problem, solved automatically.
You have a recording with several people talking — an interview, a podcast, a meeting, a lecture with Q&A. Speaker diarization finds, second by second, which speaker is talking, then writes one track per person.
Drop a 30-minute podcast with 3 hosts → get back podcast_speaker1.wav, podcast_speaker2.wav, podcast_speaker3.wav, each containing only that person's lines, every other voice silenced. The split follows the voice (timbre, pitch), not the audio channels — it works on a plain mono or stereo mix.
The chore it removes
The old way. Open the recording in an audio editor, scrub through it, cut every sentence of the wrong speaker, fill the gaps with silence so the tracks stay in sync. Hours for a one-hour podcast — and one stray word from the wrong voice contaminates whatever you feed it to next.
With Diarizer. Drop the file, set the number of speakers (or auto), click Diarize. On our RTX 5080 a one-hour, two-host podcast is done in 1 min 50 s; on the processor, count about 15–20 seconds per minute of audio. Check the timeline, press Play on each speaker, and the tracks are ready — already aligned with the original.
What creators use it for
🎤
Voice cloning
A voice model needs minutes of clean, single-speaker audio, and real recordings are rarely clean. Diarize the interview and keep the track of the person you need.
👄
Lip-sync
Lip-sync tools expect one voice. Several voices confuse the timing and give glitchy mouths. Diarize first, feed only the target speaker's track.
🎬
Dubbing — swap one voice, keep the rest
Replace one person's voice in a scene while keeping the others: diarize, regenerate the target track only, mix it back over the original.
📝
Speaker-labelled captions
The .rttm and .json files say who speaks when: pair them with a transcription to get "Host: … Guest: …" captions.
🎧
Podcast post-production
Different EQ, compression and noise reduction per speaker — possible only once their voices are on separate tracks.
🔗
Voice datasets at scale
Point it at a folder of episodes: the models load once, finished files are skipped on the next run, and you get hours of per-speaker audio in one batch.
