Voice Cloning (Base Model)

Audio version — Estimated duration: 4 min 51 sec

Clone any voice from a short reference audio sample using the Base model (Qwen3-TTS-12Hz-0.6B-Base).

How it works

The Base model differs from the CustomVoice model in one key way: it doesn’t have built-in speaker embeddings. Instead, it accepts a voice cloning prompt derived from your reference audio:

Reference audio + transcript → create_voice_clone_prompt() → voice embedding
Text chunk + voice embedding → generate_voice_clone() → audio waveform

This prompt is pre-computed once and reused for every chunk, so the reference processing overhead is amortised across the entire generation.

Basic usage

python md_to_audio_base.py sample.md \
  --ref_audio "speaker_sample.wav" \
  --ref_text "Hello, this is a sample of my voice for cloning." \
  --output cloned_output.wav

Choosing a reference audio

Quality of the cloned voice depends almost entirely on the reference. Guidelines:

Factor	Good	Poor
Duration	5–15 seconds	< 3 seconds or > 60 seconds
Background noise	Clean, silent room	Fans, reverb, background music
Speaker consistency	Same person throughout	Multiple speakers
Pace	Natural, steady	Rushed, lots of pauses
Content	Phonetically diverse	Repetitive or minimal

The reference transcript must be word-exact. Even small mismatches (“it’s” vs “it is”) degrade quality noticeably. Transcribe by listening and typing, or use a tool like Whisper and manually correct the output.

Language matching

The --lang parameter must match the language of both the reference audio and the input Markdown:

python md_to_audio_base.py sample.md \
  --ref_audio "chinese_sample.wav" \
  --ref_text "中文参考音频的文本" \
  --lang "Chinese"

If the languages don’t match, the model may produce garbled or accented speech.

Chunk size and prosody

--chunk_size	Effect
100–150	Fastest per-chunk, but sentences may be cut mid-phrase, causing unnatural pauses
200 (default)	Good balance for most content
300–400	More natural prosody (sentences stay intact), but uses more VRAM per chunk

Chunk boundaries affect how the model handles punctuation and breathing. Smaller chunks are processed faster but may sound choppier because the model doesn’t see the full sentence context.

Batch size and VRAM

The Base script defaults to --batch_size 4, which batches 4 chunks into a single GPU call. This is a throughput optimisation but uses more VRAM.

VRAM	Recommended --batch_size
6 GB	1–2
8 GB	2–4
12 GB+	4–8

If a batch fails (OOM), the script automatically falls back to processing each chunk sequentially — slower but stable.

Full examples

Short technical article (English):

python md_to_audio_base.py article.md \
  --ref_audio "narrator.wav" \
  --ref_text "Welcome to today's tutorial on neural text to speech." \
  --chunk_size 250 \
  --batch_size 2 \
  --output narrated_article.wav

Multi-language input — process separate files with matching reference per language:

# English portion
python md_to_audio_base.py en_section.md --ref_audio "en_voice.wav" --ref_text "..." --lang English --output en_part.wav
# Chinese portion
python md_to_audio_base.py zh_section.md --ref_audio "zh_voice.wav" --ref_text "..." --lang Chinese --output zh_part.wav
# Concatenate with sox or ffmpeg

---

Next: CLI Reference · Troubleshooting