Architecture

Audio version — Estimated duration: 7 min 17 sec

High-level pipeline

Both scripts follow the same flow:

Markdown (.md)
    │
    ▼
clean_markdown()      ──  Strip links, formatting, HTML, non-speech chars
    │
    ▼
split_text()          ──  Split at sentence boundaries, fallback to commas
    │
    ▼
model.generate*()     ──  GPU inference, batched or sequential
    │
    ▼
numpy.concatenate()   ──  Join chunk waveforms
    │
    ▼
soundfile.write()     ──  24 kHz mono WAV

Stage 1 — Markdown cleaning

Both scripts share a base clean_markdown() function that strips anything that doesn’t represent speech:

Pattern	Replacement	Example
[text](url)	text	[click here](link) → click here
*, _, ~, `	empty	**bold** → bold
# characters	empty	## Heading → Heading
> blockquotes	empty	> quote → quote
<...> HTML tags	empty	<br> → “

The CustomVoice script applies an additional pass — non-ASCII stripping:

text.encode('ascii', 'ignore').decode('ascii')

This removes emoji (🎉), CJK characters (你好), mathematical symbols (∑), and any other non-ASCII glyphs. These characters crash certain tokenizer backends, so stripping them is a safety measure. If your input contains non-ASCII text intentionally (e.g., Chinese), use the Base script instead.

Stage 2 — Text splitting

split_text() divides cleaned text into chunks for GPU processing.

Algorithm:

1. Split text on sentence boundaries: re.split(r'(?<=[.!?])\s+', text)
2. Accumulate sentences into chunks up to max_chars
3. If a single sentence exceeds max_chars, split it at commas: re.split(r'(?<=,)\s+', sentence)
4. Any part still too long becomes its own chunk (no further splitting)

Default values:

Script	Default max_chars	Rationale
Base (md_to_audio_base.py)	200	Targets mid-range GPUs, batch size 4
CustomVoice (md_to_audio_custom.py)	150	Tighter for 6 GB VRAM, batch size 1

Why chunk at all? The Qwen3-TTS model generates audio tokens autoregressively — one at a time. The context window is finite (1024 tokens by default). Chunking ensures each segment fits within this window and keeps GPU memory usage predictable.

Stage 3 — Model inference

Model architecture

Qwen3-TTS is a decoder-only transformer with ~600 million parameters (0.6B). It operates at 12 Hz — each audio token represents ~83 milliseconds of speech. The model is causal (left-to-right), generating audio tokens conditioned on text tokens and optional speaker conditioning.

Base model flow

voice_clone_prompt = model.create_voice_clone_prompt(
    ref_audio="voice.wav",
    ref_text="transcript"
)
 
wavs, sr = model.generate_voice_clone(
    text=[chunk1, chunk2, ...],    # batch of text chunks
    language=["English", ...],
    voice_clone_prompt=voice_clone_prompt,
    max_new_tokens=1024
)

1. create_voice_clone_prompt() encodes the reference audio into a speaker embedding using a reference encoder
2. This embedding is passed to every generation call as conditioning
3. The model cross-attends to both text tokens and the voice embedding at each step

CustomVoice model flow

wavs, sr = model.generate_custom_voice(
    text=[chunk1, chunk2, ...],
    language=["English", ...],
    speaker="Ryan",
    instruct="Speak naturally.",
    max_new_tokens=estimated_budget,
    do_sample=True,
    top_k=50,
    top_p=0.95,
    temperature=0.8
)

1. Speaker name is mapped to a learned speaker embedding within the model
2. The instruction text is prepended to each chunk as a soft prompt
3. Generation uses nucleus sampling (top-k + top-p + temperature)
4. max_new_tokens is computed per-batch by estimate_max_tokens() (see below)

Attention backends

Backend	Speed	Availability
flash-attn	Fastest	Linux only (CUDA 11.8+)
sdpa	Fast	PyTorch built-in (2.0+), works everywhere
eager	Slowest	Pure PyTorch, no optimisations

flash-attn is an optional fused-attention library that provides ~1.5–2× speedup over sdpa by avoiding materialisation of the full attention matrix. It is not available on Windows, which is a primary reason inference is faster on Linux.

The Base script hardcodes attn_implementation="sdpa". The CustomVoice script tries sdpa first and falls back to eager if it fails — this handles environments where SDPA isn’t compatible (certain Windows setups, older PyTorch versions). On Windows native this can result in a 2–6× slowdown compared to Linux with flash-attn. For optimal performance on Windows, use WSL2.

GPU acceleration flags

# Both scripts
torch.backends.cuda.matmul.allow_tf32 = True    # 8x faster matrix multiplications on Ampere
torch.backends.cudnn.allow_tf32 = True           # 8x faster convolutions on Ampere
 
# CustomVoice only
torch.backends.cudnn.benchmark = True             # Auto-tune cuDNN kernels on first run
torch.cuda.empty_cache()                          # Clear VRAM before loading model

TF32 is a precision format on Ampere GPUs that delivers float32 range with ~float16 performance for matrix math. cudnn.benchmark slightly increases first-chunk latency (benchmarking runs) but improves all subsequent calls.

Adaptive token budget

CustomVoice only. The function estimate_max_tokens() replaces the fixed max_new_tokens=1024 with a per-chunk estimate:

def estimate_max_tokens(chunk: str) -> int:
    chars_per_token = 4.5
    audio_tokens_per_text_token = 12.0   # 12 Hz
    safety_margin = 1.3                  # 30% headroom
 
    text_tokens = len(chunk) / chars_per_token
    audio_tokens = text_tokens * audio_tokens_per_text_token * safety_margin
    return int(min(max(audio_tokens, 256), 1024))

Example budgets:

Chunk length	Text tokens	Estimated audio tokens	Clamped budget
50 chars	~11	~172	256
100 chars	~22	~343	343
200 chars	~44	~686	686
400 chars	~89	~1388	1024

The 1024-token ceiling corresponds to ~85 seconds of audio — more than enough for any reasonable chunk. The 256-token floor (~21 seconds) prevents excessively short generations that would sound cut off.

Performance impact

On the RTX 3050 6 GB used during development, this optimisation reduced generation time for short documents (100–500 chars) by 40–60% compared to always using 1024 tokens, because the model stopped generating earlier on short chunks.

Stage 4 — Concatenation and output

all_wavs = []          # list of 1D numpy arrays
for chunk in chunks:
    wav, sr = model.generate(...)
    all_wavs.append(wav[0])     # wav[0] because the model returns (batch, samples)
 
final_audio = np.concatenate(all_wavs)   # join end-to-end
sf.write(output_path, final_audio, sr)   # sr is always 24000

The sample rate is determined by the model (24 kHz) and is consistent across all chunks. The concatenation is a simple join — there’s no cross-fade or smoothing at chunk boundaries. If the model generates slight amplitude discontinuities, they may be audible as soft clicks; this is rare with the default chunk sizes.

---

See Voice Cloning and Preset Voices for usage.