Audio version — Estimated duration: 11 min 32 sec

Components

This page explains each piece of code in Whisper Dictate — what it does, how it works, and why it was built that way. It’s organized in the same order as the code file, so you can follow along as you read.

1. evdev Keyboard Monitor

What It Does

The keyboard monitor is the entry point for all user interaction. It discovers all physical keyboards connected to your system, watches for key presses and releases, and calls the appropriate handler when the hotkey is pressed.

How It Works

Device discovery happens in find_keyboards():

def find_keyboards():
    keyboards = []
    for path in evdev.list_devices():
        try:
            dev = InputDevice(path)
            caps = dev.capabilities()
            if ecodes.EV_KEY in caps and ecodes.KEY_A in caps[ecodes.EV_KEY]:
                keyboards.append(dev)
        except (PermissionError, OSError):
            pass
    return keyboards

It iterates through every device in /dev/input/. For each one, it checks that the device has EV_KEY capability and specifically supports KEY_A (the A key). This filters out mice, touchpads, game controllers, and media remotes — only real keyboards pass the check.

If no devices are found, the script exits with instructions to join the input group.

Event monitoring happens in monitor_device():

async def monitor_device(device):
    async for event in device.async_read_loop():
        if event.type == ecodes.EV_KEY and event.code == hotkey_scancode:
            if event.value == 1:
                on_key_down()
            elif event.value == 0:
                on_key_up()

This runs as a coroutine in the asyncio event loop. device.async_read_loop() uses Linux’s epoll (an efficient I/O event notification system) to wait for new events without busy-waiting or blocking.

Event values explained:

• value == 1 — Key pressed down
• value == 0 — Key released
• value == 2 — Key being held (repeat/typematic) — ignored by on_key_down()’s debounce

Why evdev Instead of…

• evdev over evdev.evtest: evtest is diagnostic tool, not for programmatic use
• evdev over keyboard libraries: Most keyboard libraries are for X11 and don’t work on Wayland
• evdev over getch/curses: Those only work when the terminal is focused — we need global hotkeys

Setting Up

# You must be in the input group
sudo usermod -aG input $USER
# Log out and back in
 
# Find your desired key code
python3 -m evdev.evtest

---

2. Hotkey State Machine

What It Does

The state machine tracks whether the hotkey is currently pressed and coordinates the transition between idle, recording, and transcribing states.

How It Works

Global state variables:

_recording    = False  # True while audio is being captured
_hotkey_held  = False  # True while hotkey is physically pressed
_audio_chunks = []     # Accumulates audio data during recording
_stream_idx   = 0      # Tracks which chunks have been stream-previewed
_lock         = threading.Lock()  # Protects shared state

on_key_down() — Hotkey pressed:

def on_key_down():
    global _recording, _audio_chunks, _hotkey_held, _stream_idx
    if _hotkey_held:
        return  # Ignore repeat events (value=2)
    _hotkey_held = True
    with _lock:
        _recording = True
        _audio_chunks = []   # Reset for fresh recording
        _stream_idx = 0
    print("[whisper-dictate] Recording...")
    notify("Recording", "Speak now...")
    threading.Thread(target=_record_loop, daemon=True).start()

Step by step:

1. The first check if _hotkey_held: return handles key-repeat events. The kernel sends value=2 periodically while a key is held, but once _hotkey_held is True, all subsequent events from the same hold are ignored.
2. Set _hotkey_held = True to lock out repeat events.
3. Under the lock, set _recording = True (tells the recording loop to run), clear the audio buffer (removes any previous recording), and reset the stream index.
4. Print status, show a desktop notification, and start the recording loop in a background thread.

on_key_up() — Hotkey released:

def on_key_up():
    global _recording, _hotkey_held
    if not _hotkey_held:
        return
    _hotkey_held = False
    with _lock:
        _recording = False
        chunks = list(_audio_chunks)  # Save a copy
    threading.Thread(target=_transcribe_and_type, args=(chunks,), daemon=True).start()

Step by step:

1. Check if not _hotkey_held: return — debounce for phantom release events.
2. Set _hotkey_held = False to allow the next press (step 1 of on_key_down() will now pass).
3. Under the lock, stop recording and take a snapshot of the audio buffer.
4. Start the transcription worker in a background thread with the snapshot.

Why the snapshot matters: If the user quickly presses the hotkey again, on_key_down() resets _audio_chunks. The snapshot protects the previous recording from being overwritten. See threading for the full race condition analysis.

---

3. Audio Recording (sounddevice)

What It Does

Opens your microphone and captures audio data continuously while the hotkey is held.

How It Works

The audio callback is the core of recording:

def _audio_callback(indata, frames, time_info, status):
    if _recording:
        _audio_chunks.append(indata.copy())

This function is called by sounddevice/PortAudio each time a new block of audio is available. The block size is determined by the operating system’s audio subsystem (typically ~512-1024 samples, which is ~32-64ms at 16kHz).

Key details:

• indata is a numpy array with shape (frames, 1) — one column for mono
• .copy() is called because indata is reused by the callback; without copying, old data would be overwritten
• The _recording check is lock-free — it’s only read here, and the GIL (Global Interpreter Lock) ensures the read is atomic

The recording loop manages the stream:

def _record_loop():
    with sd.InputStream(
        samplerate=SAMPLE_RATE, channels=1,
        dtype="float32", callback=_audio_callback
    ):
        last_stream = time.time()
        while _recording:
            now = time.time()
            if now - last_stream >= _stream_interval:
                last_stream = now
                threading.Thread(target=_stream_transcribe, daemon=True).start()
            time.sleep(0.05)

Step by step:

1. Open a sounddevice InputStream with 16kHz sample rate, mono channel, float32 format, and our callback.
2. Once the stream is open, enter a loop that runs while _recording is True.
3. Every second, check if 2 seconds have elapsed since the last stream preview. If so, spawn a preview thread.
4. Sleep for 50ms to avoid busy-waiting (100% CPU usage).

Why 50ms sleep: It’s a balance — short enough to respond quickly to the stop signal, long enough to keep CPU usage near zero. At 50ms, the loop runs 20 times per second, using less than 1% of a CPU core.

Why not use blocking mode instead of callback? Blocking mode would require a separate thread anyway (to avoid blocking the event loop). The callback approach is simpler and more responsive — PortAudio calls it in its own high-priority audio thread, so we never miss audio data.

---

4. Whisper Model (faster-whisper)

What It Does

Converts audio data (speech waveform) into text. This is the AI component that does the actual speech recognition.

How It Works

Model loading happens on startup:

def load_model():
    global _model
    _model = WhisperModel(
        MODEL_SIZE,
        device=DEVICE,
        compute_type=COMPUTE_TYPE
    )

Parameters explained:

• MODEL_SIZE = "medium" — The 769M parameter model, offering good accuracy with ~5GB VRAM usage
• DEVICE = "cuda" — Use NVIDIA GPU for acceleration (CPU option available for systems without CUDA)
• COMPUTE_TYPE = "int8_float16" — Mixed precision quantization that reduces VRAM usage while maintaining accuracy

What happens during model loading:

1. faster-whisper checks if the model is already downloaded in ./models/
2. If not, it downloads from Hugging Face Hub (default: Systran/faster-whisper-medium)
3. The downloaded files include: model.bin (weights), config.json, tokenizer.json, preprocessor_config.json, vocabulary.*
4. CTranslate2 loads and optimizes the model for GPU inference

Transcription happens in _transcribe_chunks():

def _transcribe_chunks(chunks, label="->"):
    if not chunks:
        return ""
    audio = np.concatenate(chunks, axis=0).flatten()
    if len(audio) < SAMPLE_RATE * MIN_DURATION:
        return ""
    if _model is None:
        return ""
    segments, _ = _model.transcribe(audio, beam_size=5, language=LANGUAGE)
    text = " ".join(s.text.strip() for s in segments).strip()
    if text:
        print(f"[whisper-dictate] {label} {text}")
    return text

Step by step:

1. Empty check — If there are no chunks, return empty string immediately. This happens if _stream_transcribe() runs right after recording starts (before any audio is captured).
2. Concatenate — All audio chunks are joined into a single 1D numpy array. After flattening, the shape is (N,) where N is total samples.
3. Duration check — If total samples is less than SAMPLE_RATE × MIN_DURATION (4800 samples = 0.3 seconds), skip. This prevents transcribing noise from accidental taps.
4. Model check — If the model hasn’t finished loading yet, skip. This happens if the user speaks before the model is ready.
5. Transcribe — The actual AI inference. beam_size=5 uses beam search (considers 5 candidate transcriptions at each step, picks the best). language=None enables auto-detection.
6. Format text — Join all segment texts with spaces and strip whitespace.
7. Print — Output with the appropriate label (>> for stream preview, -> for final transcription).

Whisper Internals (Simplified)

Whisper is a transformer-based encoder-decoder model:

1. Encoder — Takes the audio spectrogram (visual representation of sound frequencies over time) and converts it into a sequence of feature vectors that capture acoustic patterns.
2. Decoder — Takes the encoded features and generates text one token at a time, using the previously generated tokens as context for the next one.

The model was trained on 680,000 hours of multilingual audio, making it robust to accents, background noise, and various recording qualities.

Performance Tips

• Use GPU: DEVICE = "cuda" — a medium model transcribes in ~5 seconds on an RTX 3050
• Smaller model for speed: MODEL_SIZE = "small" transcribes in ~3 seconds but with slightly lower accuracy
• Set language: LANGUAGE = "en" skips auto-detection, saving ~0.5 seconds per transcription
• CPU fallback: Works, but expect 20-60 seconds for a medium model

---

5. Text Injection (ydotool / wl-copy)

What It Does

Takes the transcribed text and injects it into the currently focused application — either by simulating keystrokes or by copying to clipboard.

How It Works

def _type_text(text):
    try:
        subprocess.run(["ydotool", "type", text], check=True, timeout=30)
    except Exception as e:
        print(f"[whisper-dictate] ydotool failed: {e}")
        proc = subprocess.Popen(["wl-copy"], stdin=subprocess.PIPE)
        proc.communicate(text.encode())
        notify("Copied to clipboard", "Press Ctrl+V to paste")

ydotool path (primary):

• ydotool type "text" sends keystrokes via /dev/uinput
• Requires ydotoold daemon (script starts it on launch)
• Works on any Wayland compositor, X11, or even in the TTY
• The text is typed character-by-character into the focused window

wl-copy path (fallback):

• If ydotool fails (e.g., ydotoold not running, permission denied, timeout), the script falls back to wl-clipboard
• wl-copy text copies text to the Wayland clipboard
• User must manually paste with Ctrl+V
• A notification tells the user what happened

Why two paths? ydotool is the ideal solution (fully automatic), but it can fail due to permissions or missing dependencies. The fallback ensures the transcription is never lost — it always reaches the user somehow.

How the Guard Works (AltGr Prevention)

Before _transcribe_and_type() calls _type_text(), it checks the hotkey state:

if not _hotkey_held:
    _type_text(text)
    notify("Typed", short)
else:
    # Hotkey was pressed again — can't type while Alt is active
    proc = subprocess.Popen(["wl-copy"], stdin=subprocess.PIPE)
    proc.communicate(text.encode())
    notify("Copied to clipboard", "Press Ctrl+V to paste")

This check is critical because:

• If the user pressed the hotkey again while transcription was processing, Right Alt is physically held
• ydotool typing at that moment would send keystrokes with Alt modifier → AltGr combinations, wrong characters, or desktop shortcuts
• Copying to clipboard is safe regardless of modifier state

---

6. Notifications (notify-send)

What It Does

Provides visual feedback through desktop notifications, keeping you informed of the system’s state without looking at the terminal.

How It Works

def notify(title, body="", icon="audio-input-microphone"):
    if not SHOW_NOTIFY:
        return
    subprocess.Popen(
        ["notify-send", "-i", icon, "-t", "2500", title, body]
    )

Key design choices:

• Uses subprocess.Popen (not run) — fires and forgets, never blocks
• 2.5 second timeout — notification auto-dismisses
• Different icons for different states (microphone for recording, OK symbol for success, etc.)
• Can be completely disabled via SHOW_NOTIFY = False

Notification events:

Event	Title	Body	Icon
Model loading	”Whisper Dictate"	"Loading model, please wait…“	microphone
Ready	”Whisper Dictate"	"Ready! Hold the hotkey to speak.”	microphone
Recording	”Recording"	"Speak now…“	microphone
Transcribing	”Transcribing…”	(none)	microphone
Typed	”Typed”	First 72 chars of text	checkmark
Too short	”Nothing detected"	"Too short”	microphone
No speech	”Nothing detected"	"No speech found in clip”	microphone
Clipboard	”Copied to clipboard"	"Press Ctrl+V to paste”	microphone

---

Summary: How Everything Connects

You press Right Alt
  → evdev detects the event (kernel level)
  → on_key_down() starts recording (threading)
  → sounddevice captures audio (callback)
  → Every 2s: stream preview prints to terminal (no typing)
You release Right Alt
  → on_key_up() stops recording, takes snapshot
  → _transcribe_and_type() runs in its own thread
  → faster-whisper transcribes on GPU
  → ydotool types text (or wl-copy fallback)
  → notify-send shows result

Now see overview for the big picture, data-flow for the detailed walkthrough, or variables for settings you can adjust.