portfolio-wispr-flow

Portfolio narrative for Wispr Flow, "local-first voice-to-text system for macOS with on-device LLM rewriting. No cloud, no latency." Status: Live.

Key extractions

• Thesis: cloud voice transcription has two flaws — latency (API round-trip kills flow) and privacy (sensitive utterances leaving the machine).
• Pipeline: audio capture → Whisper on Apple MLX → optional Dolphin3 rewrite via Ollama → paste.
• Key decision: MLX over PyTorch — 3-4x faster on Apple Silicon because it skips the CPU→GPU memory copy in unified memory architectures.
• Key decision: Ollama over custom model serving — abstraction cost minimal, model management free.
• Key decision: multi-threaded over async — Python asyncio doesn't give real-time guarantees for audio capture.
• Outcome: sub-300ms end-to-end latency on M1 MacBook. Fully offline.
• Hard problem: meeting 500ms budget required INT4 quantization of rewrite LLM, streaming transcription (don't wait for silence), careful thread sync.