whisperx-gpu-queue

Run WhisperX as a queue worker that plays nicely with one shared GPU. Drop an audio or video file in a queue; the worker transcribes it when the GPU is free, optionally tidies the text with a local Ollama model, and writes the result. No cloud, no per-minute API, nothing leaves your network.

I built this because a single consumer GPU — in my case an RTX 4060 — has to serve more than one appetite: WhisperX, a local LLM, a nightly embedding job. Run two of those at once and you don't get speed, you get CUDA out of memory.

The idea

The whole trick is one line of UNIX that predates CUDA by decades: an advisory file lock (flock). Every GPU consumer on the host takes the same lock file before touching the card:

def acquire_gpu_lock():
    handle = GPU_LOCK_FILE.open("a+")
    fcntl.flock(handle.fileno(), fcntl.LOCK_EX)   # blocks until it's yours
    return handle

• WhisperX grabs the lock, runs flat out, releases it the instant the CUDA-bound step finishes.
• Ollama (or any other GPU job) uses the same lock, so heavy work serialises instead of colliding.
• It's advisory, which is the point: it only works because every job cooperates. That's a feature — no kernel module, no scheduler, no GPU partitioning. Fifty lines and a lock file.

The queue is a single SQLite table, so it survives restarts and you can add work from anything that can run one INSERT. One job runs at a time — deliberately serial, because there is one GPU and honesty is faster than thrashing.

Run it

# 1. edit compose.yaml: set <LAN_IP>, <MEDIA_DIR>, <SHARED_GPU_LOCK_DIR>
docker compose up -d --build

# 2. queue a file (any path the container can see under /media)
curl -s -X POST localhost:8090/jobs \
  -H 'content-type: application/json' \
  -d '{"source": "/media/interview.m4a", "language": "en"}'
# -> {"id": 1, "state": "queued"}

# 3. check on it
curl -s localhost:8090/jobs/1
# -> transcript.txt and transcript.clean.txt land under ./out/job-1/

Leave OLLAMA_URL empty to skip the cleanup pass and just get the raw WhisperX transcript.

Design notes

• Bind to the LAN, not 0.0.0.0. There's no auth here; it assumes a trusted home network. Put it behind a reverse proxy if you need more.
• The lock dir must be shared. Every GPU consumer mounts the same host directory at /gpu-lock. If Ollama doesn't take the lock, the collision it was meant to prevent comes right back.
• Failures are recorded, not fatal. A bad file marks its own job error and the worker moves on. The queue never wedges on one broken input.
• Restarts are safe. State lives in SQLite; an interrupted job is simply still queued (or re-runnable) after a restart.

Where this came from

A more elaborate version of this worker runs my homelab knowledge base — transcribing decades of audio into a private, searchable archive, with the same GPU-lock discipline and a stricter post-processing pass. That stack is at knowledge-mcp. This repo is the reusable core, stripped of anything private, for anyone who just wants WhisperX to share a GPU without falling over.

More write-ups: blog.riera.co.uk

License

MIT