Task Queue Executions Climb Indefinitely With No Visible Errors
A misrouted task in a self-rescheduling handler forks an immortal execution chain per dial attempt. How to trace the loop and stop the runaway compute.
Field notes
One note per failure: the symptom the way you would search for it, the mechanism underneath, the checks that confirm it, and what I changed or would change. They come out of LiveKit Agents, WebRTC media, SIP and PSTN, the infrastructure below them, and the harness that proves a fix held.
Scheduling and capacity
A misrouted task in a self-rescheduling handler forks an immortal execution chain per dial attempt. How to trace the loop and stop the runaway compute.
Reduce document store transaction contention in a burst scheduler by probing capacity cheaply outside the transaction and deciding inside it.
How to prevent scheduled calls from dialing an hour early after a DST transition, violating calling windows, or crashing on non-existent local times.
When outbound calls exceed your SIP trunk channel limits, the cause is usually a read-then-write race condition in your scheduling logic.
SIP and PSTN
Voice agents over-report caller hangups when the session end reason is overwritten by a late SIP disconnect. Fix the race condition with a precedence rule.
Why a redeploy leaves you with duplicate SIP trunks, and why missing carrier IP allowlists cause inbound calls to fail silently before your code runs.
Fix voice agents cutting off their final sentence and leaking idle workers by awaiting audio playout and shielding room deletes from timeouts.
Outbound SIP placement requires strict ordering. Create the room and dispatch the LiveKit agent before dialing, and delete the room on failure.
How to route inbound SIP calls on a shared DID when caller numbers collide, using a deterministic ranking chain and immutable routing records.
Opaque RPC errors hide the underlying SIP response codes for failed outbound calls. Recover them from the metadata to build a working retry policy.
A standard existence check fails to prevent double-dials under queue retries. Learn how to use a dual-marker lock and sized in-flight window to protect calls.
How to diagnose and fix silent outbound dialing failures when SIP trunk IDs change after a restart, including idempotent boot-time provisioning.
Fix the missing first word on outbound AI phone calls by gating your greeting on audio track subscription, not the SIP participant active signal.
Echo-cancellation warm-up defaults erase the first word of a telephony answer. Learn how to tune SIP transport boundaries to prevent empty first turns.
Agent runtime
Why transcripts and end-of-call records vanish during voice agent shutdown, and how to fix data loss by budgeting teardown retries by wall clock.
When a voice agent joins a room but the provider closes the websocket immediately with code 1007, the plugin is sending a deprecated setup schema.
When a voice agent drops calls at exactly ten minutes, the provider is recycling the session. The agent replays illegal history on reconnect and dies.
How to fix a speech-to-speech voice agent that reads placeholder kickoff text as a user turn, causing bizarre greetings and reply request timeouts.
Why a speech-to-speech voice agent goes silent or truncates audio when calling a function tool, and how to enforce a turn-level contract in the prompt.
When a realtime voice agent reconnects successfully but stops responding, the conversational turn state is dead. Here is how to safely redrive the session.
A realtime voice agent hangs up mid-sentence or drops the caller’s final transcript. How to decouple the closing line from session teardown.
Evaluation
When evaluation pipelines misclassify perfect voice agent runs, the scorer is at fault. Fix brittle regex logic with a two-tier deterministic and LLM system.
A prompt tweak fixes one voice agent behavior but silently breaks another. Learn how pre-registered falsifiers catch prompt regressions before production.
Infrastructure
After a reboot or container recreate, outbound dialling fails silently. How to persist Redis state, configure systemd, and self-heal stale trunk identifiers.
A host reboot leaves your media server healthy and browser calls working, but phone calls silently fail until someone provisions SIP trunks by hand.