Scheduling and capacity
Dialer keeps retrying invalid phone numbers and dropping scheduled callbacks
Short answer
A single numeric retry counter cannot safely govern outbound call scheduling because failures are not interchangeable. A typed retry policy fixes this by inspecting the failure reason before scheduling the next attempt. Only unanswered rings and carrier placement errors earn a retry. Invalid numbers are skipped permanently. Capacity rejections bypass the recipient’s retry budget entirely. To schedule a valid retry, the system searches forward through legal calling windows until it finds a slot. If the reason for the call expires before a slot is found, the system abandons the attempt and logs a skip reason from a closed enumeration.
Symptoms
* A dead or invalid phone number gets dialed repeatedly every hour, accumulating carrier spend until an engineer notices the invoice and manually intervenes. * A customer asks why a specific person was never called back. You search the database, and nothing in the record can answer it because the decision not to retry was never written down. * Calls blocked by admission control disappear from the dialer schedule. The system treats the capacity wall as a call failure, consuming the recipient’s attempt budget before any connection is attempted.
What is actually happening
Retry logic written as a single numeric attempt counter cannot express the two facts that matter for outbound voice scheduling. First, failures are not interchangeable. Second, a retry has to land somewhere safe.
When a standard task queue manages outgoing calls, it treats all unhandled exceptions as generic failures. An unanswered ring means try later. An invalid number means never. An internal capacity rejection means your own system was busy, which has nothing to do with the callee. A generic retry loop treats all three identically. It increments the attempt counter and schedules a blind retry.
A retry also needs a valid landing slot. "Somewhere" is bounded by a strict calling window, by the recipient’s tolerance, and by whether the reason for the call is still valid when the slot actually arrives. When a generic queue worker fails to find a slot, it drops the message. Because queues are designed to delete permanently failed messages, the absence of a retry leaves no trace.
How to confirm it
- Query your call records for a known invalid number that your system recently dialed. Expect to see multiple outbound attempts with permanent failure responses, confirming that your dialer retries carrier rejections blindly.
- Search your application logs for the identifier of a dropped recipient who was never dialed. Expect to find a generic task exhaustion event or a dead-letter queue entry rather than a typed domain reason for the missing call.
- Inspect the retry budget for a call that hit an admission control limit. Expect the local retry counter to have incremented even though the carrier never received a request. This confirms the system is blaming the recipient for your own capacity walls.
How it works
I type the failure first, then branch. Only two classes earn a retry: an unanswered ring and a placement failure at the carrier. Both are capped at two attempts in a rolling seven-day window. This strict cap ensures a stubborn number cannot accumulate cost indefinitely.
An admission rejection is not a failed call and does not consume an attempt. It gets its own seven-day cooldown and its own typed skip reason. Retrying into a capacity wall on the next tick just moves the rejection, so separating capacity limits from call failures keeps the scheduler stable. This separation handles capacity limits at the application layer before reaching the carrier, coordinating with admission control inside a transaction (/blog/enforcing-maximum-concurrent-sip-calls-without-race-conditions/).
The retry’s landing slot is found by searching forward through legal calling windows. This forward search is bounded by the validity window of the reason for the call. If the forward search cannot find a slot before that reason expires, the retry is abandoned deliberately rather than scheduled into a slot where the call would be pointless.
Every path that declines to retry logs a skip reason drawn from a closed enumeration. I record this skip reason on every non-retry. The question "why was this person not called back" becomes a direct database lookup rather than an investigation into missing queue tasks.
What I would check first
- Check whether an unmade call failed admission control or carrier placement. An admission rejection triggers its own seven-day cooldown without consuming one of the recipient’s two rolling-window retry attempts. If the attempt counter remained at zero, the dispatch was deferred by local capacity rather than rejected by the carrier.
- Distinguish permanent carrier rejections from unanswered rings. Check whether the carrier reported an invalid destination or a ringing timeout. Only unanswered rings and carrier placement failures earn a retry; invalid numbers must terminate immediately.
- Check whether a missing retry was abandoned due to calling-window expiry. Inspect the logged skip reason to determine whether the call’s validity window expired before a legal slot opened. An expired validity window indicates an intentional cutoff, whereas an exhausted retry budget indicates repeated ring or placement failures.
Bounding the forward search by calling window expiry
Finding a safe landing slot requires more than adding a fixed delay to the current timestamp. The search algorithm iterates forward through the legal calling windows for the specific recipient. This must account for local time zones and the daylight saving boundaries that affect them, which often cause scheduled outbound calls to fire an hour early after daylight saving transitions (/blog/scheduled-outbound-calls-fire-an-hour-early-after-daylight-saving-transitions/).
An infinite forward search is dangerous because the business context has a finite lifespan. If the reason for the call has a finite validity window, placing the call after that window closes makes the conversation pointless. The reason for the call has a strict validity window.
The scheduler evaluates forward slots against this validity window. If the first available legal calling window opens after the reason has expired, the system abandons the retry attempt. This decision is explicit. The application commits a skip reason to the database indicating that the reason expired before a legal window opened. The recipient avoids a confusing phone call, and support staff can see exactly why the attempt was skipped.
Separating admission limits from recipient behavior
A common scheduling trap is treating an internal failure exactly like a network failure. If the application layer blocks a call because the concurrent session pool is full, the call never reached the carrier network boundary.
In a generic task queue, an admission control exception bubbles up and triggers the exact same retry backoff as a disconnected number. The task reschedules. If the capacity pool is still full when the task wakes up, it fails again. A few failures later, the queue permanently discards the task. The recipient was never dialed, but their database record looks like a persistent failure.
I track attempts consumed per recipient separately from admission rejections. The two configured retry attempts apply strictly to network-level placement failures and unanswered rings. When a call hits an admission rejection, it triggers a distinct seven-day cooldown.
Enforcing a closed enumeration for skip reasons
Free text logs cannot answer customer support queries at scale. If every dropped call generates a string interpolated with queue variables, you cannot build a reliable index to see why calls are failing across the platform.
I enforce a closed enumeration for every non-retry outcome. The instrumentation requires every terminal path to commit one of these predefined reasons to the call record. Examples include an invalid destination number, maximum attempts reached, reason expired before window opened, and permanent carrier rejection.
Common questions
When should you retry an outbound failed call?
Only retry unanswered rings and carrier placement failures. Invalid numbers indicate a permanent failure and should be skipped. Internal capacity rejections should not consume the recipient’s retry budget.
Why does my dialer keep retrying invalid phone numbers?
The scheduling queue treats all unhandled exceptions as generic failures. A single numeric attempt counter blindly retries permanent carrier rejections. You must type the failure before branching to a retry.
How do you handle outbound calling window expiry?
Search forward through legal calling windows to find a safe landing slot. If the reason for the call expires before the next valid slot opens, abandon the retry deliberately. Log a specific skip reason to capture the decision.
Why did my scheduled call hit a capacity limit and never retry?
A generic task queue treats an internal admission control rejection just like a network failure. The task reschedules, hits the same capacity wall repeatedly, and exhausts its attempt limit. Capacity rejections must have a distinct cooldown.