Handling email rate limits and backpressure looks trivial until your agent's inbox gets hit with 400 messages in 90 seconds and your LLM provider starts returning 429s. At that point, a naive webhook handler drops events, retries pile up, and you have no idea which emails were actually processed.
This post covers the full stack: webhook delivery mechanics, queue design, concurrency control, LLM rate limit propagation, and load-shedding strategies. Everything here applies to any inbound email webhook system — not just one provider.
Why inbound email creates unique backpressure problems
Inbound email webhooks differ from most event streams because the sender controls the arrival rate and your agent must absorb bursts without signaling back. HTTP has no native flow control — a 200 response tells the webhook emitter "I got it", not "I processed it". If your handler does actual work (parsing, LLM calls, database writes) synchronously before returning 200, every spike becomes a latency spike that eventually becomes a timeout.
AI agents make this worse in two specific ways:
LLM calls are slow and rate-limited. A GPT-4o call takes 2–8 seconds. If 50 emails arrive concurrently and each needs one LLM call, you're queuing up minutes of LLM time. OpenAI's tier-1 rate limit for GPT-4o is 500 RPM — roughly 8 per second — and that limit applies across your entire API key, not per agent.
Email threads need ordered processing. If two messages in the same thread arrive close together and are processed out of order, your agent may reply to the first message with context from the second, producing incoherent responses.
The fix is decoupling receipt from processing with an explicit queue layer.
Webhook acknowledgment: return 200 immediately
Return 200 before doing any real work. Your webhook endpoint should:
- Verify the request signature.
- Write the raw payload to a durable queue.
- Return 200.
That's it. Processing happens asynchronously. This matters because webhook providers have delivery timeouts — typically 10–30 seconds. If your handler exceeds the timeout, the provider marks the delivery as failed and retries. Retries under load are the main source of duplicate events.
import { mails } from "@mailsai/sdk";
import { Queue } from "your-queue-lib"; // Redis-backed, SQS, etc.
const inboundQueue = new Queue("inbound-email");
export async function webhookHandler(
rawBody: string,
signatureHeader: string
) {
const event = mails.webhooks.verify(
rawBody,
signatureHeader,
process.env.WEBHOOK_SECRET!
);
if (!event) {
return { status: 401 };
}
// Skip quarantined messages in your own processing logic
if (event.quarantined) {
await inboundQueue.enqueue({ ...event, _skip: true });
return { status: 200 };
}
await inboundQueue.enqueue(event);
return { status: 200 };
}
The mails.webhooks.verify call validates the X-Mails-Signature header. If verification fails, return 401 — never acknowledge an unauthenticated payload. See inbound email webhooks for how the signature is computed.
Queue design: durable, ordered, bounded
Your queue needs three properties.
Durable. Messages must survive process restarts. In-memory queues — a plain array, an EventEmitter — lose data on crash. Use Redis with AOF persistence, SQS, Cloud Tasks, or a Postgres-backed job queue (pgmq, Que, BullMQ with Redis).
Per-thread ordering. Use the email thread_id as the queue partition key. BullMQ calls this a "job group"; SQS FIFO calls it a MessageGroupId. Messages in the same thread process sequentially; messages across threads run in parallel.
Bounded concurrency. Set a max concurrency that reflects your LLM budget. If you have 500 RPM and each job makes 2 LLM calls, your max concurrency is 500 / (2 * 60) ≈ 4 workers running continuously. In practice, set it slightly higher to absorb latency variance — 6–8 workers is a reasonable starting point for a 500 RPM budget.
import { Worker } from "bullmq";
const worker = new Worker(
"inbound-email",
async (job) => {
const event = job.data;
if (event._skip) return; // quarantined, already logged
await processInboundEmail(event);
},
{
concurrency: 6,
connection: redisConnection,
}
);
LLM rate limit propagation
When your worker calls an LLM and gets a 429, pause that job and retry it — don't crash the worker or discard the event. Most LLM APIs return a Retry-After header telling you how long to wait.
async function callLLM(prompt: string): Promise<string> {
const maxRetries = 5;
let delay = 1000; // ms
for (let attempt = 0; attempt < maxRetries; attempt++) {
const res = await fetch("https://api.openai.com/v1/chat/completions", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
"Content-Type": "application/json",
},
body: JSON.stringify({ model: "gpt-4o", messages: [{ role: "user", content: prompt }] }),
});
if (res.status === 429) {
const retryAfter = parseInt(res.headers.get("Retry-After") ?? "1", 10);
await sleep(retryAfter * 1000);
delay = Math.min(delay * 2, 60_000); // exponential backoff cap 60s
continue;
}
if (!res.ok) throw new Error(`LLM error ${res.status}`);
const json = await res.json();
return json.choices[0].message.content;
}
throw new Error("LLM rate limit exceeded after retries");
}
Cap the backoff (60 seconds is reasonable), use jitter on the delay to avoid thundering-herd when many workers retry simultaneously, and treat 429 differently from 5xx errors. A 429 means "slow down". A 500 means "something broke".
Concurrency control with a token bucket
A fixed concurrency cap in your queue worker is a blunt instrument. For finer control — especially when you share an LLM key across multiple agents — implement a token bucket in Redis.
import { createClient } from "redis";
const redis = createClient();
async function acquireToken(
key: string,
ratePerSecond: number
): Promise<boolean> {
const script = `
local tokens = tonumber(redis.call('GET', KEYS[1]) or ARGV[1])
if tokens >= 1 then
redis.call('SET', KEYS[1], tokens - 1, 'EX', 10)
return 1
else
return 0
end
`;
const result = await redis.eval(script, { keys: [key], arguments: [String(ratePerSecond)] });
return result === 1;
}
This is a simplified token bucket — a production version refills tokens on a schedule using a Lua script that checks elapsed time since last refill. Libraries like rate-limiter-flexible wrap this correctly.
Rate limiting should happen before the LLM call, not after. If you can't get a token, re-enqueue the job with a delay rather than holding the worker thread.
Load shedding: when to drop vs delay
Sometimes you need to deliberately not process certain messages right away — or at all. Load shedding is the deliberate choice to discard or deprioritize work to protect system stability.
Common shedding strategies for email agents:
| Strategy | When to use | Implementation |
|---|---|---|
| Delay low-priority threads | Queue depth > 1,000 jobs | Tag jobs by urgency; use separate high/low queues |
| Deduplicate within window | Same sender emails 10x in 5 min | Fingerprint by from + subject normalized; skip duplicates |
| Skip classification on flood | Classification quota hit | Fall back to keyword matching; mark for reprocess later |
| Acknowledge but archive | Queue depth > hard limit | Write to cold storage; process in batch off-peak |
Mails.ai's email classification adds intent, entities, and urgency to inbound message.received events. The urgency field (a numeric score) gives you a natural priority signal — route high-urgency events to a fast queue, everything else to a standard queue.
Never silently drop events. Write every skipped event to a dead-letter log with enough context to replay it later.
Flow: from webhook to agent action
sequenceDiagram participant ESP as Email Provider participant WH as Webhook Endpoint participant Q as Durable Queue participant W as Worker Pool participant LLM as LLM API participant DB as Storage ESP->>WH: POST webhook payload WH->>WH: Verify signature WH->>Q: Enqueue event by thread_id WH-->>ESP: 200 OK W->>Q: Poll for next job Q-->>W: Event data W->>LLM: Prompt with email context LLM-->>W: Completion or 429 W->>DB: Write result W->>Q: Ack job complete
The critical invariant: the 200 OK to the ESP happens before LLM work starts. Failures downstream never cause webhook redelivery.
Handling webhook retries as duplicates
Even with a fast acknowledgment, webhook providers retry on network failures. Your queue must be idempotent — processing the same event twice must produce the same outcome.
Use the event id field as a deduplication key:
async function processInboundEmail(event: TypedEvent) {
const alreadyProcessed = await redis.get(`processed:${event.id}`);
if (alreadyProcessed) return;
// ... do work ...
await redis.set(`processed:${event.id}`, "1", { EX: 86400 }); // 24h TTL
}
Store the processed key with a TTL long enough to cover the provider's retry window. Most providers stop retrying within 24–72 hours. Webhook retry behavior and idempotency are covered in depth in the existing guide on retry logic and idempotency.
Monitoring queue health
Queue depth is the most important metric to watch. A growing queue means your workers can't keep up, and that growth is an early warning before any dropped events or timeouts.
Track:
- Queue depth per queue (high, standard, dead-letter)
- Job age — median time from enqueue to processing start
- Worker error rate — 429s vs 5xx vs app errors, separately
- Dead-letter queue size — events that exhausted retries
Alert on queue depth crossing a threshold (e.g., > 500 jobs waiting), not on job failures alone. By the time jobs start failing, the backlog is already large.
For agents built on Mails.ai's inbound infrastructure, the event injection_score and quarantined flag are also worth monitoring. A spike in quarantined messages may mean someone is probing your agent, which can artificially inflate your queue depth.
Frequently Asked Questions
Why not just increase worker concurrency to handle spikes?
Concurrency is bounded by your LLM rate limit, not by your server capacity. Spinning up 50 workers when you have a 500 RPM LLM budget doesn't increase throughput — it just means 50 workers compete for the same token budget and generate more 429s. Match concurrency to your actual LLM quota, then scale the quota if you need more throughput.
How should I handle the case where my queue fills up completely?
Set a hard queue depth limit and have your webhook handler respond differently when that limit is hit. Options: return 503 (which tells well-behaved providers to slow down and retry later), write directly to cold storage and bypass the queue, or drop and log. Never silently discard. A 503 from your webhook endpoint is a legitimate flow-control signal, not an error.
Does using a FIFO queue per thread prevent parallelism?
Only within a thread, which is the correct behavior — you don't want to process two messages in the same conversation simultaneously because reply ordering becomes unpredictable. Across different threads, jobs run fully in parallel up to your concurrency cap. In practice, most email workloads have many threads with few messages each, so effective parallelism is high.
What's the right retry strategy for failed jobs?
Use exponential backoff with jitter: start at 5 seconds, double on each retry, cap at 10 minutes, limit to 5–7 attempts. After exhausting retries, move the job to a dead-letter queue — never discard it. Keep enough context in the dead-letter entry (raw event payload, failure reason, attempt count) to replay manually or via a scheduled reprocessor.
Should I process reply.received events differently from message.received?
Yes, in two ways. Replies almost always belong to an existing thread, so the ordering guarantee matters more — make sure your queue partition key is thread_id. They're also often faster to process because they have more context (the existing thread) and less classification work. Consider a separate high-priority queue for replies if your volume warrants it.
How do I estimate the right worker count for my LLM budget?
Measure the average LLM calls per email and the average latency. If each email needs 2 LLM calls at 3 seconds each, that's 6 seconds of LLM time per email. At 500 RPM (8.3 calls/second), you can sustain 8.3 / 2 = 4.1 emails per second. Your worker count should be 4.1 * 6 ≈ 25 workers — but that assumes LLM calls are your only latency. Add headroom for DB writes, webhook roundtrips, and burst absorption, and start at around 30 workers with monitoring to tune from there.
Related guides: Playwright Email Testing: How to Verify Email Delivery in E2E Tests, CC and BCC in AI Agent Email Workflows, Email Circuit Breaker Pattern for AI Agent Pipelines and Detecting Auto-Replies to Prevent Agent Email Loops.
Explore the product: Inbound email API, Email classification, All features and Pricing.
