TL;DR: Inbound email webhooks turn raw MIME payloads into structured events your agent can reason about. The critical path is: verify the webhook signature, parse headers and body, classify intent, then dispatch to the right action handler — all idempotently, keyed on Message-ID.
Inbound email webhooks for AI agents are HTTP POST callbacks that deliver a structured representation of an arriving email — headers, parsed body, attachments — to an endpoint your agent controls. Getting this pipeline right determines whether your agent acts reliably on real-world email or drowns in malformed MIME, duplicate triggers, and misclassified intent.
This post goes deep on the mechanics: what the webhook payload actually contains, how to parse it safely, how to classify intent without hallucinating, and which action patterns hold up in production.
What the webhook payload actually looks like
Most inbound email providers POST a JSON envelope containing MIME-parsed fields. You do not receive raw MIME unless you specifically request it. The typical structure looks like this:
{
"message_id": "<CA+abc123@mail.example.com>",
"from": { "address": "alice@example.com", "name": "Alice" },
"to": [{ "address": "agent+task42@yourdomain.ai", "name": "" }],
"reply_to": "alice@example.com",
"subject": "Re: Q3 report draft",
"text": "Looks good. Please send the final PDF.",
"html": "<div>Looks good...</div>",
"headers": {
"In-Reply-To": "<orig-msg-id@mail.example.com>",
"References": "<orig-msg-id@mail.example.com>",
"DKIM-Signature": "v=1; a=rsa-sha256; d=example.com; ..."
},
"attachments": [],
"timestamp": "2026-08-30T10:22:00Z",
"spf": "pass",
"dkim": "pass"
}
Four fields drive almost all downstream logic: message_id (your idempotency key), In-Reply-To / References (thread continuity), the tagged to address (routing signal), and text (the content you'll classify).
Threading via headers
The In-Reply-To header contains the Message-ID of the email being replied to. References is a space-separated list of all ancestor Message-IDs in the thread. If your agent sent the original email with a known Message-ID, matching In-Reply-To against your outbox table recovers full context — no need to store entire thread bodies, just the IDs.
SELECT task_id, agent_state
FROM sent_messages
WHERE message_id = $1 -- value from In-Reply-To header
This is how you avoid passing full email history to the LLM on every reply. Retrieve the task record, pass only what's needed.
Verifying webhook authenticity
Before touching any payload field, verify the request came from your provider. Skip this and anyone who discovers your endpoint can inject arbitrary email events into your agent.
Most providers use HMAC-SHA256 over the raw request body with a shared secret. The signature arrives in a header — commonly X-Webhook-Signature or X-Mail-Signature.
import { createHmac, timingSafeEqual } from 'crypto';
function verifyWebhookSignature(
rawBody: Buffer,
signatureHeader: string,
secret: string
): boolean {
const expected = createHmac('sha256', secret)
.update(rawBody)
.digest('hex');
const received = signatureHeader.replace('sha256=', '');
return timingSafeEqual(
Buffer.from(expected, 'hex'),
Buffer.from(received, 'hex')
);
}
Use timingSafeEqual — not ===. Timing attacks against HMAC verification are real, even over networks with variable latency, because an attacker can average many attempts.
Also verify that spf and dkim fields in the payload both read pass before trusting the from address for any privileged action. A message with dkim: fail may have a spoofed sender.
Idempotency: key on Message-ID
Webhook delivery is at-least-once. Your provider will retry on network errors or timeouts. If your endpoint takes 29 seconds to process a classification LLM call and the provider's timeout is 30 seconds, you will receive duplicates.
The fix is an idempotency table keyed on message_id:
CREATE TABLE processed_messages (
message_id TEXT PRIMARY KEY,
processed_at TIMESTAMPTZ DEFAULT now(),
result JSONB
);
At handler entry:
const { message_id } = payload;
const existing = await db.query(
'SELECT result FROM processed_messages WHERE message_id = $1',
[message_id]
);
if (existing.rows.length > 0) {
return res.status(200).json({ status: 'duplicate', skipped: true });
}
// ... process the message ...
await db.query(
'INSERT INTO processed_messages (message_id, result) VALUES ($1, $2) ON CONFLICT DO NOTHING',
[message_id, JSON.stringify(result)]
);
Return HTTP 200 on duplicates — not 4xx. If you return an error, the provider retries. ON CONFLICT DO NOTHING handles the race condition where two concurrent requests process the same message.
Classification pipeline
Classification answers: what does this email require the agent to do? The answer determines which handler runs.
A practical approach avoids sending the full email body to an LLM for simple cases. Build a tiered pipeline:
Tier 1: Rule-based (regex, subject keywords, tagged address)
→ catches ~60% of cases in under 1ms
Tier 2: Embedding similarity against intent templates
→ catches structured but variable messages in ~50ms
Tier 3: LLM classification with structured output
→ handles ambiguous, long-form, or multi-intent messages
Tier 1: tagged address routing
If your agent sends email from addresses like agent+taskid_42@yourdomain.ai, the to field on the reply tells you exactly which task context to load — no LLM needed.
function extractTaskId(toAddress: string): string | null {
const match = toAddress.match(/\+taskid_(\w+)@/);
return match ? match[1] : null;
}
This is the most reliable signal available and costs nothing to compute. Mails.ai's inbound parsing exposes the full to header including tagged suffixes, so this pattern works without any custom MIME parsing.
Tier 2: embedding similarity
Pre-embed a set of intent templates:
intents = [
("approve", "Yes, approved. Go ahead."),
("reject", "No, don't proceed. Cancel this."),
("request_info", "Can you send me more details about X?"),
("schedule", "Let's meet on Tuesday at 3pm."),
("escalate", "This needs a human to look at it."),
]
At classification time, embed the incoming text field and compute cosine similarity against each template vector. If the best match exceeds a threshold (0.85 works well empirically), route directly. Below threshold, fall to Tier 3.
Tier 3: LLM structured classification
For ambiguous cases, use a structured output schema so you get machine-readable intent without parsing freeform LLM responses:
const classificationSchema = z.object({
intent: z.enum([
'approve', 'reject', 'request_info',
'schedule', 'escalate', 'unsubscribe', 'other'
]),
confidence: z.number().min(0).max(1),
extracted_entities: z.object({
dates: z.array(z.string()).optional(),
people: z.array(z.string()).optional(),
action_items: z.array(z.string()).optional(),
}),
requires_human: z.boolean(),
});
Pass only the text field (not html) plus the subject. Stripping HTML before LLM classification avoids token waste and keeps CSS and tracking pixel noise out of the context window.
Action dispatch patterns
Once classified, you need deterministic action routing. An event-driven dispatcher is cleaner than a long if-else chain:
type ActionHandler = (payload: EmailPayload, entities: ExtractedEntities) => Promise<void>;
const handlers: Record<string, ActionHandler> = {
approve: handleApproval,
reject: handleRejection,
request_info: handleInfoRequest,
schedule: handleScheduling,
escalate: escalateToHuman,
unsubscribe: handleUnsubscribe,
};
async function dispatch(intent: string, payload: EmailPayload, entities: ExtractedEntities) {
const handler = handlers[intent] ?? handlers['escalate'];
await handler(payload, entities);
}
Default to escalate for unknown intents. An agent that silently drops unrecognized messages is harder to debug than one that surfaces unknowns to a human queue.
The approval pattern
Approval flows are the highest-stakes pattern — an agent acting on a spoofed or replayed approval has real consequences.
async function handleApproval(payload: EmailPayload, entities: ExtractedEntities) {
// 1. Verify sender is authorized for this task
const task = await getTaskByMessageId(payload.headers['In-Reply-To']);
if (!task) throw new Error('No matching task for In-Reply-To');
const authorized = task.authorized_approvers.includes(payload.from.address);
if (!authorized) {
await escalateToHuman(payload, entities);
return;
}
// 2. Check DKIM passed (already verified above, but assert here)
if (payload.dkim !== 'pass') {
await escalateToHuman(payload, entities);
return;
}
// 3. Execute the approved action
await executeApprovedAction(task.id);
// 4. Send confirmation reply
await sendReply({
to: payload.from.address,
inReplyTo: payload.message_id,
subject: `Re: ${payload.subject}`,
text: `Confirmed. Task ${task.id} has been executed.`,
});
}
Always verify the authorized approver list from your own records, not from the email itself. An attacker who can send email as a domain (SPF pass, DKIM fail) should not be able to approve actions.
The escalation pattern
Not everything should be automated. Your agent needs a reliable path to surface messages that require human judgment:
async function escalateToHuman(payload: EmailPayload, _entities: ExtractedEntities) {
await createHumanTask({
source_message_id: payload.message_id,
from: payload.from.address,
subject: payload.subject,
snippet: payload.text.slice(0, 500),
reason: 'agent_classification_uncertain',
priority: 'normal',
});
// Optional: auto-acknowledge to avoid sender anxiety
await sendReply({
to: payload.from.address,
inReplyTo: payload.message_id,
text: 'Received. A team member will follow up within 24 hours.',
});
}
Handling attachments
When attachments is non-empty, decide before calling an LLM whether the attachment content is relevant to the action. Fetching and decoding a 10MB PDF for an email that just says "approved" wastes tokens and latency.
if (intent !== 'request_info' && intent !== 'schedule') {
// Don't fetch attachments for approval/rejection flows
return;
}
for (const attachment of payload.attachments) {
if (attachment.size > 5_000_000) {
// Enqueue for async processing, don't block the webhook handler
await enqueueAttachmentProcessing(attachment.url, task.id);
continue;
}
const content = await fetchAttachment(attachment.url);
// Extract text, embed, store
}
Process large attachments asynchronously. Your webhook handler must return HTTP 200 within the provider's timeout window (typically 10-30 seconds). Offload anything heavier.
Full pipeline architecture
sequenceDiagram participant MX as MX Server participant Provider as Email Provider participant Hook as Webhook Handler participant DB as Idempotency DB participant Classify as Classifier participant Agent as Action Handler MX->>Provider: SMTP delivery Provider->>Hook: POST JSON payload Hook->>Hook: Verify HMAC signature Hook->>DB: Check message_id exists DB-->>Hook: Not found Hook->>Classify: text plus subject plus tagged address Classify-->>Hook: intent plus entities Hook->>Agent: dispatch to handler Agent-->>Hook: result Hook->>DB: Insert message_id plus result Hook-->>Provider: 200 OK
Observability and debugging
Webhook pipelines fail silently if you don't instrument them. Log three things at minimum:
- Every incoming payload hash (SHA256 of
message_idis enough — don't log PII in plaintext) - Classification tier used and confidence — tells you when Tier 3 is being called more than expected
- Handler execution result — success, error, escalated, skipped
Track the distribution of intents over time. A sudden spike in other or escalate usually means a new email format is hitting your agent that your classifier hasn't seen before.
For the inbound email processing side, structured logging makes it possible to replay a failed webhook — you have the full payload in your logs, so you can re-POST it to a test endpoint after fixing a classification bug.
Frequently Asked Questions
How do I avoid acting on auto-replies and out-of-office messages?
Check for Auto-Submitted: auto-replied or Auto-Submitted: auto-generated in the headers field. Also watch for X-Autoreply: yes and a Precedence: bulk or Precedence: auto_reply header. Filter these before classification — they should never trigger agent actions.
function isAutoReply(headers: Record<string, string>): boolean {
const autoSubmitted = headers['Auto-Submitted'] ?? '';
return autoSubmitted !== '' && autoSubmitted !== 'no';
}
What's the right timeout for my webhook endpoint?
Return HTTP 200 within 10 seconds, unconditionally. Acknowledge receipt immediately, then process asynchronously via a queue or background job. LLM calls can easily blow past that window — a job queue (BullMQ, Inngest, etc.) between the webhook handler and the processing logic is the right architecture.
How do I handle multi-part MIME emails where html and text differ?
Always prefer the text field for classification and LLM input. HTML is for rendering. The text part contains what the human actually wrote; HTML often contains quoted history, signatures, tracking pixels, and layout markup that pollutes your context. If text is empty — rare, but it happens with some mobile clients — strip HTML tags from html. A simple strip_tags pass, not full DOM parsing.
Can I use the tagged to address as the only routing signal?
For internal workflows where your agent controls both sending and receiving, yes — tagged addresses are reliable and fast. For external-facing inboxes where humans send to a fixed address like support@yourdomain.ai, you need classification on top. Use tagged addresses for known reply chains, classification for open inboxes.
How should I test my webhook handler locally?
Use ngrok or cloudflared tunnel to expose your local port, configure it as the webhook destination in your provider, and send test emails. Separately, capture real payloads (sanitized of PII) as JSON fixtures and build a test suite that runs classification and handler logic against them directly — no HTTP involved. This keeps regression testing fast and avoids provider rate limits during development.
What's the difference between from and reply_to for routing?
reply_to is where you should send your agent's response. from is who actually sent the message. For classification and authorization, use from (and verify DKIM against it). For sending replies, use reply_to if present, then fall back to from. Some automated senders set reply_to to a monitoring address — always check both.
Building this pipeline correctly means your agent handles email reliably at scale — not just in the happy path. The email classification and routing layer is where most production systems break down, and the patterns above address the common failure modes: duplicates, spoofed approvals, LLM misclassification, and silent drops. Get these mechanisms right and the rest of the agent workflow follows naturally.