Idempotency Keys for Webhook Consumers

An idempotency key is a stable, unique id your webhook consumer assigns to each event, usually the provider’s own event id, so that handling the same event twice produces the same result as handling it once. You record the keys you have already processed and skip any repeat, which turns a duplicate delivery into a safe no-op instead of a second charge, a second email, or a double-counted order.

We have run webhook infrastructure long enough to say this plainly: if your handler assumes every event arrives exactly once, it will eventually do the wrong thing. Duplicates are not a bug you can patch away. They are a property of how reliable delivery works. The fix is not to chase exactly-once delivery, which does not survive an HTTP boundary. It is to make your consumer idempotent so repeats stop mattering.

Why you get the same event twice

Duplicates come from the delivery system doing its job, not from failing at it.

The most common cause is a retry after a write you already committed. Your handler receives an event, updates the database, and then the response back to the sender gets lost, or your server takes too long to reply — the acknowledgement half of the exchange is where duplicates are born. From the sender’s point of view, the delivery failed. So it sends the event again. You already did the work; now you are being asked to do it a second time. That timeout-after-success window is exactly why retries make duplicates unavoidable, and no amount of tuning removes it.

Providers redeliver for their own reasons too. Stripe replays events during outages and lets you manually resend from the dashboard. GitHub lets you redeliver any webhook from the settings UI. A load balancer hiccup, a dropped connection, a deploy that restarts a pod mid-request: each one can produce a second copy of an event you have already seen.

Webhooker sits in front of your endpoint as an inbound gateway, and it guarantees at-least-once delivery to your consumer. That is a deliberate choice. We would rather hand you an event twice than risk dropping it once. The tradeoff is that your handler has to expect repeats. If you want the longer version of that argument, what a webhook gateway guarantees covers where at-least-once comes from and why nobody sells exactly-once honestly.

What an idempotency key actually is

The key is whatever value stays identical across every copy of the same logical event. Same event, same key, every time, no matter how many attempts it takes to reach you.

That last part matters. A good key identifies the event, not the delivery attempt. If Stripe sends evt_1P9x... three times, all three carry the same event id. That id is your key. What you must not key on is anything that changes per attempt: a timestamp at receipt, a random request id your framework generates, the retry count. Key on those and every retry looks brand new, which defeats the whole exercise.

Most of the time you get the key straight from the payload. Sometimes you build it from a few stable fields. Either way, the rule is the same: derive it from the event’s identity, never from the moment it happened to arrive.

Two things share this name, and they point opposite ways

Search for “idempotency key” and most of what you find describes something else entirely, so it is worth separating them before you go further.

The other one is the Idempotency-Key request header. There, you are the client calling somebody’s API. You generate a key, usually a UUIDv4, and send it with a POST so that a retry after a timeout does not create a second charge. The server stores your key with the response it produced, and replays that stored response if the same key shows up again. Stripe’s API works this way, Square’s does, and there is an IETF draft for the header, though MDN still lists it as non-standard.

This article is about the other direction. Nobody hands you a key; the event arrives with an id the provider assigned, and you decide whether you have seen it before. You are the server, not the client, and the key is discovered rather than generated.

Both exist for the same reason, which is that retries are unavoidable and a retry must not double the side effect. If your service both calls APIs and receives webhooks, you will end up implementing both, and confusing them is a good way to build a dedupe store keyed on a value that changes every time. The rest of this page covers the inbound half.

The dedupe store: check and act in one step

You need somewhere to remember which keys you have already handled. A small table works well:

CREATE TABLE processed_events (
    event_key   TEXT PRIMARY KEY,
    source      TEXT NOT NULL,
    processed_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

The primary key on event_key does the heavy lifting. It makes a repeat physically impossible to insert twice.

Now, the mistake almost everyone makes on the first try. It looks reasonable and it is broken:

# BROKEN: race condition between the check and the write
if not db.exists("processed_events", event_key):
    handle_event(event)          # charge, email, whatever
    db.insert("processed_events", event_key)

Under load, two copies of the same event can run this at the same time. Both check, both find nothing, both proceed, both process. You just did the work twice. The gap between reading and writing is where the duplicate slips through.

The fix is to let the database decide, atomically, whether this is the first time you have seen the key:

# Atomic claim. Only one caller wins the insert.
inserted = db.execute(
    "INSERT INTO processed_events (event_key, source) "
    "VALUES (%s, %s) ON CONFLICT (event_key) DO NOTHING "
    "RETURNING event_key",
    [event_key, source],
)

if not inserted:
    return 200          # already handled, this copy is a no-op

handle_event(event)

One statement claims the key. If the row already exists, ON CONFLICT DO NOTHING returns nothing and you stop. Only the caller that actually inserted the row goes on to process the event. There is no window between checking and acting because there is no separate check.

If you prefer Redis, SET key value NX EX 604800 gives you the same atomic claim with a built-in expiry. Just know the tradeoff: Redis without persistence can lose keys on restart, and a lost key means a duplicate gets reprocessed. For anything touching money, we keep the record of truth in Postgres.

One caveat on ordering. Claim the key first, then process. But if your processing can fail, you want the claim and the work to commit together, or you want the ability to release the key on failure. Otherwise a crash after the claim but before the work leaves you with a key marked done and an event that never ran. Wrapping both in one transaction is the clean version.

Idempotent writes make the no-op free

Deduplication in a side table is your first line. Your second line is making the actual writes idempotent, so even a duplicate that somehow slips past the store does no damage.

Three patterns cover most cases. Use a unique constraint on a natural business key, so a second insert of the same order simply fails instead of creating a twin:

ALTER TABLE orders ADD CONSTRAINT uq_orders_event
    UNIQUE (provider_event_id);

Use an upsert when you want the event to set a final state rather than accumulate:

INSERT INTO invoices (id, status, amount)
VALUES (%s, 'paid', %s)
ON CONFLICT (id) DO UPDATE SET status = 'paid';

And use a conditional update when only a specific transition is valid, so replaying a payment.succeeded on an already-paid invoice changes nothing:

UPDATE invoices SET status = 'paid'
WHERE id = %s AND status = 'pending';

The goal across all three is the same. Running the write once and running it five times land you in the same state. When your writes have that property, a duplicate is boring, which is exactly what you want.

Where the key comes from

For Stripe, use the event id on the event object, the evt_ value in id. It is stable across redeliveries and is the value Stripe itself recommends for deduplication. Do not use the object id inside data (like a ch_ charge id), because one object can generate several distinct events.

For GitHub, use the X-GitHub-Delivery header, a UUID GitHub attaches to each webhook. A manual redelivery from the GitHub UI reuses the same delivery id, which is precisely the behavior you want for dedupe — verifying GitHub webhooks covers that header alongside the signature check.

That same id earns its keep twice over. Beyond deduping honest retries, it is the second half of replay protection: a timestamp window rejects stale captured requests, and the delivery id catches anything resent inside the window. Why a valid signature is not enough works through that pairing.

For other providers, check their docs for a documented event id before you invent one. If there genuinely is not one, build a key from stable fields in the payload, for example a hash of resource_id plus event_type plus the provider’s own timestamp. Avoid anything you generate at receipt.

When you put Webhooker in front of your endpoints, every event carries a stable id and a full per-attempt history, so you always have something reliable to dedupe on even when the upstream provider is stingy with ids. That per-attempt trail also makes debugging a duplicate far less painful, because you can see every delivery that shared a key. You can get stable event ids to dedupe on in a few minutes, and here is at-least-once delivery with a full per-attempt history if you want the feature detail first.

Pitfalls we see most often

The non-atomic check-then-insert from earlier is number one, and it hides well because it passes every test you run at low volume. It only breaks under concurrency, which is to say in production during a traffic spike. Always claim the key in a single statement.

Second, keys that are not actually stable. Someone keys on a receipt timestamp or a per-request UUID, and every retry reads as a new event. Test this on purpose: replay the same event and confirm the second run does nothing.

Third, a TTL shorter than your retry window. If you expire keys after an hour but your gateway can retry for a day, a late retry arrives after the key is gone and gets reprocessed. Set key retention longer than the longest possible retry span, with margin. Cheap insurance.

Frequently asked questions