Duplicate Webhooks and a Bad Deploy Replayed With the BareProxy Inbox

Two things go wrong with webhooks that have nothing to do with the app being down. Senders send the same event more than once, on purpose, because they can’t tell whether you got it. And sometimes the app is up and answering, but answering wrong, because the last deploy broke the handler.

This case runs both through the inbox, after a quick check of what it does with forged signatures. It used the released 0.16.0 binary for Linux, on 127.0.0.1. The webhooks follow the real GitHub, Shopify and Stripe formats, each signed with a test secret made for the run.

The setup

Three inboxes, each with its sender’s signature check. GitHub and Shopify put their own event ID in a header, so those two inboxes name it with key-header. Stripe puts its event ID in the body, so its inbox has none. Retries are short here, 2 seconds up to 10, giving up after 30, so the failures show up within a minute. The defaults are 10 seconds, an hour and three days.

inbox github
  deliver app
  signature github /work/github-secret
  key-header X-GitHub-Delivery
  retry 2s 10s
  give-up 30s

inbox shopify
  deliver app
  signature shopify /work/shopify-secret
  key-header X-Shopify-Webhook-Id
  retry 2s 10s
  give-up 30s

inbox stripe
  deliver app
  signature stripe /work/stripe-secret
  retry 2s 10s
  give-up 30s

bareproxy explain reads the route back in plain words before anything is sent:

Inbox github (line 11): stored in /work/state/inbox/github, answered 202 once on disk
  checked first: github signature in X-Hub-Signature-256 (401 when it doesn't match)
  delivered to pool app as POST /hooks/github, retried from 2s up to 10s apart, for 30s
Action: 202 (inbox)

Forged and stale signatures

Six webhooks. A good GitHub one and a good Stripe one got 202. A GitHub, a Shopify and a Stripe webhook signed with the wrong secret each got 401. So did a Stripe webhook with the right secret and a timestamp ten minutes old, which is how a captured request replayed later would look. Afterwards the inboxes held exactly the two good events. Nothing rejected was written to disk or reached the app.

The same event, sent again

GitHub’s redeliveries keep the same X-GitHub-Delivery GUID, so one push event was sent three times with it. All three got 202 with the same inbox ID, 18e39719dfb133d0b152ba, and the app got one delivery. Shopify’s retries keep X-Shopify-Webhook-Id. Two sends, one inbox ID, one delivery.

Stripe is different. A Stripe retry carries the same evt_ ID in the body, with a fresh Stripe-Signature header. The inbox doesn’t read bodies to find IDs, so the two sends became two events with two inbox IDs. The app got both, and since this test app keys its memory on the inbox ID, it processed the Stripe event twice.

The inbox did what its manual says here, and the gap is worth knowing about. For Stripe, the app should remember the evt_ IDs it has handled, as Stripe’s own docs advise. The inbox ID covers BareProxy’s repeats, and the evt_ ID covers Stripe’s.

A bad deploy

Then the app started answering 500 to every webhook, as an app with a broken handler would. Four webhooks came in, two GitHub, one Shopify and one Stripe. Each got 202, went to the app, came back 500, and was retried at about 2, 6, 14, 24 and 34 seconds after it arrived. That is 24 failed deliveries in all. After the sixth failure the inbox gave up on each one:

$ bareproxy inbox list --status failed
Inbox github (line 11), to pool app, in /work/state/inbox/github
  0 pending, 0 delivering, 2 delivered, 2 failed
  18e3971adb31000439bc7e  2026-10-31 10:40:18  failed   /hooks/github, 159 B, 6 attempts, last: 500
  18e3971ace9bedc8d52fc9  2026-10-31 10:40:18  failed   /hooks/github, 159 B, 6 attempts, last: 500
Inbox shopify (line 18), to pool app, in /work/state/inbox/shopify
  0 pending, 0 delivering, 1 delivered, 1 failed
  18e3971aea72596f6345af  2026-10-31 10:40:18  failed   /hooks/shopify, 111 B, 6 attempts, last: 500
Inbox stripe (line 25), to pool app, in /work/state/inbox/stripe
  0 pending, 0 delivering, 3 delivered, 1 failed
  18e3971af88fae7c2c1b32  2026-10-31 10:40:18  failed   /hooks/stripe, 162 B, 6 attempts, last: 500

One detail worth knowing. The give-up time was 30 seconds, and the last attempt came at about 34. The inbox checks the deadline when an attempt fails, so an event gets one more try if that try was already scheduled before the deadline. With the defaults, that means an event can be tried a little past three days, by up to an hour.

Failed events stay on disk. Once the fix was deployed, one command per inbox sent them back:

$ bareproxy inbox replay github --failed
Replayed 2 events; each delivery carries BareProxy-Inbox-Replay: operator.
$ bareproxy inbox replay shopify --failed
Replayed 1 event; each delivery carries BareProxy-Inbox-Replay: operator.
$ bareproxy inbox replay stripe --failed
Replayed 1 event; each delivery carries BareProxy-Inbox-Replay: operator.

All four reached the app within a quarter of a second, marked operator, and all four were processed. The senders had been told 202 at the start and never had to send anything again.

What it shows

The script and the full log are in the repository: inbox-retries-replay.sh and inbox-retries-replay.log. The use examples post has the replay recipe on its own, and the other inbox case studies cover an app deploy and a crash mid-delivery.