Idempotent Payment Notifications: Safe Retries

A payment provider sends a notification, your server records the payment, and the connection closes before the provider receives your response. The provider retries. If the handler assumes every request represents a new payment, one successful payment can trigger two fulfilments.

Idempotency means that repeating the same operation does not repeat its intended business effect. It is a property of the whole state transition, not a synonym for putting an event ID in a cache.

Payment integrations are part of the UtopiaPay project. This article is a general design guide for that class of integration, not a claim that the illustrative schema below was used in that product.

Start with the delivery contract

Before implementing the handler, establish what the provider actually guarantees. Some systems retry events, some allow manual replay, and some deliver different events about the same payment out of order.

Identify the authenticated merchant account, the provider’s event identity and the payment identity separately. An event ID answers “have I processed this notification?” A payment ID answers “which business record does it concern?” They are not interchangeable.

Do not hash the entire request body as a default deduplication key. The same logical event may arrive with different serialization or delivery metadata. Use the provider’s documented identifier, scoped to the appropriate provider and account. If no stable identifier exists, define and validate a business-operation key from the actual delivery contract.

Validate before changing state

Authenticate the notification using the provider’s documented mechanism. When signatures cover raw bytes, parsing and reserializing the body before signature verification can invalidate that check. Apply the documented timestamp and replay protections as well.

Then validate that the event belongs to the expected account and that payment reference, amount and currency agree with your own records. An authenticated event is not automatically a valid instruction to fulfil an arbitrary order.

Do not trust a browser redirect to the success page as payment confirmation. It is navigation, not authenticated evidence of settlement.

Put deduplication inside the transaction

A minimal conceptual schema could separate events from payment state:

processed_events
  provider
  merchant_account_id
  event_id
  processed_at
  UNIQUE(provider, merchant_account_id, event_id)

payments
  id
  provider_payment_id
  merchant_account_id
  amount_minor
  currency
  status

This is a model, not runnable SQL. Column types, constraints and transaction syntax depend on the selected database. Amounts need an exact representation and the correct currency exponent, not binary floating-point arithmetic or an assumption that every currency uses two decimal places.

For a synchronous database update, the transaction boundary is:

authenticate and validate the notification
begin transaction
  insert the unique event identity
  lock the relevant payment or use a conditional state update
  validate the payment's allowed state transition
  apply the business change
  write any required outbound work to an outbox
commit transaction
acknowledge according to the provider's delivery contract

If the transaction fails, the event record must roll back too. Otherwise a retry can find “processed” even though the payment change never happened.

The unique constraint, not a separate SELECT before INSERT, arbitrates concurrent deliveries. Handle a uniqueness conflict using the database’s transaction semantics: some databases require a rollback before further statements. A committed event record may justify acknowledging a duplicate because its associated business update committed in the same transaction.

For a slower integration, durable acceptance into an inbox followed by background processing is another valid design. In that case, distinguish received from processed, and give workers a retry and recovery mechanism. Do not label an unprocessed inbox entry as completed.

Different events can still describe the same business effect

Deduplicating event IDs does not stop two distinct notifications from attempting the same fulfilment. Enforce the business invariant as well, for example one fulfilment record per order, and transition a payment only from permitted predecessor states.

Do not implement payment status as “the last event wins.” A delayed pending event must not overwrite a confirmed payment. Refunds, disputes and partial captures also mean that a single ordered list of statuses may be insufficient. Model the provider’s lifecycle explicitly.

External effects need another boundary

A database transaction cannot atomically commit an email, a third-party shipping request and a row update across unrelated services.

An outbox stores the intent to perform an external action in the same transaction as the payment change. A worker delivers that intent later. This closes the gap between updating a payment and remembering that fulfilment is required, but it does not eliminate duplicates by itself.

The worker can crash after the remote service succeeds and before marking the outbox entry delivered. Use a stable idempotency key at the downstream service when supported. If it is not supported, define reconciliation and manual recovery instead of promising exactly-once external execution.

Tests that expose real failures

Scenario	Expected invariant
The same event arrives twice	One committed business change.
Two workers process the same event concurrently	The database constraint arbitrates the duplicate.
Processing fails before commit	A retry can still apply the change.
The response is lost after commit	The retry is acknowledged without repeating the change.
Two distinct events refer to the same successful payment	The order is not fulfilled twice.
An older pending event follows success	Confirmed state does not regress.
The signature, merchant, amount or currency is wrong	No unauthorized payment transition.
The outbox worker retries after a remote success	Downstream deduplication or reconciliation prevents an uncontrolled repeat.

Keep event identifiers and outcomes in operational logs, but avoid logging payment secrets or unnecessary personal data. Monitor failed processing, old inbox/outbox records and reconciliation discrepancies.

Idempotency is ultimately a promise about observable behavior. Write down that promise, place it at a durable boundary and test what happens when the acknowledgement disappears.