Offering webhooks looks like the simplest integration feature there is: when something happens, POST to a URL the customer gave you. The complexity is entirely in what happens when that POST does not succeed, and since the receiving endpoints are built by other people and hosted somewhere you cannot see, it will not succeed a meaningful share of the time.
Deliver asynchronously and never from the request that caused the event. If saving an order also sends three webhooks synchronously, a customer's slow endpoint becomes your slow checkout. Write the event, queue the deliveries, and let workers make the calls with a short timeout - two or three seconds is generous, because you are not waiting for them to do work, only to acknowledge receipt.
Retry with exponential backoff over a long window - hours rather than minutes - because the most common failure is a customer's deployment or outage, and something that comes back after twenty minutes should still receive its events. Publish the schedule so integrators know what to expect. And stop eventually: after a day or so of failures, disable that endpoint, notify the customer, and let them re-enable it, rather than retrying forever into a URL that no longer exists.
Sign every payload, and document how to verify it. A receiver has no way to know a POST genuinely came from you rather than from anyone who guessed the URL. An HMAC signature over the body with a shared secret, in a header, along with a timestamp to prevent replay, is the standard approach and takes an hour to implement. Without it you are asking customers to trust unauthenticated input, and the good ones will refuse.
Design for at-least-once, because that is what you can actually guarantee. A receiver that acknowledged slowly may have processed an event you then retried. Include a unique event identifier so consumers can deduplicate, and say plainly in your documentation that duplicates occur and their handler must be idempotent. Promising exactly-once and failing quietly is considerably worse than being honest about it.
Finally, give customers visibility, because otherwise every delivery question becomes a support ticket you answer by reading logs. A page showing recent deliveries per endpoint, with status, response code and body, and a button to replay a specific event, resolves most integration problems without you being involved. It is a day of work that pays for itself in the first month of any serious integration.
