+91 98726 60544 hello@mitstech.co Mon–Sat · 09:00–18:30 IST

Calling third-party APIs without inheriting their outages

Cloud By Mits Engineering Team 2 min read
Calling third-party APIs without inheriting their outages

Every synchronous call to an external service is a dependency your availability is now multiplied by. Four services at 99.9% each, all required to serve a request, gives you 99.6% before your own code fails at anything. Most teams discover this arithmetic during an incident caused by a payment gateway or a mapping API rather than by anything they wrote.

The first defence is a timeout, on every single external call, set deliberately. Default timeouts in HTTP clients are frequently thirty seconds or none at all, which means a slow third party holds your threads until you run out and stop serving anyone - your own outage, caused by someone else's degradation. Set the timeout from their published latency, not from optimism: if the ninety-fifth percentile is 300 milliseconds, a two-second timeout is generous.

Retries need care, because the obvious implementation makes things worse. Retry only what is safe to repeat - a GET always, a POST only if the API supports an idempotency key. Use exponential backoff with jitter, because synchronised retries from every instance arrive as a thundering herd exactly when the dependency is trying to recover. And cap the total attempts, since a request retried five times with backoff has already taken longer than the user will wait.

A circuit breaker is what stops a struggling dependency from consuming your capacity. After a threshold of consecutive failures, stop calling it and fail immediately for a period, then let one request through to test recovery. This sounds like giving up and is the opposite: it means your threads stay available for everything else, and the failing service gets a chance to recover rather than being held down by your traffic.

Then decide, per dependency, what happens when it is unavailable - and write it down, because the default is an unhandled exception the user sees. Serve stale cached data, degrade to a reduced feature, queue the work for later, or fail explicitly with a clear message. A currency conversion API being down should not prevent checkout if you can use yesterday's rate. A payment gateway being down should stop checkout, clearly, rather than taking an order you cannot fulfil.

Finally, monitor third parties as if they were yours: latency and error rate per dependency, on a dashboard, alerting separately. The worst version of this incident is the one where an external service degrades gradually, your error rate climbs, and forty minutes go on investigating your own code before someone thinks to check their status page. Knowing within a minute which dependency changed behaviour is most of the resolution.

Need help with this? Explore our Cloud Solutions & Migration services. Learn more Back to all news

Keep reading

More on Cloud