Rate limiting exists to stop one caller consuming capacity everyone else needs - whether that caller is malicious, a runaway script, or an enthusiastic integration partner who wrote a loop without a delay. The difficulty is that the limit has to be low enough to protect you and high enough that legitimate heavy users never notice, and most teams set it by picking a round number.
Start by measuring instead. Look at your actual traffic distribution per client over a month: the median, the ninety-fifth percentile, the maximum. Set the initial limit somewhere above the ninety-ninth percentile of legitimate use, and you will catch abuse while never touching a real customer. Guessing produces the two failure modes people complain about - a limit so generous it prevents nothing, or a support ticket from your largest account.
The algorithm matters less than people think, with one exception. A fixed window - a thousand requests per hour, reset on the hour - allows a caller to send a thousand at 10:59 and another thousand at 11:00, two thousand in two minutes, which is exactly the burst you were trying to prevent. A sliding window or token bucket avoids that, and a token bucket has the additional property of allowing short legitimate bursts while capping the sustained rate, which matches how real clients behave.
What you limit by is the decision that actually determines effectiveness. Per API key is right for authenticated APIs and is what you want commercially. Per IP address is the only option for unauthenticated endpoints and is weak - shared office networks and mobile carrier NAT put thousands of innocent users behind one address, while a determined attacker rotates addresses cheaply. For login endpoints, limit by account as well as by IP, or an attacker spreads a credential-stuffing run across many addresses and never trips the per-IP limit.
Tell the client what is happening. Return 429 with a Retry-After header, and publish the limit, remaining quota and reset time as headers on every response. A well-written client backs off automatically when given that information; a client told only that something failed will retry immediately and make the problem worse. This one detail converts your rate limiter from a wall into a protocol.
Two refinements worth adding once the basics work. Different limits for different endpoints, because a login attempt, a search and a report generation cost you wildly different amounts, and a single global limit is calibrated for none of them. And a rate limiter that fails open rather than closed - if the store holding the counters is unavailable, letting traffic through unlimited is usually better than rejecting everything, though that is a judgement to make deliberately rather than discover during an outage.
