Rate limiting
Also called throttling, API rate limit.
Capping how many requests a user, API key or address can make in a given time, to protect a system and share capacity fairly.
Last updated
What it is.
A rate limiter counts requests per caller over a window, such as 100 per minute, and rejects or delays the rest. The standard HTTP response for this is 429 Too Many Requests, defined in RFC 6585, which may include a Retry-After header saying how long to wait.
Common methods include fixed windows, sliding windows and token buckets, which allow short bursts while holding an average rate. Limits are often enforced at the API gateway.
Why it matters
Without limits, one buggy script, a scraper or an attacker can slow the system for everyone or run up costs, especially when each request calls a paid API or an AI model. Limits also protect logins from password guessing and set clear rules for API customers.
In a product build
We set limits per user, per tenant and per IP, store counters in a fast shared store so all servers agree, return 429 with Retry-After, and make our own clients back off politely when they hit a third-party API's limits. Limits are tuned from real traffic, not guessed once.
Common mistakes.
- 01
Limiting only by IP, which blocks whole offices behind one address and misses distributed abuse.
- 02
Counters kept in each server's memory, so limits multiply with the number of servers.
- 03
No limits on expensive endpoints, such as AI calls, exports and search.
Read the official text.
Primary sources this definition is based on.
- RFC 6585, section 4: 429 Too Many Requests (IETF)checked 2026-10-25
Work it out with a tool.
Related terms.
- API gatewayA single entry point that sits in front of a system's APIs and handles routing, auth and limits for every request.
- IdempotencyThe property of an operation that has the same effect whether it runs once or several times, so it is safe to retry.
- Message queueA buffer that holds messages, such as tasks or events, until a worker is ready to process them, so the sender does not have to wait.
- Edge functionA small piece of server code that runs on a CDN's servers around the world, close to the user, instead of in one central region.
- Token (LLM)The unit an AI language model reads and writes, and the unit most LLM APIs bill by.
