Is every HTTP 429 a temporary rate limit?

No. OpenAI's error-code guide uses 429 for a temporary rate limit and also for spend limits and an exhausted credit balance. Read the error code.

Does OpenAI name a fallback model?

The rate-limit guide tells you when to retry and when to stop. A second model is your product's decision, and it should pass the same eval set.

Home · Shipping · 05 · Fallbacks & rate limits

Concept card · Shipping

Fallbacks & rate limits

What this gets you. Retry, switch or stop cleanly when a model errors or hits a rate limit.

Fallbacks and rate limits are the product's plan for a model error, a slow response, or a rate-limit response.

Last checked 2026-10-18 5 min

Search for Fallbacks & rate limits

What changed

Oct 18: First card. The picture separates a temporary rate limit, a spend limit, and an overloaded model. Sources are OpenAI's rate-limit guide and error-code guide.

1 What it is

A rate limit caps requests or tokens in a period of time. OpenAI's rate-limit guide says limits include requests per minute and tokens per minute, and that you can hit either one first. A temporary limit returns HTTP 429. The response can include a Retry-After header, and the guide says to wait at least that long and to add a small random delay so clients do not retry together. The same guide says not to retry quota, billing, or other errors that need a person to act. A hard spend limit also returns 429, and retrying does not raise the limit. HTTP 503 with the code server_is_overloaded means the requested model is temporarily overloaded, and the guide says to follow Retry-After and then retry. Official SDKs already retry some 429 and 503 responses, so a second retry loop in your code can multiply calls. Your product still needs a plan for when retries stop: another model you have already evaled, a stored answer, or a plain error. The picture is that plan. It is not an OpenAI feature.

Example

The next step is a product decision drawn for this picture. OpenAI's guides say which errors to retry. They do not name a fallback model.

The close-time request did not return a reply.

Response429. Temporary rate limit.

ProductWait at least Retry-After, add a small random delay, then retry the same model.

OpenAI says a temporary rate limit can be retried this way. Unsuccessful requests still count toward the per-minute limit.

2 Why it matters

Cost

Retries spend the limit you were already hitting. OpenAI says unsuccessful requests count toward the per-minute limit, so a tight loop prolongs the problem. A fallback model has its own price. OpenAI also states a vendor-reported rule of thumb: once traffic reaches 1 million input tokens per minute, increase it by no more than 50 percent every 15 minutes. The exact point varies by model.

Quality

A fallback model that has not seen your eval set (saved examples you already know how to grade) can answer differently. Switch models only after that set has graded the fallback.

Risk

Treating every 429 as wait-and-retry hides a spend limit or an empty credit balance. Read the error code before you choose retry, fallback, or stop.

3 How to apply · try today

List the errors your app already catches. Mark each one retry, fallback, or stop. For a 429, read the body: a temporary rate limit can wait, and a spend limit cannot be retried into working.

Common mistake

Treating every 429 as wait and try again. OpenAI uses 429 for a temporary rate limit and for spend and credit limits. Those need different actions.

Self-check

What should the product do with a project spend-limit 429?

Questions

What are fallbacks and rate limits?: Fallbacks and rate limits are the product's plan for a model error, a slow response, or a rate-limit response.
Is every HTTP 429 a temporary rate limit?: No. OpenAI's error-code guide uses 429 for a temporary rate limit and also for spend limits and an exhausted credit balance. Read the error code.
Does OpenAI name a fallback model?: The rate-limit guide tells you when to retry and when to stop. A second model is your product's decision, and it should pass the same eval set.