# Fallbacks & rate limits

> Fallbacks and rate limits are the product's plan for a model error, a slow response, or a rate-limit response.

What the product does when a model errors, slows down, or hits a rate limit.

Track: Ship & Maintain
Access: free
Card: https://buildsparkai.com/fallbacks
Plain text: https://buildsparkai.com/fallbacks.md
Last checked: 2026-10-18

## What it is

A rate limit caps requests or tokens in a period of time. OpenAI's rate-limit guide says limits include requests per minute and tokens per minute, and that you can hit either one first. A temporary limit returns HTTP 429. The response can include a Retry-After header, and the guide says to wait at least that long and to add a small random delay so clients do not retry together. The same guide says not to retry quota, billing, or other errors that need a person to act. A hard spend limit also returns 429, and retrying does not raise the limit. HTTP 503 with the code server_is_overloaded means the requested model is temporarily overloaded, and the guide says to follow Retry-After and then retry. Official SDKs already retry some 429 and 503 responses, so a second retry loop in your code can multiply calls. Your product still needs a plan for when retries stop: another model you have already evaled, a stored answer, or a plain error. The picture is that plan. It is not an OpenAI feature.

## Why it matters

**Cost.** Retries spend the limit you were already hitting. OpenAI says unsuccessful requests count toward the per-minute limit, so a tight loop prolongs the problem. A fallback model has its own price. This card does not add one. OpenAI also states a vendor-reported rule of thumb: once traffic reaches 1 million input tokens per minute, increase it by no more than 50 percent every 15 minutes. The exact point varies by model.

**Quality.** A fallback model that has not seen your eval set can answer differently. Switch models only after that set has graded the fallback.

**Risk.** Treating every 429 as wait-and-retry hides a spend limit or an empty credit balance. Read the error code before you choose retry, fallback, or stop.

## Try today

List the errors your app already catches. Mark each one retry, fallback, or stop. For a 429, read the body: a temporary rate limit can wait, and a spend limit cannot be retried into working.

## Common mistake

Treating every 429 as wait and try again. OpenAI uses 429 for a temporary rate limit and for spend and credit limits. Those need different actions.

## Self-check

What should the product do with a project spend-limit 429?

Stop retrying. Change the limit, or show an error

## Practice and deep

Coming soon. A short list of the errors your app catches, each marked retry, fallback, or stop.

Coming soon. How OpenAI separates a 429 rate limit, a 429 spend limit, and a 503 overloaded model, including Retry-After.

## Sources

- [OpenAI, Rate limits](https://developers.openai.com/api/docs/guides/rate-limits)
- [OpenAI, Error codes](https://developers.openai.com/api/docs/guides/error-codes)

Opened 2026-10-18. Vendor, framework, or standards docs. This card does not add a rate, a usage count, or a score of its own. Numbers quoted from a source are labelled vendor-reported or as that document's own words. The 1 million input tokens per minute and 50 percent ramp are OpenAI's stated rule of thumb on the rate-limit guide.