Handling 429 Rate Limits in the OpenAI Node SDK Beyond maxRetries

Monday night the batch job that summarizes the week's repair tickets into a digest for the front desk started failing � not because the model was wrong, but because five agents now share one OpenAI tier and they all wanted the same 60 seconds. The error:

APIError: 429 Too many requests for tier. ...

If you searched "openai api 429 too many requests" you already know the annoying part: the OpenAI Node SDK retries on its own, the retry sometimes still isn't enough, and the header that tells you exactly how long to wait is hiding inside the error object where almost nobody looks. Here's where it lives, how much the built-in maxRetries already does, and a small wrapper that respects the server's number once the built-ins run out.

What the SDK already does for you

new OpenAI() ships with maxRetries: 2 � on 429s, 5xx, and timeouts it retries with exponential backoff and jitter. Before writing any custom code, tune the client:

import OpenAI from "openai";

const openai = new OpenAI({
  maxRetries: 4,     // built-in exponential backoff with jitter
  timeout: 60_000,   // and don't let one hung request eat a worker
});

That handles the casual case. It stops handling it when (a) retries exhaust on a busy night, or (b) you need your own policy � a queue-wide pause, a circuit breaker, or precise respect for the Retry-After the server is politely sending you. Then you catch the error, and the question becomes: where's the header?

Where Retry-After hides on the thrown error

When retries exhaust, the SDK throws an APIError. The response headers hang off it as .headers (a standard Headers object), so the wait time is:

import { APIError } from "openai";

try {
  await openai.chat.completions.create({ /* ... */ });
} catch (err) {
  if (err instanceof APIError && err.status === 429) {
    const retryAfter = Number(err.headers?.get("retry-after") ?? 0); // seconds
    const remaining  = err.headers?.get("x-ratelimit-remaining-requests");
    // ...
  }
}

The x-ratelimit-* siblings are worth logging (remaining-requests, remaining-tokens, reset-*) � they're how you see the cliff coming instead of driving off it. On older SDK versions these headers weren't attached to the thrown error at all; if .headers comes back undefined on your install, that's the upgrade signal, not a reason to parse response bodies. (If you're calling the API with plain fetch instead of the SDK, the same value is just response.headers.get("retry-after") � the retry-after header in Node's fetch world is always there, second verse of the same song.)

The wrapper: built-ins first, then respect the server's number

The useful pattern is a wrapper around the call, not a replacement for the SDK's retries � let the client do its thing, and only step in when the 429 survives it:

import OpenAI, { APIError } from "openai";

const openai = new OpenAI({ maxRetries: 2 });

export async function withRateLimit<T>(fn: () => Promise<T>, maxAttempts = 5): Promise<T> {
  for (let attempt = 1; ; attempt++) {
    try {
      return await fn();
    } catch (err) {
      if (!(err instanceof APIError) || err.status !== 429) throw err;

      // A billing wall is not a rate limit � retrying can't fix it.
      if ((err.error as any)?.code === "insufficient_quota") throw err;
      if (attempt >= maxAttempts) throw err;

      // Server said how long? Obey. Otherwise exponential backoff, capped.
      const retryAfterSec = Number(err.headers?.get("retry-after") ?? 0);
      const waitMs = retryAfterSec > 0
        ? retryAfterSec * 1000
        : Math.min(2 ** attempt * 500, 30_000);

      await new Promise((r) => setTimeout(r, waitMs + Math.random() * 250));
    }
  }
}

// usage:
const summary = await withRateLimit(() =>
  openai.chat.completions.create({ model: "gpt-4o-mini", /* ... */ })
);

Three details that matter in practice: the jitter (Math.random() * 250) keeps five retried workers from re-synchronizing into the same collision; the 30-second cap keeps a weird header value from sleeping your job until morning; and the exponential fallback (the classic 2^attempt curve for LLM APIs) covers the moments OpenAI doesn't send the header at all.

Not every 429 deserves a retry

Read err.error.code before looping � the table I keep taped to the wall:

Retrying an insufficient_quota is the most expensive no-op in cloud computing: it can't succeed, and it buries the real signal under identical errors.

Backoff alone isn't a throttle

One last thing the Monday-night incident taught me: retries react, they don't prevent. Five parallel agents each retrying with backoff still hammer the rate ceiling � politely, but repeatedly. When the fan-out is wide, put a concurrency limiter in front (a simple semaphore, p-limit style) sized to your requests-per-minute tier, and let the backoff wrapper handle only the tail. Prevention at the queue, reaction at the call � you need both, and neither alone.

One sentence to keep: the server already told you when to come back � err.headers.get("retry-after") � so tune maxRetries for the ordinary night, respect the header for the busy one, and never retry a quota wall.