Request limits and retries
Check the selected model and the current platform limit before sending a request.
Text requests use the selected model's verified context window. Estimated input, including tool definitions and history, plus requested max_tokens must fit that window. Check context_length and max_output_tokens in the catalogue. A null value means the provider has not published an exact limit; existing platform safeguards still apply.
A TOKEN_LIMIT error includes a stable failure_code, the limit, estimated input tokens and requested maximum output. Token estimation is approximate; the upstream tokenizer determines actual usage. Vision and staging retain their own limits.
Current platform limits
- Request rate, counted per client IP address over one minute: 60 requests for
POST /v1/chat/completionsandPOST /v1/decisions, 10 forPOST /v1/images/generations, and 120 forGET /v1/models. - Concurrency: a workspace can have up to 4 requests in progress at once, across all of its keys.
- Spending: each request reserves its maximum possible cost against the workspace balance and the key's remaining spending cap.
- Upstream deadline: the gateway waits up to 90 seconds for the model service. Set your client timeout above that.
429 and 402 responses
429 RATE_LIMITED: the per-minute rate was exceeded. The response includes aRetry-Afterheader in seconds; wait at least that long.429 CONCURRENCY_LIMIT: four requests are already in progress. Wait for one to finish, and cap your worker pool at four.429 KEY_QUOTA: the reservation would exceed the key's remaining cap. Raise the cap by creating a new key, or lowermax_tokens.402 INSUFFICIENT_CREDITS: the available balance, net of other reservations, cannot cover the reservation. Add credits or lowermax_tokens.
These refusals happen before a request is recorded or reserved, so no model call was made. You can resend the same request, with the same Idempotency-Key, after waiting. When no Retry-After header is present, back off exponentially with random jitter and a maximum number of attempts, rather than retrying in a tight loop.
How max_tokens affects the reservation
When max_tokens is omitted, the chat endpoint uses 1024, or 4096 for a request that includes tools. The reservation covers the estimated input at the model's highest input rate plus max_tokens at its output rate, and is released down to the verified usage when the request settles. A large max_tokens can therefore trigger KEY_QUOTA or INSUFFICIENT_CREDITS even when the actual answer would be short. Request only the output you need.
Uncertain outcomes
If the API reports a rate limit, reduce concurrency and retry later. If a request has an uncertain outcome, inspect its status in your workspace before sending new work with a new idempotency key. Reusing a reserved key returns IDEMPOTENCY_CONFLICT without another upstream call or debit; it does not replay the previous response. A new key starts a separate, potentially billable request. The current response format and error handling are described in the API reference.
A 502 UPSTREAM_UNCERTAIN response, including a gateway timeout, means the credit reservation stays held for reconciliation. On chat completions it also reports a failure_code and a billing_status, normally reservation_held. Do not retry it automatically with a new key. The guide LLM API timeouts: reconcile usage before retrying explains the workflow.
Usage is prepaid. Review pricing and reservations before production traffic.