Every model has a context window; this 400 means prompt + requested output didn't fit. The naive fix — truncating the prompt blindly — is how RAG apps silently get dumber. Here's how to fit the window without losing the plot.
The error
// Anthropic:
{"type":"error","error":{"type":"invalid_request_error",
"message":"prompt is too long: 214481 tokens > 200000 maximum"}}
// OpenAI-compatible:
{"error":{"type":"invalid_request_error","code":"context_length_exceeded",
"message":"This model's maximum context length is 128000 tokens..."}}Causes and fixes at a glance
| Cause | Fix |
|---|---|
| Unbounded chat history | Keep a rolling window + a running summary; don't replay the whole transcript every turn. |
| RAG retrieving too many / too large chunks | Retrieve fewer, smaller, reranked chunks — quality beats volume well before the window fills. |
| max_tokens too large for what's left | The window covers input AND output: budget = window − input. Size max_tokens to the remainder. |
| Workload genuinely needs more context | Move to a long-context model tier (e.g. 1M-window models) instead of engineering around a hard fit. |
Measure before you trim
Count tokens the same way the API does (Anthropic has a count_tokens endpoint; tiktoken approximates OpenAI-wire models). Log the count per request — the fix differs for 1.05× over versus 5× over.
Trim retrieval and history, never the instructions
Cut in this order: (1) stale chat turns → summarize, (2) retrieved chunks → rerank and keep top-k, (3) verbose few-shot examples → fewer, tighter. System instructions go last — that's where behavior lives.
Escalate to a longer window when it's structural
If legitimate inputs exceed the window weekly, that's a model-selection problem. Long-context tiers (200K–1M) exist exactly for contract analysis, codebase Q&A and multi-document work — price the upgrade against the engineering time you're spending on trimming.
If you're calling through Kunavo
Kunavo's catalog spans context tiers — including 1M-window Claude 5 family models — behind one key, so escalating a workload from a 200K to a 1M window is a model-string change, and each model page states its window next to its per-token price for budgeting the tradeoff.
FAQ
Does max_tokens count against the context window?
Yes — the window bounds input plus generated output. A 200K-window model with 195K of prompt can't honor max_tokens: 8000. Compute the remainder and clamp max_tokens to it.
Is prompt caching a fix for context overflow?
No — caching cuts the COST of repeated long prompts, not their size. The token count against the window is identical whether cached or not.
More error semantics live in the error reference; getting a key takes a minute via sign up and the authentication docs.