Blog · AI ·
Claude API Integration: The Sharp Edges We Hit So You Don’t Have To
Production lessons from the Claude API: sampling params that 400 on upgrade, thinking tokens eating streams, disconnect billing, caching. What actually bit us.
Integrating the Claude API properly is mostly not prompt engineering. It's streaming behaviour, sampling-parameter changes between model generations, cost control, caching, and deciding what to charge users when a stream dies. We run this stuff daily across our own products, mainly ScriptGrain, and here's what actually bit us in production.
The upgrade that 400'd: sampling params are not forever
We migrated ScriptGrain from Claude Sonnet 4.5 to Sonnet 5 expecting the usual: better outputs, same shape. Instead, Sonnet 5 started returning 400 errors on any call with a non-default temperature, top_p, or top_k set. Any of them. Our voice anchoring in ScriptGrain relied on top_p, so this wasn't a minor call site, it was the whole voice-matching pipeline going dark.
The fix, once we found it, was almost embarrassingly simple. We moved voice control entirely into the system prompt and stopped sending sampling params at all. No top_p, no temperature override, nothing. The model behaved fine once we stopped telling it how to behave in a way it no longer accepted.
The lesson isn't "don't use sampling params". It's that whatever assumptions you baked in against one model generation are not guaranteed to survive the next one. Pin those assumptions. Write down which model version each call site was tested against, and when you upgrade, test every single call site again, not just the ones you remember using sampling controls. We didn't do that the first time round. We do now.
Adaptive thinking will eat your stream
Sonnet 5 ships with reasoning switched on by default. Nobody warns you about what that actually does to two very different call shapes.
On non-streaming calls, the thinking tokens come out of your max_tokens budget silently. You set a limit expecting it to cover your output, and instead a chunk of it gets eaten by reasoning you never asked to see, and never see. The practical result is truncated JSON. Your parser fails, you assume the model messed up the structure, and actually the model just ran out of room because thinking took the tokens first.
On streaming calls it's worse in a quieter way. The model reasons before it emits any visible text. If your SSE proxy just forwards whatever the model sends, moment by moment, there's a window where nothing comes through at all. Edge infrastructure has timeouts for exactly this situation, connections that go quiet get cut. So the connection dies before the first real byte arrives, and from the user's side it just looks like an empty result. No error, no explanation, just nothing.
We disable thinking explicitly on the paths where this matters. If you want to keep it on for a given call, you need to be emitting keepalives during that dead window, or your infrastructure will do the cutting for you. Either way, this is not something you can leave on defaults and hope for the best.
Streaming to browsers without lying to them
Anthropic's SSE format and what a browser parser expects are not the same shape. We translate Anthropic's stream into an OpenAI-compatible envelope on the fly, server-side, so the client-side parsing stays stable regardless of which model or provider sits behind it. That's the boring, correct way to do it.
The interesting failures come from the plumbing around that translation. Buffered, invoke-style calls can't expose a stream at all, so if a code path expects streaming and gets a buffered response instead, it just breaks in ways that are hard to diagnose from the client. Raw fetch paths need a hardcoded fallback URL or a properly resolved helper function, because we've had an env var go missing at build time and send an entire stream quietly to the SPA's index.html. With a 200 status. Which, from the user's perspective, reads as "empty draft", not "infrastructure error". No error page, no red text, just nothing where content should be.
None of this is glamorous work. It's the kind of thing that never makes it into a demo, because demos don't have flaky env vars or edge timeouts. But it's most of what "integrating the Claude API" actually means once you're past the first working prototype.
Only charge for what you delivered
If a user hits Cancel halfway through a stream, or their connection drops, that is not the same event as the model erroring out upstream. Treating them the same is how you end up charging someone for a stream they never received.
Our streams report client disconnects as a distinct category from upstream errors. Undelivered output isn't persisted and isn't charged, full stop. And this matters more than it sounds: a retry wrapper that doesn't know the difference will happily retry after a genuine disconnect, which means you're now making a second call the user never asked for and possibly billing for both. We build that distinction in at the point where the disconnect happens, not as an afterthought in the billing logic.
This is one of those things that sounds like a nice-to-have until you actually watch what happens without it. Users cancel generations constantly, for perfectly ordinary reasons; they changed their mind, they saw the direction it was going, they hit the wrong button. None of that should cost them money.
Caching, budgets, and the boring cost work
Prompt caching is not exciting to write about, but it's one of the few genuinely free wins in this whole stack. We set cache_control on the parts of a prompt that don't change call to call, things like the voice profile or the extraction schema in ScriptGrain, so repeat calls pay the cached-input rate instead of the full rate every time. The trick is knowing what's actually stable versus what only looks stable. Cache the fixed part, keep the volatile part, the actual user message, out of the cached block. Get that split wrong and you either cache nothing useful or you cache something that changes and blow the cache constantly.
Cost tracking is the other unglamorous half. Every AI call we make records token usage to a usage table, and a budget check runs before each call, checked against a monthly cap. Not after, before. That ordering matters, because checking after the call has already cost you money is not a budget check, it's a postmortem.
Worth knowing if you're mid-migration between model generations: Sonnet 5's tokenizer produces roughly 30% more tokens for the same input text compared to its predecessor. Same per-token pricing, higher token count, higher real spend, even though nothing about your usage pattern changed. If your budgeting assumed the old token counts, you'll blow through caps you thought were generous.
The other lever, and one that's easy to skip because it feels like premature optimisation, is model tiering. We use Sonnet for generation and synthesis work, the tasks that need real reasoning quality, and Haiku for classification, scoring, and naming, tasks that don't. Same prompts in most cases, cheaper model, at roughly a third of the cost. It's not a compromise on quality where it matters, because the tasks we route to Haiku were never the ones needing Sonnet's ceiling in the first place.
One more thing that isn't strictly cost but sits in the same bucket of "boring but necessary": user text gets wrapped in delimiters and sanitised for jailbreak patterns before it ever reaches a prompt. Untrusted input is untrusted input, whatever the use case.
What this means if you're hiring for it
If you're evaluating an agency or a contractor for Claude and LLM API integration, the prompt itself is the least useful thing to ask them about. Anyone can write a prompt. Ask what happens when the model generation changes underneath them. Ask how they handle a client disconnect versus an upstream 500. Ask whether they've actually hit the thinking-token truncation problem or whether they just read about it somewhere.
These aren't edge cases you'll never see. They're the default behaviour of the systems you're already using, and they show up the first time real users hit real load, not in a demo. We know this because we run into it constantly building our own products, ScriptGrain being the main one where these particular scars came from.
None of it is complicated once you know it's there. It's just not the part anyone talks about, because "we handle sampling param compatibility across model generations" doesn't sound like much, until it's the reason your voice-matching feature returns a 400 the morning after an upgrade.
Drafted by the Adapt Progress Evolve agent fleet; edited and approved by a human.