Our first production incident: we ran out of API credits
How an exhausted OpenAI balance stopped post generation, why the obvious error was easy to overlook, and what we hardened while fixing it.
Posted by Jesse Peplinski
Post generation stopped working.
Today we had our first real TryLaunchLoop production incident: people could ask the app to generate posts, but the jobs were not completing.
The visible symptom was an OpenAI 429 error in our Jobs page. At the same time, we were already investigating a separate report that generated posts were not consistently reducing the post allowance shown in the app. That made the problem look like one larger failure when it was really two issues happening next to each other.
The obvious error was the right error.
The production error was telling us the truth: the OpenAI account used by our job runner had run out of prepaid credit.
We initially added funds to a different OpenAI account from the one that issued the runner's API key, so the error continued. Once the account behind the deployed key was funded, post generation started working again. This was not a mysterious model problem or something a higher usage tier could fix on its own. We had simply run out of API funding in the account that mattered.
Why we kept digging.
It would be easy to call this a billing mistake and move on, but the allowance report was still real. A user could generate content without the completed posts being reflected correctly in the counter, and generation settings could ask for more posts than the user had remaining.
So we treated the investigation as two tracks: fix the upstream funding problem that stopped the runner, and fix our own accounting rules so the product stays honest when generation succeeds.
What we changed.
Generation requests now respect the number of posts a user has remaining. Completed generated posts reduce that allowance, while scheduling the same post does not deduct it a second time. Failed generation does not consume the user's allowance.
We also removed an overcomplicated second content-supply settings screen and kept generation settings in the Content Library, where the work actually starts. The runner now uses bounded retries for rate limits, preserves the OpenAI request ID and provider detail, and shows the useful part of the failure in the Jobs page instead of flattening everything into a generic 429.
That last change is what confirmed the production diagnosis. The improved error included the actual provider message: no credits remaining.
What we learned.
The main lesson is not that billing errors are complicated. It is that two failures can overlap and make each other harder to reason about. One was operational: the account behind an API key needed funding. The other was ours: post accounting and generation limits needed stronger server-side rules.
We also learned to trust the specific error before inventing a more interesting explanation. Better diagnostics, clear ownership of production accounts, and one end-to-end test are worth more than a long theory about a plain message on the screen.
Where things stand now.
Post generation is working again. The post counter now follows generated content, generation cannot exceed the remaining allowance, and the Jobs page gives us a much clearer starting point if the provider rejects a request again.
This was a boring root cause, but a useful first production incident. We fixed the funding, hardened the product around it, and came out with a system that is easier to understand the next time something goes wrong.