Get your prompts out of your codebase

Somewhere in your repository there is a triple-quoted string that decides what your product says to every user. Changing one word in it requires a branch, a review, a CI run, and a deploy. At 2am, rolling it back requires all of that again, performed by someone who is not fully awake.

Meanwhile the person who best understands how that wording should read — support lead, product manager, the domain expert who knows what the answer is supposed to sound like — cannot touch it at all.

Four costs, one of them invisible

Deploy latency on the fastest-moving part of the product. Prompts change weekly. Application code does not. Coupling the two means the thing that iterates fastest is gated by the release process built for the thing that iterates slowest.

No rollback worth the name. “Revert the commit and redeploy” is a rollback in the sense that a fire extinguisher in a locked cupboard is fire safety. When quality falls off a cliff at 2am, you want a pointer moved, not a pipeline run.

No usable history. When did this change, who changed it, and what did it say before? is answerable from git in principle, and in practice means bisecting through unrelated commits to reconstruct the exact string that was live on 14 July. Every incident review needs this. Almost no one can get it in under an hour.

No way to run two versions at once. This is the invisible one, and it’s the expensive one. If the prompt is a constant in the binary, then at any moment exactly one version exists in production. You cannot compare. You cannot experiment. Every prompt change is a full cutover, and the only evidence you’ll ever have that it helped is that nobody complained.

That last constraint is why so many teams ship prompts on vibes rather than measurement — not because they don’t believe in evidence, but because their architecture makes gathering it a project.

Be fair to what you’ve already built

Most teams have already done something about this, and the intermediate solutions are real improvements. It’s worth being precise about what each one buys.

A prompts module in git — strings pulled out of business logic into prompts.py — gets you review, history, and diffs. It’s a genuine step up and costs nothing. It does not get you runtime changes, runtime rollback, or two versions live at once, because the file still ships in the binary.

A config file in S3 or a database, loaded at startup or on a timer, gets you runtime changes. It typically loses review and history, unless you build those, and now you own a small configuration service with a cache-invalidation problem and no audit trail.

A feature flag per prompt gets you two versions at once, which is most of the way there — but flag systems store booleans and short strings, not multi-kilobyte text with structure, and they don’t know anything about which model the text was written for.

Every one of these is a reasonable thing to have built. The point isn’t that they’re wrong; it’s that the requirement list they’re each missing part of is short and knowable.

What a prompt version actually needs

Five properties. A system that has all five solves the problem; one that misses any of them will fail you at a specific, predictable moment.

• An immutable version identifier. Once published, a version never changes. This is what makes a logged response traceable back to the exact text that produced it — without it, your analytics say “the prompt was v3” and v3 means three different things depending on the week.
• The full text, stored whole. Not a diff chain. Reconstruction-from-diffs works right up until the moment you need it most, and storage is not your constraint here.
• The model and parameters it was written for. A prompt is tuned against a specific model’s quirks. Carrying it to a new model unchanged is one of the most reliable ways to ship a silent regression, and a version that doesn’t record its target model invites exactly that.
• Author and timestamp. Who changed it and when. Every post-incident conversation starts here.
• A rollback that is a pointer move. Not a deploy, not a revert, not a build. Change which version is active and have production follow within seconds.

Notice what falls out of the fifth requirement for free. If production reads which version is active at request time, then production can just as easily read different versions for different users. Rollback and experimentation are the same mechanism.

The part where it becomes an experiment

Once prompts are addressable at runtime, running two of them side by side is a configuration change rather than a code fork:

from llmjury import Client

client = Client(experiments=["support_prompt_v2"])
llm = client.wrap(anthropic_client, "support_prompt_v2")

# Returns whichever variant this user is assigned to — deterministically, so they
# see the same one tomorrow. `default` is what runs if LLMJury is unreachable.
p = client.get_prompt("support_prompt_v2", user_id, default="You are a helpful assistant.")
response = llm.messages.create(
    model="claude-haiku-4-5", system=p.prompt,
    messages=[{"role": "user", "content": user_input}],
)

The default argument is doing more work than it looks. It means adding prompt management doesn’t add a hard dependency to your request path: if the service is unreachable, your code runs the string it always ran. A prompt system that can take your product down when it has an outage is not an improvement over a constant, and you should ask that question of anything you adopt.

Assignment is deterministic — a frozen bucketing hash over the user identifier, identical across the Python, TypeScript, and Java SDKs — so a user doesn’t flip variants between two turns of the same conversation, and editing an experiment doesn’t reshuffle people who are already in it.

LLMJury stores prompt versions with full history, one-click rollback, and stamps the config version onto every event it records, so a result always knows which text produced it. That last detail is the one that makes six-months-later questions answerable.

Where to start

You do not have to adopt a platform this afternoon. Pull the strings out of your business logic into one module today — that alone makes the next step easy and costs you an hour. Then decide whether you need runtime changes, and if you do, make sure whatever you pick has all five properties above rather than three of them.

The test of whether you got it right is simple: can you change a prompt without a deploy, see who changed it last, roll it back in ten seconds, and run two versions at once? If the answer to the last one is no, you can’t measure your own product.

Start free — no credit card — or read the five-minute quickstart to see the whole loop.