Writing and changing prompts on purpose — how to tell a rewrite helped, and why the ten outputs you looked at were the ten you thought to try.
3 posts
· 5 min read
Temperature 0 makes decoding greedy, not deterministic — and the reasons are floating-point, batching, and routing. The reframe that fixes it: one output against one output is n = 1, and nobody accepts that anywhere else in engineering.
· 5 min read
A hardcoded prompt means a deploy to change a comma, no real rollback, no history — and, invisibly, no way to run two versions at once. The five properties a prompt version actually needs.
· 3 min read
Ten playground outputs can’t tell you what a prompt change does to real traffic. The case for measuring every prompt edit the way you measure every other production change.