AI Agent Giving Different Answers? Temperature 0 Isn't a Guarantee

Setting temperature to 0 is supposed to make output deterministic — same input, same output, every time. In practice, some models still show small variation at temperature 0, due to how the underlying inference infrastructure batches and processes requests.

This catches people off guard specifically because temperature 0 is marketed as fully deterministic, so the small variation that does happen looks like a bug rather than a known limitation.

The fix:

for genuinely critical, must-be-identical outputs (compliance text, exact legal wording), don't rely on temperature alone — cache the first real output and reuse it, rather than regenerating and hoping for consistency every time.