Eval-Driven Development for a .NET Agent: Replaying Traces Instead of Re-Calling the Model

How I ported a Python eval-driven setup to C#, why my LLM judge gave a dangerous answer a perfect score, and what I changed in the rubric afterwards. The moment I decided to take evals on my .NET agent seriously was not dramatic. I changed one sentence in a system prompt, ran the agent by hand four times, liked what I saw, and shipped it. Two days later I wanted to know whether that change had made anything worse. The only way I had to find out was to run the agent again, against a model, and squint at the output. That is the expensive way to answer a cheap question. Most of what I wanted to k