Q4_K_M vs Q5_K_S on three eval suites: MMLU dropped 0.4 points, GSM8K 1.1. Keep the KV cache at higher precision and quantize weights aggressively.
Recall@k on golden sets, faithfulness scoring with an LLM judge, and a regression harness in CI.
Why naive request batching wastes 60% of FLOPs and how iteration-level scheduling fixes it.