Measure before you build (especially the thing you’re excited about)

Three times in the last month I nearly built something that would have made my own project worse — or at best, slower. All three times, a quick measurement before building stopped me. All three times, the counter-intuitive result was the point.

The first was a voice engine upgrade. Conventional wisdom says quantised means smaller and faster, so I assumed the ‘fast’ quantised model would win. I measured generation time on my actual hardware instead of assuming: the full-quality model replied in 2.13 seconds, the ‘fast’ quantised one took 7.49 seconds — three and a half times slower, because my CPU has no optimised kernels for the quantised format. The thing named ‘fast’ was the slow choice on my machine.

The second was a ‘save 2 seconds’ feature. Every voice command was re-loading a speech model from disk, and I assumed a resident server to keep it warm would save 1.5–2 seconds per command — worth building. I measured the actual cold-start time first: 0.85 seconds, steady across three runs, almost all of it transcription compute a warm server can't speed up. The realistic saving was about a third of a second. I didn't build it — a second managed process, startup ordering and a fallback path, for a third of a second nobody notices, fails the ‘don't build complexity for its own sake’ test.

The third was a GPU-accelerated option for a local model. Docs said it was supported; getting it working meant fighting the config. I measured the CPU fallback first instead: 26.85 tokens per second, a 38-word reply in under two seconds — comfortably fast for the actual use case. Skipped it.

All three were the same shape: something interesting to build, plausible reasoning behind it, and an honest ten-minute measurement that showed the interesting thing wasn't worth building. The counter-intuitive part is that the interesting thing is usually the one you most want to build — nobody gets excited about the boring measurement, but the measurement is what keeps a project fast, honest and simple.

Now I keep a small rule for anything that adds complexity or cost: estimate the benefit in writing, then measure it in under fifteen minutes before committing. If the measurement kills the idea, it cost less than a coffee. If it confirms the idea, I build it with confidence instead of hope.

Measure first. Especially for the thing you're excited about — that's the one where your optimism is loudest and your evidence is thinnest.