Buying the wrong EPYC model hurts more than paying too much

AMD EPYC server CPUs of this generation all have enough performance; the hard part is picking the right model.

A while back I was helping someone look at a machine for video retrieval, and the other party opened with "I want one with lots of cores." I pulled up the model table and saw 8 cores to 192 cores, power from 155W to 500W, dozens of SKUs under one brand. He said buying the expensive one can't be wrong. The problem is his inference service is bottlenecked on single-thread performance; giving him more cores is pure waste, and the one with higher clock speed is actually more suitable. In the end we worked backward from memory channel count and actual concurrency to settle on the model. There's no shortcut for this work—you can only match parameters one by one.

The numbers the vendor gives also need to be read carefully. Officially it says 35% improvement in general computing and 66% improvement in energy efficiency. I believe these numbers, but they're all measured relative to the previous generation under specific benchmarks. The load in your data center is not the same as their test set. In my years as a reporter, my first reaction to this kind of number is to flip to the footnote—see who it's compared against, what workload is used, whether the memory channels are fully populated. Most of the time the footnote is longer than the body text.

Looking further ahead, news of the sixth-gen EPYC Venice has already come out: 256 cores, 1.6TB/s memory bandwidth, and the first server CPU to go PCIe Gen 6. The direction is clear—cores and bandwidth keep piling up. But for ordinary enterprises, this means selection will only get more complicated, not simpler.

So my judgment is straightforward: recommend it to people whose compute needs are clearly defined and who have their own ops team able to tune BIOS and NUMA; don't recommend it to people who want to buy one machine that does everything. The latter kind of need usually ends with a lot of money spent and numbers that fall short of expectations.