Behind 1648 TFLOPs: What a MoE Pre-training World Record Means for AI Startups

At 2 AM, an AI infra founder sent me a message along with a link to an NVIDIA blog post. His tone was urgent: "Jiaxin-jie, look at this. GB300 hit 1648 TFLOPs per GPU on DeepSeek-v3 671B, setting a world record for MoE pre-training. What does this mean? Does it imply that the training costs for the hundred-billion-parameter models we discussed earlier could drop by an order of magnitude?"