Token Billing Fog: Metrics are Standardized, but Rulers Aren't [Opinion]

I ran through the numbers in this article, and the data is pretty solid. Putting together the 140 trillion daily calls, the triple difference in electricity prices, and the hundred-fold price gap based on cache hits paints an interesting picture.

This reminds me of the old "unlimited" mobile data plan ads. Carriers would say "99 RMB/month unlimited data," but the fine print hid things like "speed throttling after limit," "targeted data," and "nighttime off-peak." The seemingly unified billing unit "GB" resulted in vastly different experiences. Today's token bills are essentially the same story in a new shell.

A few points from the article stand out to me:

• Device costs baked into Token pricing: Top-tier GPU clusters start renting at the thousand-Petaflop level, and only dozens of customers nationwide can digest that. The remaining wasted compute power eventually becomes a hidden cost for all users. It's like five-star hotels spreading vacancy rates into room prices.

• Price illusions created by electricity subsidies: Electricity prices of 0.2-0.3 RMB/kWh in the Northwest rely on new energy subsidies, while 0.6-0.7 RMB/kWh in the East is market rate.

https://www.leiphone.com/category/chips/0LbefboyRMfPxvBe.html