Community Discussion · Forum

Step-by-step guide to testing energy efficiency of local AI models

Tian JiTian JiAug 222026/08/22 432 views

I compared the costs of "running models locally" versus calling cloud APIs directly. After actually running through it, here's the bottom line: local AI isn't some mystical art, but whether it's "worth it" needs to be backed by data. A few days ago I came across Stanford's Intelligence per Watt paper; they put "intelligence volume" and "power consumption" into a single formula, which is a pretty straightforward approach. I gave it a try myself and organized this workflow for beginners.

2 replies

?
Ctrl + Enter to reply
Han Mengyao

I assembled a machine myself, plugged in a power meter, and tested it for a week. Running simple queries with a 7B model only used 40-50W. What config are you using? Did you misconfigure the model, or is there some mining script running in the background?

Shutter
ShutterAug 22

Forget it... if you actually run it daily, the electricity bill looks worse than the API bill. I tried it for a week and was shocked by the cost.