Physix FrontierPhysix Frontier · News Briefing Card (HuggingFace · Nov 9, 2026)
Amazon's HOPE second-order expert pruning can halve MoE experts
KEY FACTS
- Amazon researchers propose HOPE, a method that decides which experts to prune based on pairwise expert interactions.
- HOPE is a second-order pruning approach; existing methods score experts independently and ignore expert collaboration.
- At a 50% pruning rate, HOPE achieves the best average ranking, with gains of up to 6.1% on agentic coding tasks.
- Taking Qwen3.5-122B as an example, its 48 MoE layers each contain 256 experts.
- Of the model's roughly 250GB of parameter memory, 95% is occupied by expert MLPs, while each token uses only 8 experts per layer.
KEY DATA
6.1%HOPE max gain on agentic coding tasks
50%Experimental pruning rate
48Qwen3.5-122B MoE layers
256Qwen3.5-122B experts per layer
PHYSIX OBSERVATION
MoE saves compute but eats memory, so pruning is the realistic way out. HOPE brings expert collaboration into the decision, preserving performance even when half the experts are removed, which shows that coarse independent scoring has hit its ceiling. For deployers, this means a larger model can run in the same VRAM, but second-order methods depend on calibration data, and the cost of putting them into production still bears watching.
Source: HuggingFace report