Domestic 10k-GPU Cluster Runs 236B Model: Significant, But Don't Celebrate Yet

The most valuable information in this article is: Moore Threads trained a complete 236B parameter MoE model on a real ten-thousand-card cluster using the MUSA software stack and obtained usable results. This isn't a proof of concept, but an engineering practice with actual output.

For those who have been following domestic AI infrastructure, this is an event worth serious analysis. As a tech manager, I care more about: What was the team's engineering efficiency behind this achievement? Was the ROI on resource investment reasonable? And can the conclusion that it "can train frontier models" withstand larger scales and longer cycles?

Where is the Breakthrough?

First, let's look at the technical level. A 236B parameter MoE model means complex architecture, heavy communication overhead, and extremely high requirements for cluster parallel strategies, data loading, and fault recovery. Training on a ten-thousand-card cluster has core difficulties:

• Linear Scaling Efficiency: Can adding compute cards bring near-linear throughput improvements, or does it quickly get stuck by communication bottlenecks?
• Stability: At ten-thousand-card scale, single card failures or network jitter can trigger chain reactions, requiring efficient checkpointing and fault tolerance mechanisms.
• Software Stack Maturity: Can MUSA provide operator libraries, communication libraries, and debugging tools comparable to the CUDA ecosystem?

Moore Threads running through this scale and producing results indicates at least three things:

1. Hardware Interconnect Capability Passed: Designing and tuning InfiniBand or RoCE networks at ten-thousand-card scale is hard work.

2. MUSA Software Stack Preliminarily Usable: Able to support distributed training of complex MoE models, not just simple single-card classification tasks.

3. Team Engineering Capability Commendable: From hardware deployment to model tuning, pushing this scale requires a team with considerable experience.

Issues That Need Calm Consideration

[!note]

Running a model once and stably producing reproducible training results are two different things.

As someone who has led a hundred-person AI platform team, I know that such "debut" achievements often come with massive resource tilting. For example:

• Cluster Utilization: What was the average utilization rate of the ten-thousand-card cluster during this training? If efficiency was sacrificed to get it running, the actual cost would be very high.
• Model Performance: How do the accuracy and convergence speed of the trained 236B MoE model compare to open-source models of similar scale (like Mixtral 8x22B)? Without A/B testing, we cannot judge whether the conclusion "can train frontier models" holds.
• Reproducibility: If the model structure changes or the dataset changes, can this cluster and software stack still work stably? Or was extensive manual tuning done specifically for this model?

Additionally, the embodied intelligence brain model and Peking University's 5D world model projects were smaller in scale (thousand-card level) and had academic collaboration backgrounds. Academic teams' models usually have less stringent requirements for stability and performance compared to industrial-grade products, which perhaps lowered the verification threshold for the MUSA software stack.

Observations on Organizational Strategy

Moore Threads announcing this achievement in 2026 chose a subtle timing. The domestic AI chip track is fiercely competitive, with Huawei Ascend, Cambricon, and Hygon Information all having their respective market shares. Moore Threads choosing to release a successful case study of ten-thousand-card cluster training at this time looks more like a strategic anchoring:

• Proving to the market: Domestic chips can do inference, and also large-scale training.
• Conveying to developers: The MUSA ecosystem is maturing and can handle frontier model training tasks.
• Showing investors: The technical route is feasible, and the commercialization path is clear.

But as a manager, I need to remind: Technical breakthroughs do not equal product competitiveness. Success in ten-thousand-card cluster training only solves the question of "can it be done," leaving distance to "is it easy to use" and "is the cost controllable."

The Real Challenge is the Software Ecosystem

In my view, Moore Threads' current key battlefield isn't competing on hardware specs, but perfecting the software ecosystem. The maturity of the MUSA software stack determines whether developers are willing to migrate and can efficiently use their hardware.

From this case, MUSA supporting MoE model training is a good signal. But we need to examine finer dimensions:

• Operator Coverage: Are common operators for mainstream models (LLaMA, Qwen, DeepSeek, etc.) fully supported, and how is the performance?
• Debugging Toolchain: Are profiling, debugging, and optimization suggestion tools comprehensive?
• Community Activity: Are there enough third-party libraries, model libraries, and sample codes to lower the barrier for new users?
• Compatibility: Can PyTorch native code run directly, or does it require extensive adaptation work?

These are key to winning developer trust. Running one model on a ten-thousand-card cluster only proves capability, not ecosystem.

Action Advice

If you are a decision-maker in AI infrastructure, do not equate this achievement directly with "domestic chips can fully replace others." A safer approach is:

1. Have your team run a model actually used in your business (e.g., a 7B-13B LLM or vision model) on a thousand-card cluster using Moore Threads' hardware, comparing training efficiency and convergence quality.

2. Focus on stability during training, checking for unrecoverable faults over medium-to-long cycles (e.g., more than a week).

3. Evaluate the compatibility of the MUSA software stack with existing PyTorch/CUDA code and calculate migration costs.

For Moore Threads, my advice is: Don't rush to promote "ten-thousand-card cluster training," but focus energy on the usability and documentation of the software stack. A cluster that can run a 236B model loses much of its commercial value if ordinary algorithm engineers can't quickly learn to use it.

The rise of domestic AI chips requires patience and pragmatism. This achievement deserves recognition, but true victory comes from the stability and efficiency experienced by thousands upon thousands of developers in their daily use.

Original Link: https://www.tmtpost.com/8074857.html