Liu Jin, Duan Lei, Yang Yan/Wen

In the past three years, China's big models have made rapid progress, but they have never truly stood at the forefront of the world. EpochAI's capability index shows that since 2023, every cutting-edge model has come from the United States, with Chinese models lagging behind by an average of about 7 months, ranging from the shortest 4 months to the longest 14 months.

Meanwhile, the gap is continuously narrowing. The "2026 AI Index Report" by Stanford University shows that the performance gap between the best models in China and the United States has narrowed from over 30% in May 2023 to about 2.7% in March 2026. DeepSeeker R1 will catch up with the strongest model in the United States for the first time in early 2025, but it will soon fall behind again.

Why can't we surpass the gap that keeps narrowing? The fundamental reason lies in training computing power.

According to EpochAI, a non-profit AI research institution, the training computation of DeepSeeker R1 is estimated to be approximately 4 × 10 ^ 3²⁴Subfloating point operation, Qwen3 Max is approximately 1.5 × 10²⁵Secondly; During the same period, the GPT-5 in the United States was approximately 6.6 × 10²⁵Next, Grok 4 is approximately 5 × 10²⁶Next, an order of magnitude higher.

Under the condition of limited computing power, Chinese large-scale models have embarked on a path of "catching up and improving efficiency". DeepSeeker V3 completed pre training with only 2048 H800 images, relying on a hybrid expert architecture and 8-bit precision training to compress the computational cost of formal training to approximately 5.6 million US dollars (excluding pre development and experimentation). During the same period, Meta trained Llama 3.1 405B using approximately 16000 H100 images.

However, there is a ceiling to efficiency improvement. The efficiency innovation of the Chinese team can be quickly absorbed by American laboratories, and with ten times the computing power added, the gap will widen again. More importantly, efficiency improvement will not reduce the demand for computing power, but rather expand it. Just as the efficiency of steam engines greatly improved in the 19th century, coal consumption in Britain increased instead of decreasing, known as the 'Jevons paradox'. The higher the efficiency, the stronger the model can be trained with the same investment, and the stronger the willingness of all parties to add computing power.

As long as the gap in training computing power still exists, Chinese big models will find it difficult to break out of the cycle of "catching up and being pulled apart again".

AI computing power is divided into two categories: training and inference. Although reasoning is becoming the mainstay of computing power demand, training determines whether cutting-edge models can be autonomously created and improved. The US chip export control targets the high-end chips and connectivity required for training. This article attempts to answer three questions: Where does the demand for training computing power come from? What is the core of supply? Where are the bottlenecks in China and how can they be overcome?

The Law of Scale: The model determines the demand for training computing power

Why are big models getting bigger and bigger? The answer is the Scaling Law.

The law of scale is not a theoretical derivation, but an empirical law repeatedly observed in practice: the prediction error of a model decreases steadily according to a power law as the number of parameters, data, and computation increases. This means that the amount of data and computing power invested, as well as the potential improvement in capabilities, can be estimated in advance. It is precisely because investment returns can be expected that tech giants dare to bet billions of dollars.

How much computation is required for one training session? If we imagine the model parameters as billions of knobs, training is about repeatedly debugging with massive amounts of data to find the appropriate position for each knob. The industry's empirical formula is: C ≈ 6 × N × D. Among them, C is the total computational cost, N is the number of parameters, and D is the number of tokens used for training. The coefficient of 6 is derived from the fact that for each token processed, each parameter requires 2 floating-point operations in forward computation and approximately 4 operations in backpropagation.

Taking Meta's publicly available Llama 3.1 405B as an example, with 405 billion parameters and approximately 15.6 trillion tokens, the computational cost is about 3.8 × 10 ^ 3²⁵Next time. If using an H100 to calculate continuously based on peak values, it would take about 1200 years. However, the computational complexity of cutting-edge models is still rapidly expanding: EpochAI estimates that OpenAI's GPT-3 (2020) is approximately 3 × 10²³Secondly, GPT-4 (2023) is approximately 2 × 10²⁵Secondly, GPT-4.5 (2025) is approximately 4 × 10²⁶Secondly, the GPT-6 Astra released in September 2026 has reached 10²⁷Magnitude.

In just six years, the training computation of cutting-edge models has increased by over 3000 times.

Training the supply of computing power: calculation, storage, connection, management, and supply

1、 From a card to a machine

The vast majority of training calculations are matrix multiplication and addition, and the same operation is repeated on massive amounts of data, making it naturally suitable for splitting into thousands of GPUs for parallel completion. So, the total computational load can be broken down into another formula: C ≈ P × G × u × T, which is the product of peak computing power (P), number of cards (G), effective utilization rate (u), and training time (T) for a single card.

This formula transforms the problem of demand into a problem of supply. The computational cost C is determined by the model, and the training time T is determined by competition. Dividing the two results in the effective computing power that must be online simultaneously during the training period. P depends on what chips can be obtained, while G and u depend on how many cards can be organized into a system and how well the system operates.

Why can't the training time be extended? Technically speaking, chips are produced annually, and when models trained for one or two years are launched, competitors may have already completed similar training with the new generation chips at a lower cost, and the technical roadmap may have been eliminated by the new architecture. From an economic perspective, the most advanced models enjoy phased monopolies and higher profits, which can be invested in the next round of training to form a positive cycle; And as soon as the pursuers catch up with the previous generation, once the leader lowers the price, it becomes difficult for the latecomers to recoup their investment. But the cycle cannot be infinitely compressed: with too many cards, communication overhead rapidly expands, and system stability decreases accordingly. In the past two to three years, the formal training of cutting-edge models has mostly been completed within three to four months.

Based on this, it can be inferred that if Llama 3.1 405B is to be trained within three months, it would require approximately 12000 H100 sheets based on a 40% utilization rate. Meta actually used approximately 16000 sheets. GPT-6 Astra is like this 10²⁷For a scale model, if GB200 (with a single card 16 bit peak of about 2.5 PFLOPS) is used, with a utilization rate of 40%, and trained in four months, about 100000 cards are required, which is consistent with its actual scale. If the computational load increases tenfold, it will require a cluster of millions of cards. Scale is forced by deadlines.

But parallelism is not free. After completing each calculation step, all GPUs must exchange intermediate results and synchronize parameters before proceeding to the next step. The more cards there are, the more data will be exchanged. The cutting-edge models are so large that no single card can fit them: a 400 billion parameter model requires several terabytes of video memory just for parameters, gradients, and optimizer states, while an H100 only has 80GB, and the model itself needs to be split into hundreds of cards. This determines that the training cluster is not a bunch of chips working independently, but must be connected to form a "single machine" through high-speed interconnection.

2、 Five Elements: Why Training Data Centers are Completely Different

For example, a traditional data center is like an office building, where thousands of unrelated small tasks share a common server, and a machine failure only affects itself. The training data center is like a production line, with tens of thousands of cards doing only one thing in a few months. Any pause requires waiting for the entire line. This production line consists of five elements: calculation, storage, connection, management, and supply. If any one of these elements becomes a weakness, the entire machine cannot operate efficiently.

Calculation is the starting point of ability. Traditional data centers use general-purpose CPUs as their core, while the typical configuration for training servers is 8 high-performance GPUs. According to TrendForce and industry independent research firm SemiAnalysis, acceleration cards account for 70-80% of the cost of training servers based on their selling price.

The key change in "memory" is high bandwidth video memory (HBM). GPU has an order of magnitude higher requirement for data throughput than CPU. If traditional memory is used, GPU spends most of its time waiting for data. HBM vertically stacks multiple layers of memory chips and packages them together with GPUs, transmitting them in parallel through thousands of channels. HBM is expensive and difficult to manufacture, with only SK Hynix, Samsung, and Micron able to supply on a large scale globally. Taking B200 as an example, HBM is worth approximately $3000, accounting for about half of the total material cost of the truck. However, the selling price of the truck ranges from $30000 to $40000, with the majority of the difference being Nvidia's gross profit.

'Connection' has transformed from a supporting facility to a lifeline. The training network must meet three requirements simultaneously: sufficient bandwidth, otherwise data cannot be transmitted; The latency should be low, otherwise the fastest card will have to wait for the slowest card; High reliability is required, otherwise a single link jitter may cause the entire training to come to a halt.

The cluster is therefore organized in layers according to communication density: Scale up uses dedicated high-speed interconnects to connect the most densely connected group of cards into a whole, such as Nvidia's GB200 NVL72, which combines 72 cards in a cabinet into an interconnected domain; Scale out and cluster these units together using InfiniBand (a high-performance, low latency network communication architecture) or Ethernet. The more cards there are, the more network layers there are, and the switching chips, network cards, and optical modules grow superlinear accordingly.

The CEO of Broadcom predicts that when the cluster moves towards 500000 to 1 million cards, network chips will account for 15% to 20% of AI chip spending. In traditional data centers, switches only account for a single digit percentage of capital expenditures.

'Guan' is a cluster software that enables tens of thousands of cards to operate like a single machine, including scheduling, fault detection, and breakpoint continuation training. It directly determines the effective utilization rate u.

U is multiplied by two parts: one is the proportion of computing power truly used for model calculations, which is the model computing power utilization rate (MFU); The second is the proportion of effective training time. Meta disclosed that the MFU trained on Llama 3.1 405B ranges from 38% to 43%, with the rest being consumed by communication, waiting, and system overhead; During training, there is an average of about one accidental interruption every 3 hours. With rapid recovery, the effective training time is about 90%. The larger the scale, the harder it is to maintain the utilization rate: the number of cards doubled from 8192 to 16384, and the MFU decreased from 43% to 41%; Based on the same failure rate, expanding to 100000 cards would result in an average interruption of about half an hour.

Supply "refers to electricity and heat dissipation. Traditional cabinet power ranges from several kilowatts to tens of kilowatts, while training cabinets can reach over 100 kilowatts, exceeding the capacity of air cooling and requiring the use of liquid cooling. Train 100000 GB200 cards for GPT-6 Astra, with supporting facilities consuming over 200 million watts of power. Training also requires clusters to be physically compact, so cutting-edge training parks are moving towards gigawatt level: a 1-gigawatt AI data center with an initial investment of about 38 billion US dollars and a power scale equivalent to a large nuclear power unit.

Five elements are calculated together: 1000 general-purpose servers and an investment of approximately 20 million US dollars in the data center, with the network being only a small part; Replacing with 1000 8-card H100 training servers, the total power is about 11 megawatts, which is more than ten times that of the former. The total investment is about 460 million US dollars, which is more than 20 times that of the former. 50% of the increment is in the accelerator card, 10% in the network, and 30% in the computer room, power supply, and cooling. The same number of servers, one side is office equipment in the office building, and the other side is a heavy-duty factory with its own substation.

3、 Supply bottleneck: Who is the scarcest and who holds the pricing power

The expansion speed of the five elements is not the same. Advanced processes and packaging are concentrated at TSMC, while HBM is concentrated at three storage plants. Electricity is subject to the construction cycle of the power grid, and these three expansions are calculated on an annual basis; The demand varies between generations according to the model, about one generation per year. The mismatch between supply and demand rhythm gives pricing power to scarce links, as evidenced by a quarterly increase of about 90% in server DRAM contract prices in the first quarter of 2026.

Prices are cyclical, while the power to set standards is structural. The specifications of HBM revolve around the demand for accelerators, and Nvidia directly purchases from three storage factories, which are then packaged by TSMC; Manufacturers of optical modules and liquid cooling also design products around their rack reference architecture. GPU manufacturers are both the engine of this machine and the rule makers.

Why NVIDIA is the center of power

From the buyer's perspective, the trial and error cost of training is extremely high. Once the system becomes unstable, the training cycle will be extended, and in severe cases, it may even fail, with months of time and billions of dollars invested in vain. Rational choice is to only adopt solutions that have been extensively validated, paying a premium for certainty.

Meta handed over the online inference of Llama 3.1 405B to AMD, but still used Nvidia for training. The reason is that inference can be tested with partial traffic first, but training cannot. The more people use it, the more thorough the verification of the plan, and the less likely the next buyer is to replace it, which is a typical network effect.

From the seller's perspective, Nvidia's advantages precisely cover the three variables that suppliers can control in the formula: P, G, and u. The computing power of a single card is P generation by generation, continuously leading, but the paper parameters of competitors can catch up; What is truly difficult to catch up with is G and u, that is, whether tens of thousands of cards can be connected into one machine and run stably for several months, and how much computing power is truly used for calculation during operation.

Nvidia acquired Mellanox and InfiniBand in 2020, taking control of both vertical NVLink (Nvidia's high-speed interconnect bus and communication protocol) and horizontal cluster networks; The communication library and operator library are built on the CUDA ecosystem of the past 20 years. The larger the scale, the harder it is to maintain utilization, and Nvidia's dominance comes from this system that covers chips, networks, and software, built to resist the efficiency decline caused by scale.

Therefore, the training market presents a pattern of one dominant player and a few catching up players, with the true pursuers being the entire system. Google is able to train Gemini with its self-developed TPU because it possesses chips, software, data centers, and model development simultaneously. The competition on the training side does not reward individual champions, only manufacturers who can turn tens of thousands of cards into one card and make customers who are extremely averse to risks use it with confidence.

The current situation of computing power training in China: five elements, four with shortcomings

The training of global large-scale models is mainly conducted in the United States and China. The prominent feature of the supply of training computing power in the United States is the abundance of capital. The capital expenditure of major cloud providers increased from approximately $131 billion in 2021 to approximately $416 billion in 2025. According to EpochAI's statistics, as of May 2025, the United States accounts for about three-quarters of the global GPU cluster computing power, while China accounts for about 15%.

In the United States, although chip and HBM are tight, the market-oriented expansion mechanism is operating normally, and the hardest constraint is actually electricity: the proportion of data center electricity consumption in the United States will be 4.4% in 2023, and it is expected to rise to 6.7% to 12% in 2028.

We use the same framework to examine China's computing power supply.

Calculation: High end GPUs cannot be purchased, and the bottleneck of self-developed is not in design, but in advanced processes. Behind the process is the control of lithography machines and other equipment. Huawei's Ascend 950DT for training has an 8-bit precision computing power of approximately 1 PFLOPS, which is more than 20% of the Nvidia B200; The next generation Ascend 960DT is expected to be ready in the first quarter of 2027, with a single card of about 2 PFLOPS, still less than half of the B200.

Storage: The overseas HBM procurement channels have been basically cut off, and the only clear way for domestic production is through Changxin Storage. According to reports, Changxin Storage has conducted small-scale trial production of HBM3E, which is one generation behind the international top three. The comprehensive yield of the previous generation HBM3 is estimated to be only about 25%.

'Lian': both long and short. The optical module is dominated by Chinese manufacturers, but high-speed DSP (digital signal processor) chips and high-end lasers still rely on imports; Huawei has achieved 51.2T for switching chips, while Broadcom has reached 102.4T.

Management: CUDA has accumulated nearly 20 years of experience, and the operator library, communication library, and toolchain of domestic platforms still need to be supplemented, but the gap can be caught up.

Supply: This is the strongest aspect in China. In 2025, China will add 540 million kilowatts of installed power generation capacity, which is ten times that of the United States in the same year.

The result is that domestic computing power is currently mainly used for inference, and there is no internationally advanced model that has completed pre training on domestic systems.

Using the formula from the previous text, we can deduce that the same 10²⁷The scale model, also trained within four months, requires about four to five times the number of cards required for the Ascend 950 chip, which is four to five times that of Nvidia's solution, due to the single card computing power being about one-fifth of B200. The more cards there are, the harder it is to maintain utilization, and the required HBM and interconnect devices also increase exponentially. This is precisely the essence of China's bottleneck in training computing power.

In recent years, the large number of intelligent computing centers built in various regions have also been unable to fill this gap: they are limited in scale, with different chip models, and even if there is idle computing power, they cannot be combined into a "single machine".

How to break through the bottleneck of training computing power

The short-term bottlenecks are insufficient single card computing power, shortage of HBM, and immature software ecosystem. There are at least three strategies to deal with it.

One is to compensate for single cards with the system. The gap between single cards is difficult to catch up with in the short term, so efforts should be made in the number of cards G and utilization rate u, to make up for the shortcomings of single cards with more cards, more electricity, and stronger engineering organization capabilities.

This is exactly Huawei's path: Hypernodes expand the tightly interconnected range to 4096 cards across cabinets, Lingqu bus makes thousands of cards more like a machine at the software level, and Near Encapsulation Optics (NPO) reduces the power consumption and failure rate of interconnection.

The cost is energy consumption. SemiAnalysis calculates that the unit computing power consumption of Huawei CloudMatrix 384 supernodes is more than twice that of Nvidia GB200 NVL72. But electricity only accounts for about 7% of the annualized total cost of a gigawatt level data center, and doubling electricity consumption only increases the total cost by about 10%. Electricity is precisely the most abundant resource in China.

Secondly, improve efficiency at the model level. The hybrid expert architecture only activates some parameters, which can significantly reduce computational complexity; 8-bit low precision training doubles the peak computing power and halves the communication volume of the chip, equivalent to improving the performance of a single card. DeepSeek has proven that this path is feasible.

Thirdly, make good use of imported chips. This year, China has taken a cautious attitude towards the import of Nvidia H200, and it is reported that the approved quantity may be less than half of the company's application quantity. The concern is understandable: if there are too many imported chips, domestic chips will lose the opportunity for verification. But there is a time window for cutting-edge competition, missing a generation and the cost of catching up increases exponentially.

Instead of blindly emphasizing domestic substitution, it is better to take safety and reliability as the standard and walk on two legs: cutting-edge training uses imported chips within the permitted scope to seize time; Domestic chips will first increase their production in inference and medium-sized training, gradually advancing forward. Resources should be concentrated in a few large training clusters to avoid scattered investment.

The bottleneck in the mid-term lies in mass production. Without EUV lithography machines, we can only rely on the limited stock of DUV lithography machines to approach advanced processes with multiple exposures. The equipment inventory and yield determine the production capacity of Ascend; An Ascend 960 is estimated to require 8 to 12 HBMs, with card amplification and synchronous HBM amplification.

There are three directions for breakthroughs: first, the yield and production capacity of domestic GPUs and HBMs will climb; The second is to address the interconnection shortcomings of high-speed DSP chips, lasers, and switching chips; The third is to promote the software ecosystem from being "usable" to being "easy to use". The Ascend software stack CANN announced full open sourcing in August 2025, and DeepSeek also opened sourced training components for Ascend. Open source technology is essentially anti-monopoly, and neither party can use it as a bottleneck.

In the mid-term, there is another market problem that is difficult to solve on its own: who will be the first person to eat crab?

Train buyers to be extremely risk averse, with no one willing to bet months and billions of dollars on the new platform first; If no one uses it, the platform cannot be verified, and the less verified it is, the more unused it becomes. The experience of the first crab eater can benefit the entire industry.

At this point, the government needs to intervene, but the approach is crucial. The government does not need to personally organize teams or allocate resources, but can propose demands and set rewards: whoever can complete a cutting-edge scale training on the domestic computing power system will receive rewards or guaranteed procurement contracts. If it cannot be achieved, rewards will not be issued, and the government will not bear financial risks; It's done, every penny comes with tangible results. How to implement it specifically, leave it to the market that knows the situation best.

In the medium term, we must also be vigilant about excessive concentration of risks. At present, Huawei is the only company in China that can deliver a complete set of cutting-edge training systems, and it actually plays the role of "China's Nvidia". Only when the hub is established can the ecology operate; But once the route deviates, there is no second system to provide a backup. Cultivating a second training system and promoting interface and software compatibility are important tasks in the mid-term.

The long-term determining factor is the core equipment led by lithography machines, which is a common bottleneck for advanced process chips and HBM. The real bottom battlefield lies in the research and development of domestically produced lithography machines, as well as equipment for etching, deposition, and measurement.

At the same time, we should also explore new paths to bypass process limitations, such as Huawei's proposed logic folding, but according to its disclosure, it will not be used for training chips until around 2030.

Although funding is important for equipment research and development, talent is even more crucial. Photolithography machines involve many disciplines such as optics, precision machinery, materials, and control. The cultivation of top talents takes ten years and cannot be rushed, but it must be started now.

In short, short-term efficiency improvement relies on system engineering and modeling, mid-term efficiency improvement relies on capacity ramp up and large-scale validation, and long-term efficiency improvement relies on core equipment and talent. The three stages compete for the same thing: whether they can rely on their own strength to turn tens of thousands of cards into one machine.

Improving efficiency allows us to catch up closer, while computing power enables us to stay ahead. The law of scale is still in effect, and the threshold for cutting-edge training is increasing exponentially every year. This is a long-distance race, competing for sustained endurance and direction.

(Liu Jin is a professor of accounting and finance and director of the Investment Research Department at Cheung Kong Graduate School of Business, Duan Lei is a partner at Shenzhidu Consulting, and Yang Yan is a senior researcher at Shenzhidu Consulting.)