Economic Observer reporter Zheng Chenye

Since October, stocks in the global AI industry chain have undergone a sharp adjustment.

As of October 16th, the Philadelphia Semiconductor Index in the United States has fallen for 8 consecutive trading days, setting a record for the longest consecutive decline since 2008. On October 22nd, chip and AI stocks in the Asian market once again led the decline, with the South Korean KOSPI index closing down 3.52%, SK Hynix falling more than 5%, the A-share ChiNext index falling 4.37%, and computing hardware stocks collectively weakening.

According to the guidance provided by Amazon, Microsoft, Google parent companies Alphabet, and Meta in late July, all four companies will have capital expenditures exceeding $100 billion this year, mainly invested in AI infrastructure such as data centers. The ability to recoup these investments and the extent of the risk of debt expansion are currently the focus of debate in global capital markets.

All of this will pass, for the simple reason that AI will bring about change and impact every industry and every country in a very positive way. ”On October 16th, Huang Renxun, the founder and CEO of Nvidia, who was on a tour of Europe, responded to market fluctuations in a media interview in Berlin. In his view, the transformation brought by AI is still at the starting point.

On October 21st, Huang Renxun delivered a keynote speech at the GTC (NVIDIA GPU Technology Conference) in Berlin, using the logic of manufacturing to explain the input and output of AI factories (data centers that train and run AI models). We discovered early on that AI is actually a manufacturing industry. ”He said that the way computers operate has fundamentally changed, from tools used by humans to factories that produce intelligence.

In Huang Renxun's description, AI factories use electricity as raw material to produce tokens (word elements, the basic unit for AI model processing and information generation) for use in various industries. He estimated that a 1-gigawatt AI factory would cost approximately $50 billion, including land, electricity, factory buildings, as well as computing, networking, storage, cooling equipment, and system integration, and is expected to generate tens of billions of dollars in revenue annually.

According to Huang Renxun's manufacturing logic, whether the investment in AI factories can be recovered depends on whether the factories can be built quickly and use less electricity, and whether there are enough buyers for the tokens produced after completion.

Efficiency of AI factory

When introducing the design plan of the AI factory, Nvidia summarized the goal as "producing more intelligence per watt of electricity and producing the first token in a shorter time". In terms of manufacturing, the former refers to the output per unit of electricity, while the latter refers to how quickly a new factory can be put into operation, both of which are related to how long the AI factory can recoup its investment.

The amount of tokens that an AI factory can produce largely depends on the performance of computing devices, and the improvement in performance used to mainly rely on advances in chip manufacturing processes.

Huang Renxun stated in his speech that Moore's Law has basically come to an end, and even if it is still valid, chip performance can only improve by about 100 times in ten years. He said that in the past 11 years, AI performance has improved by 1 million times, thanks to the "ultimate collaborative design" of chips, systems, software, and algorithms.

When power supply is limited, how many tokens are converted from each kilowatt hour of electricity determines the output of the AI factory. According to information released by Nvidia, according to its design plan, the power consumption of AI factories can be reduced by up to 30% under the same output conditions; After testing Nvidia's power management software on the previous generation GB300 system, UK AI cloud service provider Nscale was able to deploy 37% more GPUs and increase the total amount of tokens produced per unit time by nearly 50% while maintaining the same power supply quota.

Huang Renxun gave an example that if a 1-gigawatt AI factory brings in $40 billion in revenue annually, delaying production for three months would result in a shortfall of approximately $10 billion in revenue. To shorten the production time, Nvidia released a reference design for the entire AI factory at the conference. According to this design, partners first install equipment in containerized modules, complete integration and verification, and then transport them to the construction site. On site deployment time can be shortened from 6 months to 14 days, provided that the site facilities are ready.

Huang Renxun believes that embedding assembly work into the supply chain is a complete transformation of the global supply chain, which will change the installation and construction methods of data centers, reduce idle and wasteful computing power and capital investment, and improve the profitability of all parties in the industrial chain. He also mentioned that Nvidia had previously put the assembly of the entire cabinet into the supply chain, which allowed it to supply the construction of hundreds of data centers worldwide at the same time. "We are now producing computing equipment on an unprecedented scale.

Huang Renxun hopes that Europe will also build its own AI factory. To become a part of the AI economy, one cannot just be a consumer of AI, but a producer of AI. ”He said, "We can definitely achieve all of this in Europe, in fact, we have already started

Token Demand Scenario

On the demand side, Huang Renxun believes that infrastructure must be implemented before the AI model, just like the communication network before the Internet. In his view, the main force consuming tokens in the future is intelligent agents, as they think, reason, iterate repeatedly, and use tools. The more useful they are, the more tokens they consume, and the demand for GPUs will also increase significantly.

To truly land in enterprises, intelligent agents need to obtain access to files, networks, and various tools. The usefulness of an intelligent agent depends on how much access permission you give it, "said Huang Renxun. Of course, the first priority is to ensure the security of the intelligent agent, which should be as small as possible and monitored. The OpenShell security reference architecture released by Nvidia at the conference places the intelligent agent in an isolated environment and limits what it can access and do at the chip level, which facilitates the secure deployment of intelligent agents for customers.

According to Huang Renxun's judgment, in the future, every company will use both closed models and customized open models, and token buyers will also spread across various industries. He also stated that Nvidia will publicly disclose the parameters, complete training methods, and data of the open model, so that enterprises can reproduce and customize the model based on this.

Huang Renxun also mentioned that some people believe that the advancement of AI will bring software and tool companies to an end, but his view is exactly the opposite. He believes that there will be more users of these tools than ever before, and every company that makes professional software tools will become an AI company, an intelligent agent company. In the case presented by Nvidia, the engineering agent of Dassault Systemes, a French design and simulation software company, has been able to identify problems in vehicle design through simulation and improve the design accordingly. Of course, such simulations themselves require a significant amount of computing power.

Further and greater demand comes from physical AI (AI that enables cars and robots to perceive and act in the real world). According to Huang Renxun's introduction, three types of computers are required to create a physics AI system, which are used to train models, teach machines skills in digital twins, and run on robots. The first two types of computers are usually placed in data centers, which also means that physical AI has to consume the computing power of data centers before it can be deployed on a large scale.

In Huang Renxun's view, autonomous taxis are a huge opportunity, but the market is extremely fragmented. He announced at the aforementioned conference that the next generation models of car companies such as Volvo Cars and Mercedes Benz will be compatible with NVIDIA's standardized autonomous driving architecture Hyperion. According to his vision, with this industry standard and the collaboration of the entire ecosystem of enterprises, a huge "virtual fleet" that everyone can access can be formed.

As for humanoid robots, Huang Renxun compared them to smartphones, personal computers, and Nvidia's own NVL72 full cabinet system, saying that their rapid progress is due to standardized configurations that allow many customers to use them, which in turn leads to sales and drives technological iteration faster and faster. Based on this, he believes that the form of robots should also be standardized as much as possible, and humanoid robots can be applied to many scenarios, so they are likely to become the best-selling type of robot.

When asked about the return on investment in AI in the capital market, Nvidia's factory reference design and various standard architectures presented at the conference may be aimed at maximizing market size through standardized configurations.

(Author)