AI Processing is strecthing Datacenter Capex
The semiconductor supply chain is being dramatically stretched by the current AI boom. While all eyes are on Nvidia and their amazing results, the company’s success is creating ripples in the supply chain.
Upstream, Nvidia’s suppliers are struggling to follow the crazy demand for GPU’s for AI:
TSMC’s 4nm production line is running full steam while Nvidia is climbing the customer list of the Taiwanese technology leader.
Memory companies are fighting to launch new High Bandwidth Memories and increase the yields to sensible levels.
Outsourced Assembly and Test companies (OSAT) are franticly trying to handle the new packaging technologies CoWoS that enables the high bandwidth connection between HBM and the GPU enabling the performance needed for AI Learning and Inference.
Datacenter construction companies and the cooling industry will need to handle a new level of direct liquid cooling needed for Nvidia’s Blackwell architecture.
Silicon Wafer companies will need to supply more high performance wafers to the memory companies as HBM memories will require 2-3x the silicon area compared to more ordinary DDR DRAM.
The networking companies needs to deliver new high performance optical and silicon products capable of feeding the frantic GPU’s with sufficient amounts of data.
The need for standard memory in the Data centre is also increasing dramatically. Micron recent good quarterly result was not driven by HBM but by more “ordinary” DDR5 Dram and SSD Flash.
And the list goes on. The HGX-100 and 200 systems have more than 35.000 individual parts, most of which are at new performance levels that need to be supplied.
This could be a short lived ripple before the supply chain returns to a more balanced state as has happened before although indications are that it will take some time before AI becomes saturated.
Cathie Wood and Ark Invest has given their view of the AI revolution in a white paper.
Ark Invest believe that AI will represent the highest technological impact on GDP ever.
It will coincide with a number of other technologies that will be interlocked but AI will be the dominant technology
Most of the equity growth from 2023 to 2030 will come from distruptive AI based technologies.
The downstream view
The results of Nvidia clearly shows that the ARK Investment predictions have taken off.
What is less visible is what goes on downstream in the complex AI/GPU supply chain, other than there is an incredible demand for GPU and AI.
Most of the large hyper scalers and cloud companies that are buying the GPU systems, does not give direct insights into the size of their datacenter’s and their operations.
While the large public companies report their quarterly Capital Expenditure, they rarely give insight into where the Capital is expended.
It is also quite complicated to compare companies with very different business models.
Microsoft is a software company that increasingly is selling cloud services and products.
Amazon sells everything from toilet paper to cloud services.
Apple is a product company with increasing service sales.
Google and Meta are focused on advertising
Below these companies are a large group of companies that already are in AI or have an interest in expanding into GPU/AI business.
Jensen Huang logo-dropped quite a few at GTC 2024:
The data centre Capex of these companies has to cover a lot more than just the GPU systems of predominantly Nvidia and AMD. The modern Datacenter is a complex web of supply systems contained in a big box that needs significant supply of power and cooling water and a controlled environment. The AI server is just one of many parts of this system.
In order to get insight into the downstream demand, it is necessary to take a deeper into the financials of the large AI companies.
Although the look is deeper, we are not accountants or financing specialist and the analysis we do, is to generate insights, not to balance the books of a large public corporation.
The simplified financial trail we are following is that when a company buys new equipment, it is first showing up in the third of the three ledgers: The cashflow statement under investment cashflow. The addition to property, plant and equipment (PPE) or Capital Expenditure as it is more commonly known.
Once bought, the value of the CapEx is added to Property, Plant and Equipment in the Balance sheet. Rather than deducting the total value of the purchase from the profits of the Profit and Loss sheet, most companies depreciate the assets over its lifetime to expand the depreciation from profit over the lifetime of the asset.
Companies that have semiconductor based assets, will know that the useful life of these products are very short compared to other equipment and buildings.
Semiconductor based assets like server cards and memory is normally written of over 4 years that is also close to its useful life. This will be well known to anybody with a PC or MAC older than 4 years. At the other end of the spectrum is buildings that typically gets depreciated over 20 years or more.
Semiconductors age as gracefully as dead fish
This way of depreciating can help us gain insight into the size and the investment needed in replacement and new equipment of the large Datacenter companies.
The Downstream AI demand.
Our recent analysis of the major Datacenter owners shows the book value of the Datacenter assets are significant.
It is no surprise to find Amazon as number one, while slightly more surprising that top 4 is quite close. All of these companies are on the datacenter treadmill, they will have to run fast just to stay put. An average of 4 year in depreciation rates means that a significant part of the capital investments have to go into replacing equipment before investment in expansions. Each of the companies have different depreciation rations as can be seen in the graphs. We have used individual depreciation ratios for each company. This shows Microsoft and Amazon have more aggressive depreciation than Google and Meta. This could just be a financial decision or there could be a deeper reason why the two companies equipment degrades faster financially, also the depreciations could be temporary and over a longer time, the companies would look more similar.
With the massive increase in computing power, the Datacenter owners will have to depreciate faster than today. What is an H100 system worth once Blackwell become available? What about the next Nvidia generation. It could be that the datacenter owners already are accelerating the depreciation of GPU’s.
The output of the analysis, allows for allocation of the Datacenter Capital Expenditure into two categories. The first is the Capital that is needed to replace equipment so it is in the same state as last quarter (measured in financial value - PPE). The 2nd part, the Addition Capital Expenditure is what is left for real expansion of the Datacenter and will make the Datacenter a more valuable asset.
The overall allocation of Capital of the top 6 companies, is 61.5% replacement and 38.5% addition capital. This also gives an idea of which of the Datacenter owners are expanding right now. Apart from Apple, all of the other companies are expanding significantly, with Meta and Google as the most aggressive. This could be an indication that Apple is not focused on building their own Datacenter AI capability right now, but will rely on others. AI on Apple devices, might be a different issue as the neural performance of the new Apple Silicon suggests.
Processing costs in the Datacenter
The Datacenter processing revenue of the semiconductor companies delivering to this market, has grown more than 150% the last 4 quarters. The increase in revenue is not driven from the supply side of the processing companies. The costs are not increasing at the same rate as the revenues.
Rather this increase is driven by the datacenter owners, that cannot get enough of Nvidia’s latest GPU’s for their AI applications. The size of the processing bill is now becoming significant. It takes the Datacenter Capex of the 5 largest Datacenter owners to pay the processing bill for the industry.
A new datacenter consists of many different elements of which the server part is becoming increasingly important. As could be seen from the depreciation graph, over time, the server element of the cost increases as servers depreciate faster than other equipment in the data centre. A disproportionate cost of the Data Center Capex is spent on processing, and as this analysis have shown, it is increasing significantly at the moment.
The rest of the Datacenter Market
This analysis is not an attempt of sizing the global Datacenter market but to get an idea of the scale of the global investment is still reasonable.
Gartner just released the forecast for IT spending in Europe where datacenter spending is detailled:
The Datacenter spending shown here is only the IT part of the investment, roughly translated into the server and networking elements of the Data Center investment. Europe accounts for approx 15% of the global economy. Using that assumption gives a 333B$ global spend. Our forecast of datacenter processing revenue is 130B$ for 2024. This makes processing revenue at 39% of all IT costs in the datacenter. As the datacenter economy is skewed towards US the share could be lower, but this rough estimation can certainly highlight that the processing share is significant.
Next steps of our model
While the method and models we use are based on approximations and assumptions it is not guess work. We uses hard financial data from each of the companies that is tracked. Over time, we are going to refine the model and triangulate with other data, to get a tighter fit with reality.
While the method is not exact, it is still useful to reveal key insights into the spending behaviour of the Datacenter giants. Insights rarely need decimal accuracy.
Exact is not the same as useful, nor is inaccurate the same as useless
Obviously insiders will know more, but only about their own company and not about others. While we do not want insiders to break the law, we would like inputs from others in the industry. Should you have data or information that paints a different picture, we would love to hear from you.