Will Google’s TurboQuant algorithm hurt AI demand for memory chips?

{"text":[[{"start":9.35,"text":"Samsung Electronics’ blowout first quarter has eased investor concerns that a new Google algorithm might threaten the AI-driven boom in South Korea’s memory chip industry."}],[{"start":22.93,"text":"Citing an “unprecedented supercycle” in the memory chip market, Samsung this week estimated higher profits in a single quarter than in the whole of last year, with no sign that memory was becoming less of a bottleneck for AI companies."}],[{"start":39.019999999999996,"text":"The earnings guidance sent Samsung shares close to all-time highs and eased two weeks of anxiety sparked by TurboQuant, a technology outlined in a Google Research blog post in late March, which promises to drastically reduce the amount of memory required for AI."}],[{"start":58.81999999999999,"text":"The post ignited a fierce and ongoing debate about future demand for high-bandwidth memory, the advanced chips made by Samsung and its South Korean rival SK Hynix that power AI servers."}],[{"start":73,"text":"Some investors believe the memory boom will turn to bust, others think TurboQuant will have little impact, while optimists argue that if the technology does make AI cheaper, it will simply create demand for even more AI, and thus more chips."}],[{"start":90.03,"text":"TurboQuant “potentially slashes the cost of running large language models by a factor of four to eight”, said Kwon Seok-joon, a professor at Sungkyunkwan University in Seoul. “At first glance, this appears to threaten demand for high-bandwidth memory chips.”"}],[{"start":107.92,"text":"However, “dramatically cheaper inference unlocks workloads previously too expensive to run”, such as real-time coding assistants and multiple AI agents running at the same time, added Kwon, “driving total compute demand higher, not lower”."}],[{"start":125.66,"text":"TurboQuant works by compressing the so-called key value cache — the short-term memory that allows AI models such as ChatGPT and Claude to retain conversational context — and reconstructing it when needed, with little apparent loss in accuracy."}],[{"start":142.85,"text":"As AI interactions lengthen and user numbers rise, demands on the KV cache are surging, putting strain on how much memory AI services can afford to use."}],[{"start":155.54999999999998,"text":"TurboQuant offers a way out, reducing the “cost per token”, the amount of computing and memory expense required to process each unit of data handled by an AI system. Google’s researchers claim the approach could cut memory usage by as much as sixfold."}],[{"start":173.33999999999997,"text":"The blog post caused shares of Samsung and SK Hynix to fall sharply last month. But analysts and researchers now suggest that if TurboQuant does work, it is more likely to expand overall memory demand than reduce it — an example of the Jevons paradox, in which greater efficiency increases overall usage of a resource."}],[{"start":null,"text":"

"}],[{"start":197.47999999999996,"text":"Economist William Stanley Jevons noted in his 1865 book The Coal Question that James Watt’s more efficient steam engine had resulted in greater usage of the fuel because it made coal-powered technologies economically viable in far more contexts."}],[{"start":215.36999999999995,"text":"Han In-su, one of the researchers upon whose work TurboQuant is based, told the FT that the algorithm “can serve as a foundation for realising previously impossible high-difficulty tasks, such as processing much longer contexts within limited memory resources without sacrificing accuracy, or implementing high-performance AI on smaller devices”. "}],[{"start":240.54999999999995,"text":"In a research note, Kim Young-gun of Mirae Asset Securities invoked “déjà vu” over Kubernetes, a Google-designed “containerisation” technology that made it possible to run multiple applications on a single server, greatly improving hardware efficiency."}],[{"start":258.98999999999995,"text":"Upon its widespread adoption in the late 2010s, there were concerns that demand for servers and memory would fall as companies would need fewer resources to produce the same results. In practice, the opposite occurred, with lower costs encouraging much greater usage."}],[{"start":277.81999999999994,"text":"“The market has largely misread TurboQuant,” said Ray Wang of research firm SemiAnalysis. “We continue to believe that increasing memory demand will be required for both training and inference as AI models evolve and innovation advances.”"}],[{"start":295.8999999999999,"text":"Any potential blow to the South Korean chipmakers would be cushioned by the increasing use of long-term contracts from AI service providers seeking to lock in supply, said Wang."}],[{"start":308.1799999999999,"text":"“Memory is becoming a bit less cyclical, driven by accelerating and sustainable AI demand,” he said. “Contract pricing now matters more than spot pricing.”"}],[{"start":319.51999999999987,"text":"At Samsung’s annual meeting last month, co-chief executive Jun Young-hyun said the company was pursuing “contracts of three or five years with major clients, shifting from the existing quarterly and annual terms”."}],[{"start":333.85999999999984,"text":"For now, TurboQuant remains a concept in a blog post. Its real-world impact will become clear after it is presented at the International Conference on Learning Representations in Brazil in late April and people outside Google are expected to be able to test it. Its ultimate success will depend on whether the largest tech groups are able to use it at scale."}],[{"start":356.5099999999998,"text":"“We never imagined that a technology that started from the academic question of ‘How can we compress data more perfectly?’ would cause such a huge social and economic ripple effect,” said Han."}],[{"start":378.9499999999998,"text":""}]],"url":"https://audio.ftcn.net.cn/album/a_1775978878_7334.mp3"}