Feb
“Distillation” Is Shaking Up The AI Industry
jerry9789 0 comments artificial intelligence, Brandview World
Copyright: Airam Dato-on
Paradigm Shift
We’ve recently written about recent AI advancements and popularity, particularly generative AI like that of ChatGPT, driving renewed demand for data centers not seen in decades. This surging demand pushed tech investors to put $39.6 billion into data center development in 2024, which is 12 times the amount invested back in 2016.
A recent development, however, has stirred things up, especially the concept that billions of dollars needed to be spent for AI advancement. Developed by a Chinese AI research lab, an open-source large language model named DeepSeek was released and performed on par with OpenAI, but it reportedly operates for just a fraction of the cost of Western AI models. OpenAI, however, is investigating if DeepSeek utilized distillation of the former’s AI models to develop their systems.
Copyright: cottonbro studio
What Is “Distillation?”
According to Labelbox, model distillation (or knowledge distillation) is a machine learning technique involving the transfer of knowledge from a large model to a smaller one. Distillation bridges the gap between computational demand and the cost for training large models while maintaining performance. Basically, the large model learns from an enormous amount of raw data for a number of months and a huge sum of money typically in a training lab, then passes on that knowledge to its smaller counterpart primed for real-world application and production for less time and money.
Distillation has been around for some time and has been used by AI developers, but not to the same degree of success as DeepSeek. The Chinese AI developer had said that aside from their own models, they also distilled from open-source AIs released by Meta Platforms and Alibaba.
However, the terms of service for OpenAI prohibits the use of its models for developing competing applications. While OpenAI had banned suspected accounts for distillation during its investigation, US President Donald Trump’s AI czar David Sacks had called out DeepSeek for distilling from OpenAI models. Sacks added that US AI companies should take measures to protect their models or make it difficult for their models to be distilled.
Copyright: Darlene Anderson
How Does Distillation Affect AI Investments?
On the back of DeepSeek’s success, distillation might give tech giants cause to reexamine their business models and investors to question the amount of dollars they put into AI advancements. Is it worth it to be a pioneer or industry leader when the same efforts can be replicated by smaller rivals at less cost? Can an advantage still exist for tech companies that ask for huge investments to blaze a trail when others are too quick to follow and build upon the leader’s achievements?
A recent Wall Street Journal article notes that tech executives expect distillation to produce more high-quality models. The same article mentions Anthropic CEO Dario Amodei blogging that DeepSeek’s R1 model “is not a unique breakthrough or something that fundamentally changes the economics” of advanced AI systems. This is an expected development as the costs for AI operations continue to fall and models move towards being more open-source.
Perhaps that’s where the advantage for tech leaders and investors lies: the opportunity to break new ground and the understanding that you’re seeking answers from unexplored spaces while the rest limit themselves and reiterate within the same technological confines. Established tech giants continue to enjoy the prestige of their AI models being more widely used in Silicon Valley — despite DeepSeek’s economical advantage — and the expectation of being the first to bring new advancements and developments to the digital world.
And maybe, just maybe, in that space between the pursuit of new AI breakthroughs and lower-cost AI models lie solutions to help meet the increasing demand for data centers and computing power.
Copyright: panumas nikhomkhai
Featured Image Copyright: Matheus Bertelli
Top Image Copyright: Airam Dato-on
Aug
The Future Is Here
“Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”
With just 22 words, we are ushered into a future once heralded in science fiction movies and literature of the past, a future our collective consciousness anticipated but has now taken us by surprise upon the realization of our unreadiness. It is a future where machines are intelligent enough to replicate a growing number of significant and specialized tasks. A future where machines are intelligent enough to not only threaten to replace the human workforce but humanity itself.
Published by the San Francisco-based Center for AI Safety, this 22-word statement was co-signed by leading tech figures such as Google DeepMind CEO Demis Hassabis and OpenAI CEO Sam Altman. Both have also expressed calls for caution before, joining the ranks of other tech specialists and executives like Elon Musk and Steve Wozniak.
Earlier in the year, Musk, Wozniak, and other tech leaders and experts endorsed an open letter proposing a six-month halt on AI research and development. The suggested pause is presumed to allow for time to determine and implement AI safety standards and protocols.
Max Tegmark, physicist and AI researcher at the Massachusetts Institute of Technology and co-founder of the Future of Life Institute, once held an optimistic view of the possibilities granted by AI but has now recently issued a warning. He remarked that humanity is failing this new technology’s challenge by rushing into the development and release of AI systems that aren’t fully understood or completely regulated.
Henry Kissinger himself co-wrote a book on the topic. In The Age of AI, Kissinger warned us about AI eventually becoming capable of making conclusions and decisions no human is able to consider or understand. This is a notion made more unsettling when taken into the context of everyday life and warfare.
Working With AI
We at Cascade Strategies wholeheartedly agree with this now emerging consensus and additionally, we believe that we’ve been obedient in upholding the responsible and conscientious use of AI. Not only have we long been advocating for the “Appropriate Use” of AI, but we’ve also made it a hallmark of how we find solutions for our client’s needs with market research and brand management.
Just consider the work we’ve done with the Expedia Group. For years, they’ve utilized a segmentation model to engage with their lodging partners by offering advice that could lead to the partner winning a booking over a competitor. AI filters through the thousands of possible recommendations to arrive at a shortlist of the best selections optimized for revenue.
With the continued growth and diversification of their partners, they then needed a more effective approach in engaging and appealing to them, something that focuses more on that associate’s behavior and motivations. We came up with two things for Expedia: a psychographic segmentation formed into subgroups based on patterns of thinking, feeling, and perceiving to explain and predict behavior, and more importantly, a Scenario Analyzer that utilizes the underlying AI model but now delivers recommendations in very action-oriented and compelling messaging tailor-fit for that specific partner.
The best part about the Scenario Analyzer is whether the partner follows any of the advice recommended or does nothing, Expedia still stands to make a profit while maintaining an image of personalized attentiveness to their partner’s needs. And ultimately, it’s the partner who gets to decide, not the AI.
Copyright Tara Winstead
Our Future With AI
This is how we view and approach AI- it’s not the end-all, be-all solution but rather an essential tool in increasing productivity and efficiency in tandem with excellent human thinking, judgment, and creativity. Yes, it is going to be part of our future but in line with the new consensus, we believe that AI shaped by human values and experience is the way to go with this emerging and exciting technology.
info@cascadestrategies.com-
2032 Newport Way NW,
Issaquah, WA 98027 - (425) 677-7430
