Recently, OpenAI and Anthropic have been on the offensive against thirdparty attempts to train new AI models by using their publicly available chatbots and APIs, a process called “distillation.
In that discussion, Chinese companies have employed distillation to produce openweight models that are almost as good as those made in the United States but considerably less expensive.Tech workers, however, often believe that American labs employ these strategies against one another in order to stay ahead of rivals.
We now know that it is accurate in at least one instance: when Elon Musk was questioned on the witness in a federal court in California on Thursday about whether xAI has trained Grok using distillation techniques on OpenAI models, he said that this is a common practice among AI firms.When asked if that meant “yes,” he said, “Partly.
Musk is currently suing Greg Brockman, CEO Sam Altman, and OpenAI, claiming that by converting the company to a forprofit structure, they violated OpenAI’s initial nonprofit goal. The tech leader testified in the trial, which started this week.
Musk’s acknowledgment is noteworthy because distillation poses a threat to AI behemoths by eroding the edge they have established through computational infrastructure investments. This makes it possible for other software developers to produce models that are almost as powerful at a lower cost. Given that Frontier Labs is allegedly violating copyright laws in order to obtain enough data to train their models, there is a great deal of irony in this situation.
It should come as no surprise that Musk’s xAI, which was founded in 2023, years after OpenAI, would attempt to learn from the industry leader at the time. Although it is unclear if distillation is specifically prohibited, it might go against the terms of service that businesses provide to their customers.
According to reports, OpenAI, Anthropic, and Google have started a project through the Frontier Model Forum to exchange knowledge about how to thwart Chinese distillation attempts.These usually entail methodically querying models to comprehend their internal mechanisms.Frontier Labs is attempting to stop users from submitting dubious mass inquiries in an effort to halt the operations.
A request for comment on Musk’s admission was not answered by OpenAI at the time of publication.
Musk was questioned later in his testimony regarding his assertion from the previous summer that xAI would soon surpass all companies except Google. In response, he listed the top AI providers in the world, with Anthropic coming in first, followed by OpenAI, Google, and Chinese open source models. He described xAI as a significantly smaller business, employing only a few hundred people.
Leave feedback about this