Elon Musk's supercomputer, The Memphis Supercluster, has gone live, prompting the billionaire to claim its deployment was the fastest ever recorded.
The cluster was built by xAI, Musk's AI firm, and consists of 100,000 liquid-cooled H100 graphics processing units (GPUs) running on a single RDMA fabric. Musk described it as "the most powerful AI training cluster in the world."
"We were able to install and bring online a massive new training centre... in Memphis. And from getting hardware to beginning training, it was only nineteen days," said Musk in an interview with Jordan Peterson that aired on X last night.
"That's the fastest by far anyone's been able to do that," he claimed.
The project comes at a significant cost. Each H100 GPU is estimated to cost between $30,000 and $40,000, meaning it could have taken an investment of between $3 and $4 billion.
Upon launch, the cluster was touted as "the largest capital investment by a new-to-market company in Memphis history."
Elon Musk says xAI's new supercomputer in Memphis was installed in just 19 days and will be used to train Grok 3, which is expected by December, and will be the most powerful AI in the world pic.twitter.com/b1byBVZmcy
— Tsarathustra (@tsarnick) July 22, 2024
And while the speed at which xAI's Memphis supercomputer was stood up appears impressive in its own right - other supercomputers appear to have been deployed more quickly.
In 2022, the Texas-based startup Cerebras was able to get its own AI supercomputing up and running in three days.
Andromeda, an Exascale supercomputer, is composed of 16 CS-2 servers, with 13.5 million AI cores, and is driven by 284 Gen 3 EPYC processors.
In comparison, the Memphis supercomputer's claim to fame is the use of RDMA (Remote Direct Memory Access) network fabric.
Musk's cluster will be used to train Grok, a large language model that was developed as a response to the rise of ChatGPT.
Musk was a co-founder of ChatGPT's parent company OpenAI, and has admitted to "being a huge idiot" for having let go of much of his stake in the company.
Through the Memphis supercomputing cluster, Musk has claimed xAI is close to catching up with the big league companies that have been around for longer. In his estimation - Grok will be "the world's most powerful AI by every metric by December this year."
Whether the actual specifics keep up with these claims remains to be seen. We have asked for clarification and will update this story if we find out more about Musk's allegedly record-breaking feat.
