Cerebras: an Engineering Marvel to Rival NVIDIA

It's a compelling story about how a successful exit can empower an innovative itch. Months of brainstorming led to the ambitious decision to tackle a seemingly insurmountable challenge, ultimately resulting in a groundbreaking product. Cerebras Systems not only dreams big but also acts big – their AI chip is so large it could be compared to a dinner plate or a pizza box, making it the largest single piece of silicon ever produced. And it works.

Cerebras Systems, an eight-year-old company, recently introduced the third version of their wafer-scale engine (WSE-3), a massive 5nm-based chip boasting 4 trillion transistors and 900,000 AI-optimized compute cores, which powers the CS-3 AI supercomputer. Last week, they also announced a collaboration with Dell Technologies to address the growing AI workload demands.

We will discuss what it all means, why they are less known than NVIDIA despite continuously claiming to outpace NVIDIA's chips, and how their valuation recently reached over $4 billion (with the next stop being an IPO?) in our AI Infra Unicorn series.

In today’s episode:

1. Starting point of Cerebras Systems: daunting challenge and metaphors from Andrew Feldman

2. Becoming a unicorn - financial situation

3. But what exactly does Cerebras offer?

4. Mission

5. Training capabilities and inference challenges

6. Cerebras vs. NVIDIA: another analogy and key differences

7. Can Cerebras’s chips Replace NVIDIA GPUs?

8. How does the company make money?

9. Conclusion

Starting point of Cerebras: daunting challenge and metaphors from Andrew Feldman

It was a tremendous success. In five years, Andrew Feldman and Gary Lauterbach built SeaMicro, a novel power-efficient computer server for data processing, and sold it to AMD for $334 million. Serving on the board for another two years, in 2014 they finally decided to get some rest and quit. But once an entrepreneur, always an entrepreneur, especially when tremendous talent still floats around you, ready to follow. Feldman and Lauterbach stayed in touch with three other colleagues from SeaMicro: Michael James, J.P. Fricker, and Sean Lie, and gradually started to brainstorm, each bringing unique expertise in software, hardware, and systems architecture. All five of them shared the ambition to create something big, not just another incremental improvement in the tech world.

The idea of building a new type of server, optimized for Intel's groundbreaking 3D XPoint memory, initially captivated them. This technology promised to transform computing with its unprecedented speed and durability. However, the team quickly realized the limitations imposed by Intel's dominance over the technology. They shifted their focus to an even bolder vision: creating a computer optimized for artificial intelligence.

Feldman envisioned a machine solely dedicated to AI tasks, eschewing all other functionalities. This concept involved constructing a wafer-scale chip, a colossal 60 times larger than any existing chip, with unparalleled compute power.

So when they said they wanted something big, they meant it literally. But it was a daunting challenge, reminiscent of the failed efforts of Trilogy Systems decades earlier, who were also trying to build a wafer-scale systems. And, because it was a daunting challenge, it became so inspiring: solving such a complex problem would grant them a unique market advantage with few competitors.

Thanks to their phenomenal exit and network built by Feldman, they could raise money solely for an idea. How? Andrew Feldman loves analogies and metaphors. In an interview with Mark Leslie, he compared specialists and generalists to cheetahs and hyenas (WSE-GPU). In Spectrum, he compares GPUs to tailors who can’t make one suit together. In The New Yorker, he says, “We invented a technique such that you could communicate across that little bit of cookie dough between the two cookies.” In TechCrunch, he likens the challenge to climbing Mount Everest: “It’s like the first set of guys failed to climb Mount Everest, they said, ‘Shit, that first part is really hard.’ And then the next set came along and said ‘That shit was nothing. That last hundred yards, that’s a problem.’” He is the closest thing we have encountered to ChatGPT in terms of producing analogies and metaphors on the fly, and we bet this style works phenomenally on investors. His founding team was very solid, his metaphors were embracing, and investors fell for it, not fully realizing what a complicated story they were signing up for.

In less than a week of conversations to test the level of interest from potential investors, Feldman had received over $100 million worth of commitments. In March 2016, Cerebras was launched, with Andrew Feldman, Gary Lauterbach, Michael James, J.P. Fricker, and Sean Lie as co-founders.

Becoming a unicorn – financial situation

Thank you for reading, please feel free to share with your friends and colleagues. 🤍