I'm on stage with Jonathan Ross, co-host of the "All In" podcast, and legendary tech entrepreneur and investor, Jonathan Shieber. We're here to talk about Jonathan's unique origin story and the company he founded, Groq.
Jonathan is known for having one of the most unique founder stories in Silicon Valley. He's the only high school dropout to have started a billion dollar company. He dropped out of high school and started taking classes at a university on the side. He eventually fell under the wing of a professor and transferred to NYU, where he started taking PhD courses as an undergrad. However, he dropped out of NYU and ended up at Google.
At Google, Jonathan worked on building giant test systems for ads and was part of the team that built Google's custom silicon, the TPU. The TPU was built as a side project and was funded out of what a VP referred to as his "slush fund." It was never expected to be successful, but it became one of the three projects that actually won.
Jonathan left Google to join the Google X team, but he eventually left to start Groq. He was looking to take something from concept to production and saw an opportunity to build something in the AI space.
At Groq, Jonathan and his team have focused on building scaled inference for machine learning models. They've built a massive matrix multiplication engine and have created a kernel-free approach to programming their chips. This sets them apart from companies like Nvidia, who have a kernel-based approach.
Nvidia is known for their software and ecosystem, but they are not a software-first company. They have a very expensive approach and have locked up the market on certain components, such as high bandwidth memory (HBM) and interposers. Groq, on the other hand, has used older technology and interconnect to build their chips, which has allowed them to be more cost-effective.
Groq's chips are typically 5 to 10x faster than Nvidia's chips and are about 1/10th the cost. They are also more power-efficient, which is important as power is limited at the moment.
Jonathan's goal with Groq is to provide a low-cost alternative to companies building AI applications. He wants to make sure that startups are not shipping all of their money back to Nvidia or other big tech companies. Instead, he wants to provide a cost-effective solution that will allow startups to build companies from 1/10th or one 100th of the cost.
In conclusion, Jonathan's unique origin story and his work at Groq are emblematic of the entrepreneurial spirit that drives innovation in Silicon Valley. By focusing on building scaled inference for machine learning models, Groq is providing a cost-effective solution that will allow startups to build AI applications without breaking the bank.