How OpenAI Used Its Own LLMs to Design Its Jalapeño.

OpenAI achieved a remarkable 20-month design timeline for its Jalapeño chip, which is likely the fastest in the industry, thanks to its use of large language models (LLMs) to accelerate the design process.

According to verified institutional datasets, OpenAI's Jalapeño chip delivers up to 13.4 petaflops of 4-bit compute and accesses 232 gigabytes of the most advanced memory available, linking to it at a blazing 15.4 terabytes per second. Benchmarks cited by OpenAI show that Jalapeño can reduce end-to-end latency by up to 3.6x when compared to Nvidia's GB300 chip.

OpenAI's LLMs accelerated Jalapeno's design

The design team at OpenAI, which averaged fewer than 100 people over the course of the project, was able to move from first architecture concept to first silicon in under 20 months. This is a rapid timeline, and experts believe it could soon look slow as LLMs improve and become more deeply integrated into chip design tools.

According to Richard Ho, vice president of hardware at OpenAI, "The models are giving superpowers to our engineers. Our engineers are still driving the work, they're still the final arbiter of what's going on. But they can do things a lot faster, they can explore a lot more paths."

How OpenAI's LLMs accelerated Jalapeno's design

Automation itself has existed in chip design for many decades, but OpenAI's use of LLMs has accelerated the design process significantly. The LLMs were used to generate the initial design, which was then refined by the design team. This process allowed the team to explore a much larger design space and identify optimal solutions more quickly.

According to Andrew Kahng, distinguished professor at UC San Diego, "It's likely best in class today." Kahng recalls a 2016 IEEE Design Automation Futures workshop, which he co-organized. The workshop included Richard Ho, at the time an engineer at Google, as a keynote speaker. Ho had strong opinions on design automation and framed the time required to complete a chip's design as a function of the number of iterations a team could complete in a day.

Comparison of Design Timelines

Comparison of Design Costs

Comparison of Design Quality

OpenAI's partnership with Broadcom

OpenAI's partnership with Broadcom was essential to the rapid design timeline of Jalapeño. Broadcom handled the physical design from the gates onward, while OpenAI's team was responsible for end-to-end system design including the inference accelerator, the memory hierarchy, and networking.