Redpanda raises $100M as streaming data demand grows

Redpanda is seeing major growth for its streaming data platform, fueled in part by the continuing need for real-time data and new use cases in artificial intelligence (AI).

Today the company announced it has raised $100 million in a series C funding round, bringing total funding for the San Francisco-based startup to $165 million. The new funding represents a big uplift from the $50 million series B Redpanda secured in February 2022.

Redpanda’s streaming data platform allows companies to collect, store and analyze data in real time. A key part of the company’s growth and success is how it is enabling organizations across different industry verticals to benefit from streaming data. In particular, Redpanda is finding strong use by with artificial intelligence (AI) and machine learning (ML) vendors, including generative AI image generation service Midjourney.

>>Follow VentureBeat’s ongoing generative AI coverage<<

Redpanda is compatible with the popular open-source Apache Kafka streaming technology. The company claims it provides better performance and easier management for real-time use cases than competitive alternatives.

“From a market perspective we basically quintupled revenue last year, and we started winning every deal,” Alex Gallego, founder and CEO of Redpanda, told VentureBeat. “So the idea here is to fuel the growth, we are continuing the investment in what we see as the future of streaming.” 

How Redpanda is enabling real-time AI/ML workloads 

Enabling real-time AI/ML data pipelines is a key use case for Redpanda.

Gallego said that there is a common pattern across the different AI/ML companies Redpanda works with. When a user interacts with a generative AI service like Midjourney, they input a text prompt. The prompt in turn accesses an API at the specific service, which then generates a result. As part of the process, all the user prompts are recorded in a data storage system. Increasingly that system is a real-time streaming system like Redpanda.

The streamed prompts are used to help enable real-time inference, as well as to help with additional training.

“Redpanda has been used in a bunch of AI/ML use cases [because it’s] a super-scalable storage engine for real-time ML,” Gallego said. “That’s really where Redpanda fits, as there just aren’t many storage engines that can keep up with the volumes of data that these models are demanding in real time.”

Apache Iceberg support is coming and it’s a big deal

Looking ahead, one area where Redpanda is working to provide expanded capabilities is with support for Apache Iceberg.

Iceberg is an open-source data lake table format that has become increasingly popular in recent years, gaining the support of vendors including Cloudera, Snowflake and Google.

Gallego explained that users connect to Redpanda via a Kafka API, because that is how people consume and produce data. Going a step further, Redpanda has a tiered storage format so users can keep data locally as well as upload data to a data lake. Redpanda is now working on enabling Iceberg support so the data loaded into the data lake will be formatted in a way that can be easily accessed via SQL query engines.

“We effectively make the data lake real-time, and the format that we upload to the data lake will be Apache Iceberg, so that a user can go from a stream to SQL in seconds,” he said.