Investing on the edge
Back to Top

Gravity Fund

The BerriAI Investment Thesis

The label “large” refers to the number of values (parameters) the model can change autonomously as it learns. Some of the most successful LLMs have hundreds of billions of parameters. LLMs are trained with immense amounts of data and use self-supervised learning to predict the next word in a sentence, given the surrounding context. The process is repeated over and over until the model reaches an acceptable level of accuracy.

Modern LLMs emerged in 2017 and use transformer neural networks, commonly referred to as transformers. With a large number of parameters and the transformer model, LLMs are able to understand and generate accurate responses rapidly, which makes the AI technology broadly applicable across many different domains.

A key recent advancement has been combining specialized AI hardware, scalable-friendly architectures, frameworks, customizable models and automation with robust “AI-first” infrastructures. That’s making it feasible to deploy and scale production-ready LLMs within a wide range of mainstream commercial and enterprise-grade applications on public and private clouds and via APIs.

Once an LLM has been trained and fine-tuned, it can perform a wide range of NLP tasks, including:

  • Building conversational chatbots like ChatGPT.
  • Generating text for product descriptions, blog posts and articles.
  • Answering frequently asked questions (FAQs) and routing customer inquiries.
  • Analyzing customer feedback from email, social media posts and product reviews.
  • Translating business content into different languages.
  • Classifying and categorizing large amounts of text data for more efficient processing and analysis.

BerriAI is a Y Combinator-backed company that lets users build production-ready ChatGPT apps in under 2 minutes by easily connecting user data to an LLM. The BerriAI API allows businesses to programmatically spin-up custom ChatGPT instances for millions of users on the individual level. This is a powerful toolkit that allows developers to easily add AI to their applications.

The main problems that BerriAI addresses are:

  • Users often ask questions about their specific data, such as “What do my lab results say?” or “What is my order status?” ChatGPT can not provide a specific answer to these questions.
  • Creating a fleet of customized ChatGPT apps is challenging. After building one app, you must manually create individual apps for all users. This involves going into Google Collab, ingesting the user’s data, chunking it to parse with an LLM, embedding the data, storing the embedding, keeping the embedding in sync, searching through the embeddings for each question, and finally deploying the application.
  • Scaling this process to all users is overwhelming. You need to monitor question and answers for each application and improve performance by fine-tuning based on each user’s preferences.

LLMs need to be connected to outside systems to bring in dynamic data and take actions on behalf of the user. BerriAI addresses this challenge through prompt chaining. Prompt chaining gives LLMs the ability to load data from a wide variety of sources, formulate and send additional prompts to themselves or other LLMs, and even use external APIs. By extending LLM capabilities through prompt chaining, these orchestration tools enable much more dynamic and personalized applications.

Langchain has taken the early lead as an open-source framework for programming with LLMs, giving them access to other tools, and chaining prompts together. Langchain is not designed for ease of use, and is difficult to utilize for spinning up mass application batches. Other more structured, scalable, and guided development experiences for working with LLMs are on offer from companies like BerriAI, Dust, re:tune and Fixie. These tools are particularly important because they are effectively a scalable liaison between LLMs and the rest of software — multiplying the powers of each.

While others (such as those listed in the previous paragraph) in the sector are trying to connect personalized user data to foundational LLMs, we believe BerriAI has built a no-code/low-code platform that offers developers an ease-of-use value proposition that none of the other players offer. Additionally, BerriAI is rapidly expanding its DevTools offering beyond promp chaining. Offering an easy-to-use platform where developers can build, deploy, scale and monitor their LLM apps will be key. This is BerriAI’s vision.

Not only is building, scaling and deploying custom LLM apps to millions of end users difficult and time consuming, it’s also extremely expensive. The expense associated with fine-tuning millions of customized LLM apps is prohibitive for developers attempting to deploy directly by calling the LLM endpoint. BerriAI’s chunking and search strategies lead to efficiencies in how many calls they need to make on the underlying LLM endpoint, which allows them to offer free fine-tuning for hobbyist and hacker users, and a capped free fine-tuning package for enterprise users. Thus, BerriAI offers developers a compelling, low hurdle value prop in terms of both cost and time/effort. We believe that such a value prop will be invaluable in terms of capturing market share in a rapidly expanding market.

Products that leverage LLMs—whether internally built or through an API with passed along costs—typically have higher marginal costs and latency than traditional software. This has implications for how developers think about feature tradeoffs, scalability, and business model. For LLMs to evolve from prompt-based automations for high-cost human work to always-on elements of a new generation of software, cost and latency will need to come down through continued optimization and innovation across the LLM stack. This is an area where BerriAI really shines.

Since launching its self-serve go to market model six weeks prior to YC Demo Day, the Company has attracted 20+ paying customers and 4000+ developers (these numbers are now out of date!) using BerriAI to deploy more than 5,000 application instances. BerriAI offers an enterprise plan priced at $999 per month plus usage fees. As of YC Demo Day, this model had scaled to $2.5K MRR and BerriAI’s largest customer had scaled its usage to $1.5K MRR. In addition, BerriAI had three additional enterprise trials underway that the team expects to convert to enterprise monthly plans imminently.

For enterprises, LLMs offer the promise of boosting AI adoption hindered by a shortage of workers to build models. With BerriAI, foundational LLMs can be easily leveraged by organizations without AI expertise — a huge plus. Estimates say that unstructured data makes up more than 80% of the enterprise data and is growing at the rate of 55% per year. In order for companies to truly get insights from unstructured data, it is important to develop the required technology.

Within the infra layer, there are few areas that we find more interesting than LLM agnostic DevTools, no/low code, fine-tuning, prompt chaining, retrieval, actions, and experimentation frameworks. Creating a reliable and adaptable layer of developer tools will help enterprises unlock the power and value of LLMs for more users and applications. BerriAI fits under this thesis perfectly.

THE MARKET

Analysts say that the NLP market is rapidly growing from $11B (2020) to $35B+ (2026). A recent survey from venture fund Pitango First and Intel’s acceleration program, Intel Ignite, shined a light on some of the ways that CTOs and VPs of R&D will adopt Generative AI and LLMs in their work. More than 4 in 5 (85%) of respondents are already deploying, or plan to deploy, GenAI or LLM technologies either as part of their company offering or internally to help their business. The survey, conducted earlier this month and involving 48 VPs of R&D and CTOs from different industries, also revealed that almost two-thirds (60%) of respondents have found novel uses for LLMs to solve their “hard problems.” More than half of CTOs think that LLMs will become part of providing services to their users and that LLM-based technologies will be part of their products, but the majority of them (58%) have not yet tested tools utilizing LLMs provided by small startups.

We increasingly see an expanding universe of foundational LLM models developed and maintained by both large companies and open source projects. We believe a handful of these models will rise to the top, and others are destined to consume and build their products and business on top of those models. In many ways, foundational LLMs resemble the cloud revolution where infrastructure was commoditized by a few large companies and allowed for novel hyperscale applications to be built upon that infrastructure. Developer tooling that sits between foundational LLM infrastructure and enterprise developers represents the key to unlocking a significant portion of the LLM market opportunity.

While the data relative to enterprise adoption of LLMs is encouraging, it lags behind raw consumer adoption. Apps like ChatGPT have significantly outpaced the most successful hyperscale Web2 applications in terms of number of days to reach 100M users, and the trend shows no signs of slowing. Consumers are adopting LLMs at an incredible pace. Increasing usage and familiarity with LLM interfaces will lead to consumer demand for LLM functionality in other applications and products.

We expect significant consumer demand pull to result in demand for AI DevTools from enterprises that don’t have in-house AI native developer talent. LLMs have weird APIs — you provide natural language in the form of a “prompt” and get a probabilistic response back. It turns out that mastering this API requires a lot of tinkering and experimentation – to solve a new task, you’ll probably need to try a lot of different prompts (or chains of prompts for custom apps) to get the answer you’re looking for. Simply getting comfortable with the probabilistic nature of LLM output takes time — developers need to do extensive testing to understand the boundary cases of their prompts. At its core, BerriAI’s suite of products solves this problem by significantly reducing the learning curve requirement.

One good way to gain a lot of traction is to be the first tool someone needs to adopt an important new technology! Nothing beats selling picks shovels during a gold rush. We expect that tools such as BerriAI will play a critical role in the LLM ecosystem– inspiring amazing new use cases and features.

THE COMPETITIVE ENVIRONMENT

Enterprise adoption of the last wave of machine learning was somewhat slow. As a result, the market for ML DevTools remains immature to this day. Often, this was because of how much complexity there is before getting out V1 of an ML feature. A lot of that complexity couldn’t be automated away.

Building with LLM APIs is fundamentally different than building on previous generations of AI/ML. Nearly any developer can build LLM features, and they don’t need to capture a ton of data first. As a result, we expect that this market will mature very quickly. Anywhere developers go, DevTools reliably follow. The same buzz surrounding prompting frameworks will quickly grow around downstream tools like labeling, fine-tuning, model management, monitoring, testing, and more. There is an incredible opportunity to build DevTools right now! BerriAI sits at the right place at the right moment in time and is continuously expanding its platform to address multiple developer workflows.

With that said, we are still in the early innings of the current AI development and adoption cycle. While the current challenges facing developers are fairly well defined and the potential solutions are coming into focus, there are many outstanding questions that will undoubtedly shape the ecosystem in the future. One of the most frequently asked questions about the generative AI space is, “Where will the value accrue?” A big input to answering this is how and where competition forms across the value chain for AI products.

In many markets, solutions to hard problems come packaged as part of a single integrated solution rather than as a set of modular ones. In terms of building integrated solutions to the problems facing enterprise developers, foundational model companies like OpenAI are strong candidates and a potential threat to modular solutions like BerriAI.

The OpenAI API plugin launch included over a dozen plugins from Klarna, Zapier, Instacart, and others. Many hailed this as OpenAI’s app store moment. However, within a few days Langchain showed how any language model could use these plugins, and created further buzz by announcing an integration with Zapier’s Natural Language API. This example speaks to the inherent interoperability of tools designed for use by one foundational LLM with any other LLM. As long as the interaction between models and other systems or workflows occurs via natural language, lock-in will be difficult to build. While this doesn’t make it impossible for someone like OpenAI to build integrated closed solutions, it does mean that closed model players will have a harder time building differentiated solutions while their models remain accessible via API. Restricting API access will be difficult, as this is the primary monetization model for the foundational LLMs.

Thus, as it stands today, it appears a wide array of competition will continue to exist at the foundational model layer in the form of open source LLMs that are strong on general language tasks, pre-trained on use case specific data, and that can be run locally. This should allow for a flourishing modular developer tooling environment to emerge atop the open source foundational LLM infrastructure.

Developer tooling competition will be fierce. The foundational LLM infrastructure is in place and developers need tools to build on that infrastructure. Consumer demand for LLM applications is vast. Currently, developer tooling is the bottleneck, and as such, the opportunity. The above market map is from Base10, an AI venture fund. It’s a few weeks old, so they have BerriAI grouped under the ‘Orchestration’ cluster of tools, but BerriAI has since expanded their platform and now bridges into the ‘Deployment, Scalability & Pre-Training’, ‘Content & Embeddings’ and ‘QA & Observability’ clusters as well.

At Gravity, we believe that once you define your thesis – in this case the need for LLM developer tooling – it’s a matter of finding the best-in-class team building the best-in-class tech under that thesis. We believe that the BerriAI team is one of the best teams operating in the sector (given their backgrounds building AI developer tools at Coinbase and Microsoft), and we are now beginning to see the BerriAI tech really differentiate itself in an otherwise crowded marketplace. An easy to use low-code / no-code DevTooling platform solving multiple developer needs without needing to stack modular applications should be a winning value proposition in this market.

We couldn’t be more proud to be backing the BerriAI team!