Scaling Foundational AI Models Will Follow 2 Directions

Thank you for subscribing and for all the support this week. I appreciate everyone who took the time to comment and engage with the articles. Please keep that coming so I know what's popular and what isn't.

Subscribers can access office hours, and I am adding a new time. Currently, there is a session on Fridays at 9 am PT, and next week, I will be running another hour on Mondays at either 6 am or 6 pm PT. Let me know if you have a preference. So far, the feedback has favored the 6 am PT time.

I have 2 spots reserved for subscribers in the next cohort of my Data and AI Product Management Certification. You can learn more about the course here and enroll here if it meets your needs.

As I work to scale this newsletter, there are many parallels to scaling generative AI. It's essential for everyone to realize that generative AI is just the first wave. It solves one problem, but there are many more behind it.

Solving The First Problem, But Not The Last

LLMs help machines to communicate and interact with people better than any technology that's come before. We can collaborate with any machine running an LLM interface in the same way we do with each other. The LLM lowers the learning curve for everything from programming to working with complex application suites like SAP, Microsoft, or AWS.

Multimodal models will extend that interaction to include images, audio, and video. Machines will finally become conversational across media formats. We can collaborate on projects that include any media type. Coke recently launched an advertising campaign with help from OpenAI and Bane.

The first iteration was released in March. The company opened a platform for creatives that allowed people to develop art with Coke's library of digital assets. The second iteration came in the form of a commercial developed with Stable Diffusion.

Coke's campaigns targeted the creative process and emphasized how foundational models can help people be more creative. They wanted to highlight how these models can support our uniquely human ability to create engaging experiences. The generative models broke down the technical barriers to creating digital art and experiences.

SAP released an ad campaign of its own with a different creative angle. They fed the day's most popular headlines into a generative model. An illustrator worked with the generative model's output to deliver a daily image for two weeks. The image was a recommendation and a launching point for the illustrator to build on.

SAP ran the ad campaign on billboards in some of the US's most public locations (New York, Atlanta, and LA). Both campaigns are examples of building products with generative models in public and showing off pragmatic applications. In SAP's case, that was more intentional, whereas Coke's objectives were entirely creative and community-building.

SAP's goal was to showcase how quickly an advertising campaign image could be turned around with generative models and people collaborating. What used to take a week or more was getting done in a few hours. SAP showed how models allow people to iterate faster with the model than they could before. It's a tangible example of the productivity leaps that are possible by leveraging the human-machine collaboration paradigm.

The End Of The Beginning

Generative AI is already peaking, and that's an excellent place for the technology to be right now. Businesses want to build on LLMs and need stability. There will be incremental improvements over the next 24 months and a few major breakthroughs. However, today data scientists don't know which problems to solve.

We won't know where the models fall short until people and businesses use products vs. early prototypes. Progress is happening so fast and in so many different directions because no problems have risen to the top. We need to see how these models work in the real world to prioritize issues and learn which are the most important or valuable to focus on.

The hard work is these incremental improvements. It's not as glamorous as the splashy announcements of early releases, but this is where value is created. Incremental problem-solving and innovation support productization and commercialization. This is where novel use cases, products, operating models, and business models will be discovered.

What comes next will follow a cycle. Model architectures are easy to propose and validate. Most researchers don't have access to the data necessary to build anything tangible. That leads to the first problem, data. Once the model is published, businesses and applied researchers curate data sets to evaluate the model further.

This leads to another wave of more pragmatic publications either supporting or refuting the model as a significant advancement. Applied data scientists pick up on these publications and build early prototypes. These experiments usually result in dead ends. Models can be impressive but infeasible to build with, integrate into a product, and support.

Another iteration of model improvements and optimizations kicks off at this point. Models usually slim down both in computing and data needs. These improvements often result in a model that meets real-world demands, and the unit economics work for some use cases.

Once products are shown to be feasible, more researchers, enthusiasts, and businesses get involved. Iterations move even faster, and generative AI is nearing the end of this cycle. The early improvements address low-hanging fruit. They don't cost very much and deliver massive leaps forward. There isn't much left in that category, so the research community will move on to the next promising area.

Scaling In Two Directions

Applied data scientists are working to scale the models in two directions. The first is horizontal scaling. Horizontal scaling benefits from greater generalization. As the models improve horizontally, they become more capable across a broad range of use cases. Horizontal improvements aim to make the models more accurate for all their tasks.

Horizontal capabilities are like an LLM baseline functionality. Gains here are what early researchers focused on. This crop of LLMs has hit the upper bounds in horizontal capabilities. Data is their main avenue for new horizontal improvements. More data or higher quality data are required to deliver improvements in horizontal reliability and functionality.

SAP and Coke are both beneficiaries of horizontal scaling. Their ad campaigns are built on out-of-the-box functionality. LLMs are powerful enough to support these use cases with existing generalized capabilities. Both companies are proving that LLMs are ready to productize and deliver value with.

The second direction is vertical scaling. LLMs have notorious reliability issues, and there are limits on how well they generalize to support use cases. Horizontal scaling can only take us so far. We must scale the models for narrow use cases to support higher reliability requirements.

This is a more targeted process where progress is measured against a smaller set of functionality or utility. Applied data scientists working in business settings make most of the advances in vertical scaling. The incremental improvements are slower and less glamorous but much more lucrative.

Rather than focusing on improving the model, data is the main driver for functional and reliability gains. IBM's watsonx platform is designed to enable businesses to scale foundational models vertically. It provides access to Hugging Face's ecosystem and IBM's bedrock models. Watsonx facilitates the process of retraining existing foundational models with the business's data. Instead of selling models, IBM is selling tools to support vertical scaling.

Azure and AWS have made models available so customers can run stand-alone instances on both cloud platforms. They don't have the same suite of supporting utilities built to facilitate model retraining, but that's probably coming very soon. Bane, IBM, and SAP have launched consulting arms that provide services to support vertical scaling and implementation.

Platform and services providers have realized that most companies lack the talent and infrastructure to leverage foundational models. There is a booming market for products and services that fill the gaps. Time to market and monetization are on every CEO's mind, so demand for support with vertical scaling will likely grow.

Data Defines Near-Term Advantage

Data is an advantage that will endure over the next 12-18 months. Horizontal scaling is only an advantage, while some companies struggle to integrate LLMs into their products and operations. Services and platform providers are rushing in to level that playing field. IBM, Microsoft, SAP, and AWS already offer products. Salesforce and many others are a month or two away from entering the arena. This space will soon be as overcrowded as today's data engineering and MLOps landscape.

Vertical scaling will provide more opportunities for businesses to differentiate their products and streamline operations. Proprietary data enables both monetization routes. Advantages are built on what competitors cannot easily copy. Companies must move beyond what LLMs can do horizontally and develop vertical capabilities.

Most businesses struggle to understand what data they currently have and share that view across the business. Few business units have access to the data they need when they need it. Generative AI will force firms to confront these challenges, or they will fail to see the benefits.

If you want to learn more about monetizing LLMs, check out my self-paced course.