Stability AI Releases Its First Large Language Model: StableLM
By Adam Pease
Stability AI Releases Its First Large Language Model: StableLM
Large language models (LLMs) have taken the world by storm since the release of ChatGPT, but they come on the heels of last year’s revolution in text-to-image models, which showed that AI could be powerfully leveraged to generate images from scratch. Stability AI, the company behind Stable Diffusion, by far the most popular text-to-image model, has now waded into LLM territory with the release of StableLM.
What Is StableLM?
StableLM is a new language model trained by Stability AI. Like most model releases, it comes in a few different sizes, with 3 billion, 7 billion, and 15 and 30 billion parameter versions slated for releases. These parameter counts roughly correlate with model complexity and compute requirements, and they suggest that StableLM could be optimized for a variety of different devices depending on resource constraints.
The new models are trained on the foundations of a dataset called The Pile, which combines large volumes of internet data, and has been shown to be a powerful dataset for training LLMs. Additionally, Stability leverages the techniques pioneered by Stanford researchers in their Alpaca language models, which extend the base model through fine tuning to make it more receptive to user inquiries, and more naturalistic to communicate with.
The Language Model Wars
When it released Stable Diffusion last year, Stability AI’s only serious competition was DALL-E 2, OpenAI’s closed-source text-to-image model. By offering a high-quality, open-source alternative, Stability AI was able to breathe life into the emerging generative content market and catalyze a frenzy of development efforts and new business model concepts.
Now, it steps into a more competitive space. As many of tech’s big players chase the ‘lightning in a bottle’ quality of ChatGPT, Stability will need to demonstrate that its model can deliver the kind of quality that many users have come to expect from language models since interacting with the offerings of market leaders. Currently, it is too early to tell how StableLM will fare in the market, and there will doubtless be considerable benchmarking done in the next several weeks to draw accurate comparisons between this new offering and other language models on the market. One promising point in its favor, though, is the 4096 token context window of the model, which supports better memory and recall than other open source alternatives.
Bottom Line
Like many others, Stability AI is diving into the large language model market. Previously, it energized the open source community around generative content and drove innovation; it remains to be seen if Stability can bring the same transformative power to LLMs.
Sign up for Aragon Research’s latest expert-led webinars!
 |
 |
tPaaS–The Market Required for Digital Business Transformation |
Aragon Research’s Q2 2023 Research Agenda |
Have a Comment on this?