The State of Open-Source Language Models
By Adam Pease
The State of Open-Source Language Models
Microsoft recently announced the release of its new Orca open-source language model. This blog discusses the release of Orca and the implications of open-source projects for the generative AI space overall.
The Open-Source Landscape
Recently there has been much uproar and debate amongst the machine learning community about the release of open-source models, such as Meta’s LLaMa model, which catalyzed a surge of open-source development akin to last year’s release of Stable Diffusion for image generation. These open-source models are usually smaller and more lightweight than proprietary behemoths like OpenAI’s GPT-4.
Open-source models have given rise to a widespread community of contributors that are developing novel applications for language models. In particular, the open-source community has been swift to pursue device optimizations for ML models, creating highly-efficient LLMs, that, while operating with reduced sophistication, can run on cheap devices. The growth of open-source has led to some concern among industry leaders that open-source models could be used for malicious purposes because they lack the same degree of moderation.
Enter Microsoft Orca
Among the concerns about misuse, prominent AI industry leaders have also sounded the alarm about the risks of potential competition from open-source models. Google was famously quoted as saying ‘we have no moat’ when comparing the progress of open-source to its own proprietary language models. Still, other researchers have suggested there is little reason to worry. One paper called “The False Promise of Imitating Proprietary LLMs” suggested that open-source will have difficulty catching up, and that smaller open-source models are essentially cheap imitations.
The release of Microsoft Orca, however, seems to disrupt some of these counterarguments against open-source. This new open-source language model developed by Microsoft excels on several significant benchmarks. More importantly, there is evidence that Orca can mimic the reasoning capabilities of more advanced models like GPT-4, potentially even learning the same underlying understanding. While it’s too early to tell what open-source developers will do with Orca, it’s clear that Microsoft sees value in the rising tide of open-source research in machine learning.
Bottom Line
The open-source machine learning movement is growing as independent contributors pursue the development of new, optimized approaches to developing and deploying language models. Microsoft’s Orca release suggests that industry leaders see the value of investing in these developments.
See Adam LIVE in our upcoming webinar!
Prompt Engineering for the Enterprise: Accelerating Workflows with Generative AI
Prompt Engineering for the Enterprise: Accelerating Workflows with Generative AI
Join us for a webinar where Aragon’s analyst, Adam Pease, will explore the process of prompt engineering for large language models within an enterprise context. This session will not only introduce you to the fundamental principles of prompt engineering but also showcase how forward-thinking businesses are already leveraging generative AI to revolutionize their operational efficiency.
On Thursday, July 13th, Adam’s talk will cover:
- How businesses can deploy generative AI in a cost-effective way
- The critical importance of prompt engineering for successfully leveraging generative AI
- Actionable best practices for prompt engineering in different workplace use cases
Have a Comment on this?