\n<\/p>\n
In the early days of high-performance computing, the major tech companies of the day each invested heavily in developing their own closed source versions of Unix. It was hard to imagine at the time that any other approach could develop such advanced software. Eventually though, open source Linux gained popularity \u2013 initially because it allowed developers to modify its code however they wanted and was more affordable, and over time because it became more advanced, more secure, and had a broader ecosystem supporting more capabilities than any closed Unix. Today, Linux is the industry standard foundation for both cloud computing and the operating systems that run most mobile devices \u2013 and we all benefit from superior products because of it.<\/span><\/p>\n I believe that AI will develop in a similar way. Today, several tech companies are developing leading closed models. But open source is quickly closing the gap. Last year, Llama 2 was only comparable to an older generation of models behind the frontier. This year, Llama 3 is competitive with the most advanced models and leading in some areas. Starting next year, we expect future Llama models to become the most advanced in the industry. But even before that, Llama is already leading on openness, modifiability, and cost efficiency.<\/span><\/p>\n Today we\u2019re taking the next steps towards open source AI becoming the industry standard. We\u2019re releasing Llama 3.1 405B, the first frontier-level open source AI model, as well as new and improved Llama 3.1 70B and 8B models. In addition to having significantly better cost\/performance relative to closed models, the fact that the 405B model is open will make it the best choice for fine-tuning and distilling smaller models.<\/span><\/p>\n Beyond releasing these models, we\u2019re working with a range of companies to grow the broader ecosystem. Amazon, Databricks, and NVIDIA are launching full suites of services to support developers fine-tuning and distilling their own models. Innovators like Groq have built low-latency, low-cost inference serving for all the new models. The models will be available on all major clouds including AWS, Azure, Google, Oracle, and more. Companies like Scale.AI, Dell, Deloitte, and others are ready to help enterprises adopt Llama and train custom models with their own data. As the community grows and more companies develop new services, we can collectively make Llama the industry standard and bring the benefits of AI to everyone.<\/span><\/p>\n Meta is committed to open source AI. I\u2019ll outline why I believe open source is the best development stack for you, why open sourcing Llama is good for Meta, and why open source AI is good for the world and therefore a platform that will be around for the long term.<\/span><\/p>\n When I talk to developers, CEOs, and government officials across the world, I usually hear several themes:<\/span><\/p>\n Meta\u2019s business model is about building the best experiences and services for people. To do this, we must ensure that we always have access to the best technology, and that we\u2019re not locking into a competitor\u2019s closed ecosystem where they can restrict what we build.<\/span><\/p>\n One of my formative experiences has been building our services constrained by what Apple will let us build on their platforms. Between the way they tax developers, the arbitrary rules they apply, and all the product innovations they block from shipping, it\u2019s clear that Meta and many other companies would be freed up to build much better services for people if we could build the best versions of our products and competitors were not able to constrain what we could build. On a philosophical level, this is a major reason why I believe so strongly in building open ecosystems in AI and AR\/VR for the next generation of computing.<\/span><\/p>\n People often ask if I\u2019m worried about giving up a technical advantage by open sourcing Llama, but I think this misses the big picture for a few reasons:<\/span><\/p>\n First, to ensure that we have access to the best technology and aren\u2019t locked into a closed ecosystem over the long term, Llama needs to develop into a full ecosystem of tools, efficiency improvements, silicon optimizations, and other integrations. If we were the only company using Llama, this ecosystem wouldn\u2019t develop and we\u2019d fare no better than the closed variants of Unix.<\/span><\/p>\n Second, I expect AI development will continue to be very competitive, which means that open sourcing any given model isn\u2019t giving away a massive advantage over the next best models at that point in time. The path for Llama to become the industry standard is by being consistently competitive, efficient, and open generation after generation.<\/span><\/p>\n Third, a key difference between Meta and closed model providers is that selling access to AI models isn\u2019t our business model. That means openly releasing Llama doesn\u2019t undercut our revenue, sustainability, or ability to invest in research like it does for closed providers. (This is one reason several closed providers consistently lobby governments against open source.)<\/span><\/p>\n Finally, Meta has a long history of open source projects and successes. We\u2019ve saved billions of dollars by releasing our server, network, and data center designs with Open Compute Project and having supply chains standardize on our designs. We benefited from the ecosystem\u2019s innovations by open sourcing leading tools like PyTorch, React, and many more tools. This approach has consistently worked for us when we stick with it over the long term.<\/span><\/p>\n I believe that open source is necessary for a positive AI future. AI has more potential than any other modern technology to increase human productivity, creativity, and quality of life \u2013 and to accelerate economic growth while unlocking progress in medical and scientific research. Open source will ensure that more people around the world have access to the benefits and opportunities of AI, that power isn\u2019t concentrated in the hands of a small number of companies, and that the technology can be deployed more evenly and safely across society.<\/span><\/p>\n There is an ongoing debate about the safety of open source AI models, and my view is that open source AI will be safer than the alternatives. I think governments will conclude it\u2019s in their interest to support open source because it will make the world more prosperous and safer.<\/span><\/p>\n My framework for understanding safety is that we need to protect against two categories of harm: unintentional and intentional. Unintentional harm is when an AI system may cause harm even when it was not the intent of those running it to do so. For example, modern AI models may inadvertently give bad health advice. Or, in more futuristic scenarios, some worry that models may unintentionally self-replicate or hyper-optimize goals to the detriment of humanity. Intentional harm is when a bad actor uses an AI model with the goal of causing harm.<\/span><\/p>\n It\u2019s worth noting that unintentional harm covers the majority of concerns people have around AI \u2013 ranging from what influence AI systems will have on the billions of people who will use them to most of the truly catastrophic science fiction scenarios for humanity. On this front, open source should be significantly safer since the systems are more transparent and can be widely scrutinized. Historically, open source software has been more secure for this reason. Similarly, using Llama with its safety systems like Llama Guard will likely be safer and more secure than closed models. For this reason, most conversations around open source AI safety focus on intentional harm.<\/span><\/p>\n Our safety process includes rigorous testing and red-teaming to assess whether our models are capable of meaningful harm, with the goal of mitigating risks before release. Since the models are open, anyone is capable of testing for themselves as well. We must keep in mind that these models are trained by information that\u2019s already on the internet, so the starting point when considering harm should be whether a model can facilitate more harm than information that can quickly be retrieved from Google or other search results.\u00a0<\/span><\/p>\n When reasoning about intentional harm, it\u2019s helpful to distinguish between what individual or small scale actors may be able to do as opposed to what large scale actors like nation states with vast resources may be able to do.<\/span><\/p>\n At some point in the future, individual bad actors may be able to use the intelligence of AI models to fabricate entirely new harms from the information available on the internet. At this point, the balance of power will be critical to AI safety. I think it will be better to live in a world where AI is widely deployed so that larger actors can check the power of smaller bad actors. This is how we\u2019ve managed security on our social networks \u2013 our more robust AI systems identify and stop threats from less sophisticated actors who often use smaller scale AI systems. More broadly, larger institutions deploying AI at scale will promote security and stability across society. As long as everyone has access to similar generations of models \u2013 which open source promotes \u2013 then governments and institutions with more compute resources will be able to check bad actors with less compute.\u00a0<\/span><\/p>\n The next question is how the US and democratic nations should handle the threat of states with massive resources like China. The United States\u2019 advantage is decentralized and open innovation. Some people argue that we must close our models to prevent China from gaining access to them, but my view is that this will not work and will only disadvantage the US and its allies. Our adversaries are great at espionage, stealing models that fit on a thumb drive is relatively easy, and most tech companies are far from operating in a way that would make this more difficult. It seems most likely that a world of only closed models results in a small number of big companies plus our geopolitical adversaries having access to leading models, while startups, universities, and small businesses miss out on opportunities. Plus, constraining American innovation to closed development increases the chance that we don\u2019t lead at all. Instead, I think our best strategy is to build a robust open ecosystem and have our leading companies work closely with our government and allies to ensure they can best take advantage of the latest advances and achieve a sustainable first-mover advantage over the long term.<\/span><\/p>\n When you consider the opportunities ahead, remember that most of today\u2019s leading tech companies and scientific research are built on open source software. The next generation of companies and research will use open source AI if we collectively invest in it. That includes startups just getting off the ground as well as people in universities and countries that may not have the resources to develop their own state-of-the-art AI from scratch.<\/span><\/p>\n The bottom line is that open source AI represents the world\u2019s best shot at harnessing this technology to create the greatest economic opportunity and security for everyone.<\/span><\/p>\n With past Llama models, Meta developed them for ourselves and then released them, but didn\u2019t focus much on building a broader ecosystem. We\u2019re taking a different approach with this release. We\u2019re building teams internally to enable as many developers and partners as possible to use Llama, and we\u2019re actively building partnerships so that more companies in the ecosystem can offer unique functionality to their customers as well.\u00a0<\/span><\/p>\n I believe the Llama 3.1 release will be an inflection point in the industry where most developers begin to primarily use open source, and I expect that approach to only grow from here. I hope you\u2019ll join us on this journey to bring the benefits of AI to everyone in the world.<\/span><\/p>\n You can access the models now at llama.meta.com<\/a>.\u00a0<\/span><\/p>\n ?,\u00a0<\/span><\/p>\n MZ<\/span><\/p>\n<\/p><\/div>\nWhy Open Source AI Is Good for Developers<\/span><\/h2>\n
\n
Why Open Source AI Is Good for Meta<\/span><\/h2>\n
Why Open Source AI Is Good for the World<\/span><\/h2>\n
Let\u2019s Build This Together<\/span><\/h2>\n