In a blockbuster announcement in the present day designed to coincide with the Microsoft Encourage convention, Meta introduced its new AI mannequin, LLaMA 2 (Giant Language Mannequin Meta AI). Not solely is that this new massive language mannequin (LLM) now accessible, it’s additionally open-source and freely accessible for industrial use — in contrast to the primary LLaMA, which was licensed just for analysis functions.

The information, coupled with Microsoft’s outspoken help for LLaMA 2, means the fast-moving world of generative AI has simply shifted but once more. Now the various enterprises speeding to embrace AI, albeit cautiously, have an alternative choice to select from, and this one is completely free — in contrast to chief and rival OpenAI’s ChatGPT Plus, or challengers like Cohere.

Rumors surrounding the brand new launch of LLaMA have been swirling within the trade for a minimum of a month, as U.S senators have been questioning Meta concerning the availability of the AI mannequin.

The primary iteration of LLaMA was accessible for lecturers and researchers beneath a analysis license. The mannequin weights underlying LLaMA have been nonetheless leaked, inflicting some controversy resulting in the federal government inquiry. With LLaMA 2, Meta is brushing apart the prior controversy and transferring forward with a extra highly effective mannequin that might be extra broadly usable than its predecessor and probably shake up the complete LLM panorama.

Microsoft hedges its AI bets

The LLaMA 2 mannequin is being made accessible on Microsoft Azure. That’s noteworthy in that Azure can also be the main residence for OpenAI and its GPT-3/GPT-4 household of LLMs. Microsoft is an investor each in Meta’s former firm Fb and in OpenAI.

Meta founder and CEO Mark Zuckerberg is especially smitten by LLaMA being open-source. In an announcement, Zuckerberg famous that Meta has a protracted historical past with open supply and has made many notable contributions, significantly in AI with the PyTorch machine studying framework.

“Open supply drives innovation as a result of it permits many extra builders to construct with new know-how,” Zuckerberg acknowledged. “It additionally improves security and safety as a result of when software program is open, extra folks can scrutinize it to determine and repair potential points. I consider it could unlock extra progress if the ecosystem have been extra open, which is why we’re open sourcing Llama 2.”

In a Twitter message, Yann LeCun, VP and chief AI scientist at Meta, additionally heralded the open-source launch.

“That is large: [LLaMA 2] is open supply, with a license that authorizes industrial use!” LeCun wrote. “That is going to vary the panorama of the LLM market. [LLaMA 2] is out there on Microsoft Azure and might be accessible on AWS, Hugging Face and different suppliers”

What’s inside LLaMA?

LLaMA is a transformer-based auto-regressive language mannequin. The primary iteration of LLaMA was publicly detailed by Meta in February as a 65 billion-parameter mannequin able to a wide selection of frequent generative AI duties.

In distinction, LLaMA 2 has plenty of mannequin sizes, together with seven, 13 and 70 billion parameters. Meta claims the pre-trained fashions have been educated on an enormous dataset that was 40% bigger than the one used for LLaMA 1. The context size has additionally been expanded to 2 trillion tokens, double the context size of LLaMA 1.

Not solely has LLaMA been educated on extra knowledge, with extra parameters, the mannequin additionally performs higher than its predecessor, in accordance with benchmarks offered by Meta.

Security measures touted

LLaMA 2 isn’t all about energy, it’s additionally about security. LLaMA 2 is first pretrained with publicly accessible knowledge. The mannequin then goes by a collection of supervised fine-tuning (SFT) phases. As an extra layer, LLaMA 2 then advantages from a cycle of reinforcement studying from human suggestions (RLHF) to assist present an additional diploma of security and accountability.

Meta’s analysis paper on LLaMA 2 supplies exhaustive particulars on the great steps taken to assist present security and restrict potential bias as effectively.

“It is very important perceive what’s within the pretraining knowledge each to extend transparency and to make clear root causes of potential downstream points, comparable to potential biases,” the paper states. “This will inform what, if any, downstream mitigations to contemplate, and assist information applicable mannequin use.”