At Intel Imaginative and prescient 2024, Intel had so much to say about AI and what it’s been engaged on in that space. The corporate introduced a brand new AI accelerator known as Gaudi 3, its plans to collaborate on an open platform for enterprise AI, and subsequent technology processors.

Gaudi 3 makes use of Ethernet to attach tens of hundreds of accelerators, which the corporate believes will allow a “vital leap in AI coaching and inference for world enterprises trying to deploy GenAI at scale.”

Every accelerator can run 64,000 operations in parallel, which helps the computational complexity required by deep studying algorithms. Its reminiscence capability is 128 GB and it additionally has 3.7 TB of reminiscence bandwidth and 96 MB of obtainable on-board static RAM. In keeping with Intel, these reminiscence specs make it potential to effectively serve LLMs and multimodal fashions.

The software program Gaudi runs on integrates with PyTorch and supplies optimized fashions from Hugging Face, which the corporate says makes it simple to port fashions throughout totally different {hardware} varieties. 

Gaudi 3 additionally introduces a peripheral part interconnect specific (PCIe) card that’s useful for workloads like fine-tuning, inference, and retrieval augmented technology. 

In comparison with its competitor Nvidia H100, Intel expects Gaudi 3 to be 50% quicker to coach throughout Llama2 with 7B and 13B parameters and GPT-3 with 175B parameters. It additionally is predicted to have 50% extra throughput basically and 40% extra for inference power-efficiency, in comparison with Nvidia’s. 

Intel anticipates making Gaudi 3 obtainable to producers, together with Dell Applied sciences, HPE, Lenovo, and Supermicro, within the second quarter of this yr.

“Within the ever-evolving panorama of the AI market, a big hole persists within the present choices,” stated Justin Hotard, govt vice chairman and common supervisor of the Information Middle and AI Group at Intel. “Suggestions from our prospects and the broader market underscores a need for elevated alternative. Enterprises weigh issues comparable to availability, scalability, efficiency, price, and power effectivity. Intel Gaudi 3 stands out because the GenAI different presenting a compelling mixture of worth efficiency, system scalability, and time-to-value benefit.”

Alongside the announcement of Gaudi 3, the corporate additionally introduced that it was collaborating with various firms to create an open platform for AI within the enterprise. 

To assist this effort, Intel can be releasing reference implementations for GenAI pipelines of Intel Xeon and Gaudi-based techniques, publish a technical conceptual framework, and add extra infrastructure capability within the Intel Tiber Developer Cloud. 

The opposite firms who’re working collectively on this venture embrace  Anyscale, Articul8, DataStax, Domino, Hugging Face, KX Programs, MariaDB, MinIO, Qdrant, RedHat, Redis, SAP, VMware, Yellowbrick, and Zilliz.

And at last, the corporate introduced the following technology of its Intel Xeon processors. The brand new Intel Xeon 6 processors embrace Environment friendly-cores (E-cores) and Efficiency-core (P-cores). The E-cores supply a 4x efficiency enchancment and a couple of.7x higher rack density than the 2nd technology Intel Xeon processors. P-cores add assist for the MXFP4 knowledge format, lowering token latency by 6.5x in comparison with the 4th technology Intel Xeon processors.

In keeping with Intel, the Xeon 6 processors with E-cores will launch this quarter and processors with P-cores will launch after that.

The corporate additionally teased that the following technology of Intel Extremely processors will launch later this yr and can have over 100 platform tera operations per second (TOPS) and over 45 neural processing unit TOPS. 

“Innovation is advancing at an unprecedented tempo, all enabled by silicon – and each firm is rapidly turning into an AI firm,” stated Pat Gelsinger, CEO of Intel. “Intel is bringing AI in all places throughout the enterprise, from the PC to the info heart to the sting. Our newest Gaudi, Xeon and Core Extremely platforms are delivering a cohesive set of versatile options tailor-made to fulfill the altering wants of our prospects and companions and capitalize on the immense alternatives forward.”