{"id":18510,"date":"2024-04-10T15:49:20","date_gmt":"2024-04-10T15:49:20","guid":{"rendered":"http:\/\/scannn.com\/introducing-our-next-generation-infrastructure-for-ai\/"},"modified":"2024-04-10T15:49:20","modified_gmt":"2024-04-10T15:49:20","slug":"introducing-our-next-generation-infrastructure-for-ai","status":"publish","type":"post","link":"https:\/\/scannn.com\/lv\/introducing-our-next-generation-infrastructure-for-ai\/","title":{"rendered":"Introducing Our Next Generation Infrastructure for AI"},"content":{"rendered":"


\n<\/p>\n

\n

The next generation of Meta\u2019s large-scale <\/span>infrastructure is being built with AI in mind,<\/span><\/a> including supporting new generative AI products, recommendation systems and advanced AI research. It\u2019s an <\/span>investment we expect will grow<\/span><\/a> in the years ahead, as the compute requirements to support AI models increase alongside the models\u2019 sophistication.<\/span><\/p>\n

\"A<\/a><\/p>\n

Last year, we unveiled our<\/span> Meta Training and Inference Accelerator (MTIA) v1<\/span><\/a>, our first-generation AI inference accelerator that we designed in-house with Meta\u2019s AI workloads in mind. It was designed specifically for our deep learning recommendation models that are improving a variety of experiences across our apps and technologies.\u00a0<\/span><\/p>\n

MTIA is a long-term bet to provide the most efficient architecture for Meta\u2019s unique workloads. As AI workloads become increasingly important to our products and services, this efficiency will be central to our ability to provide the best experiences for our users around the world. MTIA v1 was an important step in improving the compute efficiency of our infrastructure and better supporting our software developers as they build AI models that will facilitate new and better user experiences.\u00a0<\/span><\/p>\n

The next generation of MTIA is part of our broader full-stack development program for custom, domain-specific silicon that addresses our unique workloads and systems. This new version of MTIA more than doubles the compute and memory bandwidth of our previous solution while maintaining our close tie-in to our workloads. It is designed to efficiently serve the ranking and recommendation models that provide high-quality recommendations to users.<\/span><\/p>\n

This chip\u2019s architecture is fundamentally focused on providing the right balance of compute, memory bandwidth and memory capacity for serving ranking and recommendation models.\u00a0<\/span><\/p>\n

\"A<\/a><\/p>\n

MTIA has been deployed in our data centers and is now serving models in production. We are already seeing the positive results of this program as it\u2019s allowing us to dedicate and invest in more compute power for our more intensive AI workloads.<\/span><\/p>\n

The results so far show that this MTIA chip can handle both low complexity and high complexity ranking and recommendation models which are key components of Meta\u2019s products.\u00a0 Because we control the whole stack, we can achieve greater efficiency compared to commercially available GPUs (graphics processing units).\u00a0<\/span><\/p>\n

This slideshow requires JavaScript.<\/p>\n

Meta\u2019s Ongoing Investment in Custom Silicon<\/span><\/h2>\n

MTIA will be an important piece of our long-term roadmap to build and scale the most powerful and efficient infrastructure possible for Meta\u2019s unique AI workloads.<\/span><\/p>\n

We\u2019re designing our custom silicon to work in cooperation with our existing infrastructure as well as with new, more advanced hardware (including next-generation GPUs) that we may leverage in the future. Meeting our ambitions for our custom silicon means investing not only in compute silicon but also in memory bandwidth, networking and capacity, as well as other next-generation hardware systems.<\/span><\/p>\n

\"An<\/a><\/p>\n

We currently have several programs underway aimed at expanding the scope of MTIA, including support for GenAI workloads. And we\u2019re only at the beginning of this journey.<\/span><\/p>\n<\/p><\/div>\n