The second annual SmartNICs Summit took place at the Doubletree Hotel in San Jose from June 13-15, 2023. Emerging technologies, such as generative Artificial Intelligence (AI), are driving SmartNIC adoption in hyperscaler environments and beyond. The event had 228 attendees and featured keynote presentations from Intel, Microsoft, NVIDIA, AMD, and VMware. Exhibitors included AMD, Intel, Keysight, Microsoft, CodiLime, Canonical, and Arm, among others. I attended partially on Tuesday, June 13, and fully on Wednesday, June 14; this article summarizes my observations from those days.
NVIDIA defines a SmartNIC as “a programmable accelerator that enhances data center networking, security, and storage efficiency,” a perspective shared by most vendors in the space. More specifically, SmartNICs offload specialized workloads from Central Processing Units (CPUs), enabling them to focus on general-purpose tasks. Key SmartNIC categories discussed at the conference, along with their industry definitions, are listed below:
- Data Processing Unit (DPU): DPUs offload networking and communication workloads, such as encryption and firewall rule application, from general-purpose CPUs. Essentially, DPUs are designed to handle data movement within data centers.
- Graphics Processing Unit (GPU): GPUs accelerate graphics rendering and complex workloads like supercomputing, AI, machine learning, database calculations, and big data analysis.
- Infrastructure Processing Unit (IPU): IPUs offload network and storage infrastructure tasks from CPUs. They manage functions like Open vSwitch (OVS) network forwarding, traffic shaping, quality of service, NVMe over Fabrics (NVMeOF), and storage I/O.
Hyperscalers like Amazon Web Services, Google Cloud, Microsoft Azure, Oracle Cloud, and IBM Cloud dominate the SmartNIC market. According to data from Dell’Oro Group published on the SmartNICs Summit website, the 2021 SmartNIC market was approximately $737M. Amazon and Microsoft accounted for over 80%, and the top four hyperscalers represented more than 90% of market revenues. The market is projected to reach $1.9B by 2027, with non-hyperscaler revenues exceeding 20%.
Intel’s keynote emphasized the transformative role of IPUs in hyperscaler data centers. They discussed an architectural paradigm shift involving the migration of hypervisor storage, network, and security functions from CPUs to IPUs. Intel’s IPUs are deployed in 6 of the 8 global hyperscalers, and they have co-developed their IPU E2000 series with Google. Intel aims to facilitate IPU adoption among telcos and smaller data centers by building a robust hardware and software ecosystem.
Gerald Degrace presented Microsoft’s keynote, which focused on the “Disaggregation of Cloud Networking via SmartNICs.” He described Software for Open Networking in the Cloud (SONiC) as the foundation for Disaggregated APIs for SONiC Hosts (DASH). SONiC is an open-source Network Operating System (NOS) supported by numerous switch vendors and used extensively in Azure. Degrace presented several DASH use-cases, including SDN flow rule offloading to increase client-server connections to tens of millions per second. The prior day (Tuesday pre-conference), Degrace’s colleague, Kristina Moore, Paul Cummins of NVIDIA, and Mircea Dan Gheorghe and Chris Sommers of Keysight presented at a DASH Workshop — I was not able to attend unfortunately. Nevertheless, more details on this abstraction model can be found at the SONiC and DASH GitHub sites.
Kevin Deierling’s NVIDIA keynote, “SmartNICs and DPUs Accelerate Generative AI at Data Center Scale,” highlighted the computational requirements of ChatGPT. Deierling cited the application as the fastest-growing in history, reaching 100 million users within two months. He emphasized the role of data centers as “AI factories,” supporting this with data showing that GPT-2 and GPT-3 consume over 10,000,000 and 100,000,000 petaFLOPS, respectively. Deierling also demonstrated that offloading IPSec encryption to an NVIDIA BlueField-3 DPU resulted in a 34% power savings per server.
During the Networking Applications session on June 14, Intel and AMD shared visions for using IPUs to boost 5G performance. Intel offers both FPGA-based IPUs like the C5000X-PL and F2000X-PL and ASIC-based IPUs like the E2000. AMD provides the Pensando DPU, highlighting use-cases like layer 1 and GPRS Tunneling Protocol acceleration. Both vendors anticipate telcos will lag behind hyperscalers in IPU adoption.
From my perspective as an aerospace industry professional, DPUs and IPUs may have potential applications in offloading network and security tasks from general-purpose CPUs in both military and commercial drones. For instance, commercial drones could offload IPSec encryption for video streams, and military drones could potentially seek NSA approval for SmartNIC-based encryption under the Commercial Solutions for Classified (CSfC) framework. Any DPU or IPU platform would need to fit within a drone’s Size, Weight, and Power (SWaP) envelope, but I see promising applications down the road.