ALLPCB
Overview
Earlier discussions covered compute chips and memory. This article explains how compute chips output performance to form the compute foundation behind generative AI systems such as ChatGPT.
Compute Cards in Servers
In current compute servers, chips are not simply stacked; instead, compute cards are inserted side by side into slots. Most servers host eight or sixteen compute cards per system.
The core of a compute card is the compute chip, paired with large, high-bandwidth memory and caches, and often a CPU for scheduling. To support data transfer, high-speed channels are used. PCIe plays that role in the system by providing the bus channels.
PCIe in AI Heterogeneous Systems
PCIe is widely used in data processing. For example, a CPU controlling and scheduling chipset work often uses the PCIe bus. Interconnects between compute cards may use SerDes links. Writing computed data to SSDs also uses PCIe. The internal CPU of a switch and the switch chip within a compute center can communicate over PCIe as well.
As a general bus standard, PCIe links the heterogeneous components of AI systems. Using PCIe interfaces reduces the complexity of interconnecting compute chips, accelerators, memory, storage, gateway chips, and switches.
Key PCIe Characteristics
- Efficient data transfer
- High scalability
- Real-time processing capability
- Continuously optimized low-power performance
Testing and Reliability
Because PCIe connects core components and handles data transfer, performance and reliability are critical. Testing typically starts with measuring peak upload and download speeds on the PCIe interface, and verifying stable transfer rates, which are essential for efficient computation. Electrical characteristics should also be tested, including transmission power and low-power features such as high-speed logic wake, as illustrated below.
Stability testing should also cover hot-plug behavior, isolation, and EMC performance. Some vendors develop proprietary interconnect standards to exceed PCIe speeds. For example, Intel provides Xe Link for compute card interconnects, an extension based on PCIe 5.0; NVIDIA developed NVLink as an alternative to PCIe, offering bandwidth and energy efficiency several times that of PCIe 5.0. These proprietary schemes introduce additional testing challenges. Whether using standard PCIe or proprietary high-speed interconnects, appropriate test solutions are required to meet the related testing requirements, as illustrated below.
