Interconnects Become Critical at AI Scale
In 2024, scaling tens, hundreds, or even tens of thousands of AI accelerators brings interconnect technology into sharp focus. Nvidia offers NVLink and InfiniBand, Google¡¯s TPU pods utilize Optical Circuit Switches (OCS), and AMD provides Infinity Fabric for intra-chip and inter-node traffic. Traditional Ethernet also remains in play.
The challenge isn't just building massive meshes, but mitigating performance loss and bandwidth bottlenecks from off-chip communication. These interconnects also don¡¯t address a key limitation: the fixed ratio between compute and high-bandwidth memory (HBM).
Celestial AI¡¯s Optical Fabric Approach
¡°The industry is effectively using Nvidia GPUs as the most expensive memory controllers in the world,¡± said Dave Lazovsky, founder of Celestial AI, which recently raised $175 million in Series C funding to commercialize its photonic fabric platform.
Last year, Celestial revealed its silicon photonics-based architecture composed of interconnect chiplets, interposers, and optical bridges. Less than a year later, the company is collaborating with several hyperscale customers and a major processor vendor to integrate its technology¡ªthough specific partners remain unnamed.
One notable backer is AMD Ventures. On the same day Celestial¡¯s funding was announced, AMD¡¯s SVP and Product Architect Sam Naffziger discussed the possibility of co-packaged silicon photonics chiplets, suggesting deeper collaboration. However, AMD¡¯s investment does not guarantee adoption in future Epyc CPUs or Instinct GPUs.
Chiplets as the Integration Path
Celestial¡¯s architecture consists of three primary components: chiplets, interposers, and an optical bridge¡ªOMIB¡ªcompatible with Intel EMIB or TSMC CoWoS. Lazovsky emphasized that the most accessible way to implement photonic fabric today is through optical chiplets.
These chiplets can serve either to expand HBM capacity or act as optical interconnects, comparable to optical NVLink or Infinity Fabric. Slightly smaller than HBM stacks, the chiplets offer 14.4 Tb/s (1.8 GB/s) of off-package optical bandwidth. Future chiplets could support up to 1.8 Tb/s per mm2 using second-generation technology with 112 Gb/s PAM4 SerDes and 8 channels¡ªdoubling the bandwidth.
Memory Expansion With Photonic Links
While chip-to-chip photonic links are straightforward¡ªoptical chiplets aligned with fiber¡ªthe memory expansion use case presents more complexity. A 14.4 Tb/s link can become a bottleneck for multiple HBM3 or HBM3E stacks. However, replacing a single HBM stack with two remains feasible.
Celestial¡¯s solution is a memory expansion module with two HBM stacks (72 GB total) and four DDR5 DIMMs supporting up to 2 TB of additional capacity. The module uses Celestial¡¯s silicon photonics interposer to bridge HBM, interconnects, and control logic.
At the heart of this module is a 5nm switch ASIC that effectively turns HBM into a direct-write cache for DDR5. It provides the capacity and cost benefits of DDR along with the bandwidth and pseudo-channel advantages of HBM¡ªmasking latency in the process.
According to Lazovsky, the energy cost of memory transactions using Celestial¡¯s photonic structure is about 6.2 pJ per bit, compared to roughly 62.5 pJ for NVLink/NVSwitch-based transactions. Latency is also competitive: roundtrip remote memory access via the photonic fabric is about 120 ns, just slightly more than local DRAM access, and faster than retrieving data from a remote Grace CPU via Hopper GPU.
Building Optical Memory Pools
Up to 16 of these memory modules can be aggregated into a memory switch and linked via fiber to form larger pools. This allows chips interconnected via Celestial¡¯s photonic fabric to not only communicate efficiently, but also share a common memory pool.
¡°This enables highly efficient machine learning operations like broadcast and reduce without switching,¡± said Lazovsky.
Competition and Timeline
Celestial expects to sample photonic chiplets to customers in the second half of 2025, with products potentially reaching the market by 2026 and volume scaling in 2027.
However, Celestial is not alone in the silicon photonics race. Ayar Labs¡ªbacked by Intel¡ªhas integrated photonic interconnects into prototype accelerators. Lightmatter raised $155 million in Series C funding and is using its Passage interposer for similar purposes, reportedly targeting supercomputers with up to 300,000 nodes. Eliyan, another startup, is focused on bypassing or enhancing interposers through its NuLink PHY.
Regardless of who wins, the shift toward co-packaged optics and silicon photonic interposers appears to be a matter of when¡ªnot if.
