The advancements in artificial intelligence (AI) have transformed various industries. As the size of AI models increases exponentially, traditional electronic systems are struggling to meet the compute demand due to their scaling limitations. This struggle necessitates a large network of disaggregated electronic chips for a single computational task and highlights the importance of optical technologies in data centers, which complement electrical systems by enhancing data transfer. Optical interconnect technology is progressing to integrate more closely with electronic chips, driven by the need for greater bandwidth capacities. However, as the increase in serial communication speeds becomes a challenge, strategies such as space division multiplexing and frequency division multiplexing are being explored to achieve larger bandwidth. Additionally, even within a single electronic chip, researchers are examining ways to lower power consumption related to data transfer in traditional von Neumann architectures by considering alternatives like compute-in-memory (or in-memory computing) architectures. By performing simple computations like multiplication and addition directly within the memory units, this approach eliminates the need to repeatedly load entire sets of raw data, thereby minimizing data bottlenecks caused by separating memory and processing units.
The evolution of modern data centers into hybrid opto-electronic computing machines is leading physicists to reevaluate the role of optics in performing computational tasks, especially linear operations like matrix-vector multiplication (MVM). Recent proposals have highlighted the energy efficiency of various optical MVM systems. Particularly promising are three-dimensional (3D) optical systems that use scalable free-space optics, though many still primarily rely on space division multiplexing, leaving the frequency dimension largely untapped. A recent paper published in Optica by the NTT Research Physics & Informatics (PHI) Lab introduces a hyperspectral compute-in-memory architecture that merges space and frequency division multiplexing to boost computational efficiency and throughput. The co-authors, as they appear in the byline, are PHI Lab Post-Doctoral Fellow Mostafa Honari Latifpour, PHI Lab Research Intern Byoung Jun Park, PHI Lab Director Yoshihisa Yamamoto, and PHI Lab Scientist Myoung-Gyun Suh.
This system, detailed in “Hyperspectral In-Memory Computing with Optical Frequency Combs and Programmable Optical Memories,” uses a two-dimensional spatial light modulator (SLM) as a programmable optical memory, enabling spatial parallel operations. This setup allows for energy-efficient parallel data processing by optics, while electronics enhance programmability. Given that space multiplexing alone does not match the density of electronic systems, the architecture also integrates frequency division multiplexing, inspired by hyperspectral imaging and advanced optical fiber communication technologies. This addition allows each pixel to handle multiple frequency signals concurrently. Comparing this early-stage optical computing with mature digital electronic technologies poses challenges, but the PHI Lab team cautiously estimates that their system might achieve 100 PetaOPS (Peta-operations per second) with a power efficiency near 2 W/PetaOPS, significantly outperforming contemporary electronic GPUs.
The authors also highlight the limitations of their proof-of-concept demonstration and discuss future research directions. “There is still a lot of work to do to transition our demonstration to practical use,” Dr. Myoung-Gyun Suh said. “Tailored development of key components, such as a novel 2D opto-electronic ‘neuron’ array, is crucial.” This 2D ‘neuron’ device creates a direct link between each photodetector pixel and its corresponding modulator (or light emitter) pixel using through-silicon via (TSV). Such a device could support uninterrupted parallel processing, significantly reducing delays and energy consumption associated with serial data conversion.
In the proposed system, light transfers data orthogonally to and from the 2D plane of the opto-electronic device array, similar to CMOS image sensors or optical display devices, enabling massively parallel communication and computation. “Our hyperspectral compute-in-memory architecture can be seen as a 3D opto-electronic computing system, where multiply-accumulate (MAC) operations are performed in an exceptionally parallel manner,” Dr. Suh said. By connecting the optical memory ‘synapse’ chip to the ‘neuron’ chip optically, thereby removing the need for physical wires, the system achieves efficient chip-area usage and exceeds PetaOPS/mm^2 in compute density. This high-compute density could significantly lower the economic and energy expenses related to electrical-to-optical (EO) and optical-to-electrical (OE) conversions. Importantly, as electronic operations are performed locally at each pixel during computation, electronic data movement is minimized, with most data communication occurring through optics.
The use of technologies like inverse-designed meta-optics and frequency micro-combs could also simplify optical assemblies, potentially leading to system miniaturization. The system’s modular nature could further benefit from technological advancements. “With continual improvements in component technology and the growing importance of optics in data centers,” Dr. Suh said, “this 3D opto-electronic computing architecture could transform high-performance accelerated computing hardware in future data-center applications.”