Timothée Leleu is Senior Research Scientist and Head of the Algorithms & Applications Group in the NTT Research Physics & Informatics (PHI) Lab. Dr. Leleu was previously Project Associate Professor at the International Research Center for Neurointelligence at the University of Tokyo, where he worked on neuromorphic computing algorithms and architecture. He is primarily interested in the combinatorial aspects of neural computation, with a particular focus on the analysis of neuronal activity for the inference of network structure and the use of artificial neural networks for solving combinatorial optimization problems. Dr. Leleu invented the chaotic amplitude control (CAC) algorithm, which is highly relevant to the PHI Lab’s work on coherent Ising machines (CIMs). For more about Dr. Leleu and his research, please see the following Q&A:
You recently joined the PHI Lab as Senior Research Scientist. What is the purpose of the Algorithms & Applications Group? How do you see your role as head of this group?
The Algorithms & Applications Group at NTT Research’s PHI Lab primarily focuses on the development of algorithms that leverage post-Von Neumann computing paradigms, particularly those related to combinatorial optimization, machine learning and neuromorphic computing.
My role involves bridging the gap between the fundamental physics that underpins hardware development and its computational applications, and overseeing the development of groundbreaking algorithms, aligning them with innovative computing paradigms and ensuring that these technological advancements are applicable and beneficial in solving real-world problems.
How does this group play a role in what you described in your NTT Research video as the PHI Lab’s more realistic way of looking at quantum computing?
Our focus extends beyond the realm of traditional “gate-based” quantum computers and the development of quantum algorithms tailored for them. Instead, we aim to establish new computational paradigms that are not confined by the limitations of these conventional models.
How did you become interested in the Ising model (and CIMs) and neuromorphic computing? Are you approaching these areas more from a perspective of applied mathematics or physics or neuroscience or another discipline?
I did my PhD in computational neuroscience from an applied mathematics perspective, and my initial instinct in interpreting the coherent Ising machine (CIM) was to seek parallels with neural networks. The Ising model, due to its general applicability, has long served as a foundational model for understanding the brain. It was an ideal topic for conducting interdisciplinary research.
You are credited with developing the chaotic amplitude control (CAC) algorithm. When did that first appear? Was it in the 2021 paper in Communications Physics, “Scaling advantage of chaotic amplitude control for high-performance combinatorial optimization,” or an earlier work?
The concept of chaotic amplitude control originated from a previous study we published in Physical Review Letters in 2019, titled “Destabilization of Local Minima in Analog Spin Systems by Correction of Amplitude Heterogeneity.” Preliminary ideas about this algorithm were also presented in an earlier paper published in Physical Review E in 2017. The term “chaotic amplitude control” was officially coined in our 2021 paper.
Could you briefly describe CAC and how it works? Is the general idea to remove the impediments to gradient descent dynamics that limit the computational power of Ising models?
The core concept involves introducing dynamics that extend beyond mere gradient descent. The fundamental reason this is necessary for solving non-convex optimization problems is that the loss landscape these algorithms aim to minimize often contains numerous local minima or metastable states, where gradient descent typically becomes trapped. While there are other methods that employ a momentum term to aid in escaping these traps, Chaotic Amplitude Control (CAC) is distinct in its approach. It generates asymmetric interactions within the system to expedite the search for good solutions.
We noticed that Dr. Toru Aonishi, co-author of a paper reporting a first practical application of the Cyber-CIM, recently said that CIM-CAC has improved the performance of L0 regularization-based compressed sensing. Do you see other physicists beginning to use CAC?
While there are some empirical justifications for the effectiveness of Chaotic Amplitude Control (CAC) as an algorithm, its underlying statistical mechanics remain poorly understood. This is largely because CAC belongs to a category of systems that are challenging to analyze analytically. As theoretical tools for understanding dynamics in complex systems improve, other physicists may become interested in unraveling the reasons behind the algorithm’s good performance.
What other topics are you exploring? Any other papers or projects you’d like to mention?
As large-scale machine learning models continue to evolve, there is a growing interest in understanding the dynamics of optimization within complex systems. Our focus is especially on concepts that combine machine learning, combinatorial optimization and the fundamental physics underlying hardware development.
