University of Tokyo
Using Principles of Neural Networks to Increase the Efficiency of Ising Machine Simulators
The human brain is an inspiring model of what can be achieved computationally. To design a more effective simulation of a Coherent Ising Machine (CIM), we can use what we understand of the brain’s balance between speed, size, and energy efficiency.
“The brain computes using billions of neurons using only 20 Watts of power and operates at a relatively low frequency,” said Timothee Leleu, Project Professor of Information and Electronics at the University of Tokyo, in his talk at Update 2020, the virtual NTT Research Summit. “These impressive characteristics motivate us to try to investigate what kind of new inspired principles may be useful for designing better Ising machines.”
The limits of an Ising Machine are tied to compromises made for efficiency. Any human-built in-silico simulation of an Ising Machine must consider the relationship between energy efficiency, speed, and scalability. A change in any of these not only affects the others, but greatly affects overall performance and viability.
An FPGA simulator may have good speed and size, partially due to its reconfigurability, but is currently limited in terms of energy efficiency. “The reason for the good performance of FPGA is not so much that they operate at a higher frequency, nor are they particularly energy efficient, but rather that the physical wiring of its elements can be reconfigured in a way that limits the von Neumann bottleneck,” Leleu said.
Looking at the human brain as inspiration, one can derive that it is indeed possible to create a CIM that is both low frequency, low energy, and fast. By analyzing three principles of the brain, we can, at least temporarily, push past the limitations of in-silico simulations. These three principles are:
- Microstructure, or how a local structure is repeated
- Asymmetry of connections and how they incite connectivity
- Hierarchical organization of activity, or how these local structures organize communication
Interestingly, introducing a microstructure in an FPGA affects its symmetry. Leleu proposed replacing the representation of one analog spin (or one degenerate optical parametric oscillator) with a pair: one analog spin and one error-encoding variable that controls the amplitude of the spin. “The addition of this microstructure introduces asymmetry in the system, which in turns induces… a chaotic search, rather than an annealing process for searching for the ground state of the Ising Hamiltonian,” he says.
When it comes to hierarchical organization, Leleu relates the FPGA efficiency bottleneck to the overall structure of the FPGA. Having larger logic blocks, rather than breaking them up into smaller blocks, is more efficient as the system is scaled up. But that requires improving the fan-in and fan-outs of the system, as well as the long distance startup path within the FPGA.
Consider the way the brain is organized, with larger sections responsible for different tasks that must communicate with each other. If the brain were composed of an array of multitudinous smaller sections, the communication between these blocks would require much more energy. The brain instead balances the efficiency of communication with the sections that must communicate.
Leleu presented a proof of concept for implementing these brain-inspired ideas into an FPGA, which are discussed in a paper he wrote with colleagues including F. Khoyratee and R. Hamerly. The results are promising when compared to other proposed CIM solutions. While other methods succeed in speed on smaller scales, the work of Leleu’s team shows improvements in scalability and speed when considering larger groups of spins (>1000). Taking it a step further, he said, “This projection shows that we can probably solve prior SK problems of size 2000 spins… with 99% success quality in less than one second, which is much faster than all the other proposed approaches.”
Leleu and his colleagues plan on improving this simulator by September 2021 before making it open-access for researchers to run their own instances.
For the full transcript of Timothee Leleu’s presentation, click here.
Watch Timothee Leleu’s full presentation below.
Neuromorphic in Silico Simulator For the Coherent Ising Machine