NTT Research President and CEO Kazuhiro Gomi and Physics & Informatics (PHI) Lab Research Scientists Logan Wright and Tatsuhiro Onodera discussed physical neural networks on July 19 during the VentureBeat Transform 2022 event held in San Francisco. The innovative research of Drs. Wright and Onodera and colleagues at Cornell University, published earlier this year in the influential journal Nature, introduced a machine-learning algorithm able to train unconventional, non-digital hardware. It’s a novel approach that challenges commonly held assumptions about computing.
“We’ve gotten so used to the idea of using digital computers, we can’t conceive of other ways to have numbers turn into numbers,” said Onodera. The idea of non-digital computing looks new but harkens to an earlier day – and to nature itself. “From the very early history of artificial intelligence, people were not trying to think about how to make digital computers think,” said Wright. “They were trying to think about how we could emulate the brain, which of course is not digital.”
Deep neural networking, which makes up most of today’s machine learning (ML) or artificial intelligence (AI), features several layers of interconnected nodes. Inspired by the brain’s analog calculations, this form of AI and ML could benefit from a more natural approach. Not only is the brain a non-digital analog system, but it performs its calculations far more efficiently than today’s digital deep neural networks.
To demonstrate the point that something other than digital computers can compute, Tatushiro pointed to a favorite example: waves in a bathtub, when measured at their height on one side and then the other, are in effect encoding a set of numbers. “You can use nature itself to perform computation,” he said, and at minimal cost. “The thing here is that the water didn’t need a power supply.” The larger demonstration – the application of neural network training to unconventional analog (and hybrid analog-digital) hardware – is described in the Nature article. What they forecast are physical systems very different from conventional digital electronics that can perform ML faster and with much less power.
“AI is great, and as we’ve been hearing all day today, AI is changing our business, changing our world, but the energy consumption is just like this,” Kazuhiro Gomi said, gesturing sharply upward with his hand. (He has spelled out concerns surrounding the external costs and environmental impact of ML in this Forbes article). Overall, the potential efficiency gains are staggering. Wright said that it is theoretically possible to devise systems, using photonics, for instance, that can “do modern, deep neural networking much faster and with, one day, billions of times more energy efficiency than current GPU-based implementations.”
What about in practice? After all, this event’s sponsor, VentureBeat, is a media company that aims “to help business leaders make smart decisions and stay on top of breaking news.” Senior Editor and Writer Sharon Goldman, who led the panel discussion, asked about research priorities, whether businesses should invest, and possible timetables, among other questions.
Gomi’s recommendation to business leaders was to be aware of potential reductions of energy consumption in data centers, as well as the benefits of deploying AI/ML on the network edge. Early niche applications, according to Tatsuhiro, could include cameras and smart sensors. Wright said that scalability is a near-term priority and that photonic chips made of lithium niobate are one promising avenue. The longer-term prospects depend, he said.
“To answer your question about time, probably the near-term applications we’re talking about with these hybrid analog-digital systems, I can see something about that happening in the next five years,” Wright said. “And in terms of very large-scale systems that are doing, say, large language model inference in a server, I can see that in the next ten years, or maybe five years, if someone was really aggressive.”
The full panel discussion was recorded and has been posted on Vimeo.
