Dr. Gautam Reddy quantifies “a-ha” moments in paper published by National Academy of Sciences
Sunnyvale, Calif. – July 12, 2023 – NTT Research, Inc., a division of NTT (TYO:9432), today announced that a paper by Physics & Informatics (PHI) Lab Scientist Dr. Gautam Reddy, which makes a novel contribution to the understanding of learning, has been published in the Proceedings of the National Academy of Sciences (PNAS), one of the world’s most cited and comprehensive multidisciplinary scientific journals. The paper, titled “A reinforcement-based mechanism for discontinuous learning,” uses data on spatial navigation of labyrinths by laboratory mice to provide a simple explanation for the “a-ha” moment of sudden insight. Dr. Reddy shows that the outcome of reinforcement learning (RL) algorithms, which operate gradually, can appear discontinuous for certain kinds of tasks. Specifically, this is true when the artificial RL agent learns the direct path from point A to point B in certain structured environments.
Labyrinths or mazes have a similar complex structure to knowledge graphs, which represent how humans mentally explore and navigate a knowledge database. A recent paper by neuroscientists at Caltech describes how mice navigating through a maze to find water show “rapid learning, sudden insight and efficient exploration.” Dr. Reddy analyzes the data in that paper and produces a learning rule to explain the phenomenon of sudden insight. He quantifies the relationship between learning rule, derived from biologically based learning discontinuities; learning curve, which represents the speed of reinforcement waves; and exploration bias, or the probability that a mouse will take a certain action at a certain intersection. The paper’s novel contribution is that RL algorithms can reproduce “a-ha” moment discontinuities in simulations.
“Congratulations to Dr. Reddy on his publication in this prestigious journal and for his creative discovery,” PHI Lab Director Yoshihisa Yamamoto said. “His important findings will advance our understanding of how we reason, solve problems and gain insight.”
Long-standing RL algorithms incrementally reinforce rewarding actions through accumulated experience. Yet, past and recent experiments find sharp discontinuities in learning. Dr. Reddy sheds light on the underlying mechanism.
“We show that the nonlinear dynamics of RL-based learning, together with continuous exploration, lead to discontinuous learning curves in tree-like structured environments,” Dr. Reddy said. “We develop a quantitative theory which explains the origin and highlights the generality of the phenomenon. The theory explains existing data and provides specific testable predictions.”
Looking ahead, Dr. Reddy plans to extend the experimental framework beyond tree-like graphs to topologies with loops. He also hopes to examine how an animal learns the neural representation of its location in a maze, a prerequisite to its evaluating differing actions at a particular location.
Along with these research initiatives, Dr. Reddy also serves as the NTT Research PHI Lab lead on a five-year joint research project with scientists at Harvard University to study animal neuro-responses, in the hope of informing future artificial intelligence (AI) systems. The PHI Lab’s mission is to uncover fundamental principles and novel technologies that advance information processing beyond state of the art. Its members and partners explore the space between quantum information science, neuroscience and optical technologies. In October 2020, PHI Lab Director Yamamoto co-authored a paper in Applied Physics Letters (APL) titled, “Coherent Ising Machines: Quantum optics and neural network perspectives,” which underscored the interdisciplinary nature of the PHI Lab’s pathbreaking research agenda.
In addition to Harvard, eight other universities have agreed to conduct joint research with the PHI Lab. These include the California Institute of Technology (Caltech), Cornell University, Massachusetts Institute of Technology (MIT), Notre Dame University, Stanford University, Swinburne University of Technology, the Tokyo Institute of Technology and the University of Michigan. The NASA Ames Research Center in Silicon Valley has also entered into a joint research agreement with the PHI Lab.
About NTT Research
NTT Research opened its offices in July 2019 as a new Silicon Valley startup to conduct basic research and advance technologies that promote positive change for humankind. Currently, three labs are housed at NTT Research facilities in Sunnyvale: the Physics and Informatics (PHI) Lab, the Cryptography and Information Security (CIS) Lab, and the Medical and Health Informatics (MEI) Lab. The organization aims to upgrade reality in three areas: 1) quantum information, neuroscience and photonics; 2) cryptographic and information security; and 3) medical and health informatics. NTT Research is part of NTT, a global technology and business solutions provider with an annual R&D budget of $3.6 billion.
###
NTT and the NTT logo are registered trademarks or trademarks of NIPPON TELEGRAPH AND TELEPHONE CORPORATION and/or its affiliates. All other referenced product names are trademarks of their respective owners. © 2023 NIPPON TELEGRAPH AND TELEPHONE CORPORATION
| NTT Research Contact: Chris Shaw Chief Marketing Officer NTT Research +1-312-888-5412 chris.shaw@ntt-research.com | Media Contact: Stephen Russell Wireside Communications® For NTT Research +1-804-362-7484 srussell@wireside.com |
