Technion Study: Learning Reveals Hidden Information About Network Structure
Technion researchers have developed a mathematical description of learning that reveals hidden differences between networks that behave identically
Learning in the brain involves changes in the connections between neurons, but its effects show up in the system’s overall behavior. How do the many changes in these connections combine to alter the system’s function? In a new study, Technion researchers examined this relationship using mathematical models of neural networks, showing that a network’s function and learning reveal different aspects of its internal structure.The paper by Prof. Omri Barak and research student Yoav Ger of the Technion’s Ruth and Bruce Rappaport Faculty of Medicine was selected for an oral presentation at NeurIPS 2026, one of the world’s leading conferences in artificial intelligence and computational neuroscience. Of the 30,709 papers submitted to the conference, only 112 (approximately 0.36%) were selected for oral presentations.
The researchers focused on a particular type of recurrent neural network, used to model how the brain processes information. They developed a mathematical description that makes it possible to calculate how a network changes during learning using a small number of variables. Their analysis distinguished between two types of properties: those that determine the network’s actual behavior and those that do not affect its current performance but influence how it learns in the future. These “hidden” properties are not apparent when examining the system’s behavior alone, but they may emerge when the system is required to learn a new task.
In one example in the study, two networks with different internal structures responded identically to the same inputs. However, when trained under the same conditions, they followed different learning trajectories: one experienced a period in which improvement slowed considerably, while the other improved more steadily. Learning itself can therefore serve as a research tool, revealing differences that are not apparent when examining only the system’s current function.
The study also demonstrated that, under certain conditions, these hidden properties can retain traces of the network’s learning history. Networks initially trained on different tasks and subsequently trained on a shared task achieved similar performance, yet their hidden structural properties still made it possible to identify each network’s training history.
“The question of how changes in the structure of a complex system affect its behavior arises in many fields of science,” said Prof. Barak. “In the networks we studied, we found that understanding what the system does today is not enough to describe how it will learn in the future. The way it changes through learning can reveal structural differences that remain hidden in its current function.”
The results were obtained using mathematical models and computer simulations. They suggest a direction for experiments in biological systems: examining how systems with similar behavior change in response to the same learning process, and determining which internal differences can be revealed in this way.
The research was supported by the Israel Science Foundation (ISF) and the Human Frontier Science Program.
