Technion & UCLA Researchers Identify a Structured Code used for Representing Speech Movements by Neurons in the Human Brain

35The researchers were able to directly decode vowels from the neural activity which leads to their articulation – a finding which could allow individuals who are completely paralyzed to “speak” to the people around them through a direct brain-computer interface

Technion and UCLA researchers succeeded in directly decoding vowels from the neural activity which leads to their articulation – a finding which could allow individuals who are completely paralyzed to “speak” to the people around them through a direct brain-computer interface. Prof. Shy Shoham and Dr. Ariel Tankus of the Technion Department of Biomedical Engineering, together with Prof. Itzhak Fried of the University of California Los Angeles (UCLA) and the Tel Aviv University and Medical Center departments of Neurosurgery, describe in a new research published in the scientific journal Nature Communications the way in which neurons in different areas of the human brain encode different speech segments (vowels) during their articulation. The discovery allows indirectly, to decode the content of the subjects’ speech based on brain activity alone. One of the possible applications of speech decoding from brain activity is the creation of a brain-computer interface that can restore speech faculties in paralyzed individuals who have lost them.

“There are diseases in which the patient’s entire body is paralyzed, he is effectively ‘locked in’ (locked-in syndrome) and is unable to communicate with the environment, but his mind still functions”, explains Prof. Shoham, Head of the Neural Interface Engineering Laboratory in the Technion Department of Biomedical Engineering. “Our long term goal is to restore these patients’ ability to speak using systems that will include implanting electrodes in their brains, decoding the neural activity that encodes speech, and sounding artificial speech sounds. For this purpose, we wanted to first understand how the information about the articulated syllable is encoded in the electrical activity of an individual brain neuron and of a neuron population. In our experiments we identified cell populations that distinctly participate in the representation. For example, cells we registered in an area in the medial frontal lobe that includes the anterior cingulate cortex, surprised us in the manner in which they ‘sharply’ represented certain vowels but not others, even though the area is not necessarily known as having a major role in the speech generation process”.

The experiments were conducted in the UCLA Medical Center with the participation of epilepsy patients, in whose brain Prof. Fried and his team implanted depth electrodes. The objective of the implant is to locate the epileptic focus, which is the area in the brain where epileptic seizures begin. After the surgery, the patients were hospitalized for a week or two with the electrodes in their brain, and waited for the occurrence of spontaneous seizures. During that time, Dr. Tankus, who was a post-doctoral fellow at UCLA and is now a researcher at the Technion, conducted experiments in which he asked the patients to articulate vowels as well as syllables comprising a consonant and a vowel, and recorded the resulting neuronal activity in their brain. The researchers discovered two neuron populations that encode the information about the vowel articulated in an entirely different way. In the first population, identified in the medial frontal lobe, each neuron encodes only one or two vowels by changing its firing rate, but does not change its activity when other vowels are articulated. However, in the second population, located in the superior temporal gyrus, each neuron reacts to all vowels tested, but the cell’s reaction strength changes gradually between vowels. Moreover, the researchers were able to deduce a mathematical arrangement of the manner in which the vowels are represented in the brain, showing it to match the phonetic vowel trapezoid, which is built according to the location of the highest point of the tongue during articulation. Thus, the researchers succeeded in connecting the brain representation with the anatomy and physiology of vowel articulation.

As aforesaid, understanding brain representation of speech generation constitutes also a significant step on the road to decoding cellular activity using a computer, as Dr. Tankus explains: “we have developed a new algorithm that improved greatly the ability to identify from brain activity which syllable was articulated, and this algorithm has allowed us to obtain very high identification rates. Based on the present findings, we are currently conducting experiments toward the creation of a brain-machine interface that will restore people’s speech faculties”.

Above: The two language areas where cell reactions during speech were researched. The graphs present a selective code for vowels in an area in the frontal lobe and a non-selective code in an area in the temporal lobe (each color represents a neuron).
The image highlights the brain areas where neurons with a structured code for speech generation were found. In the frontal region neurons were highly vowel-selective while in the temporal region (on the right) the code was broad and non-selective
Credit: Ariel Tankus

Give me a Voice. Brain Science & Speech

Technion & UCLA researchers have identified a structured code for representing speech movements by neurons in the human brain. The researchers were able to directly decode vowels from neural activity – a finding which could allow individuals who are paralyzed to “speak” to people around them through a brain-computer interface

In the photo: the two language areas where cell reactions during speech were researched. The graphs present a selective code for vowels in an area in the frontal lobe and a non-selective code in an area in the temporal lobe (each color represents a neuron). The image highlights the brain areas where neurons with a structured code for speech generation were found. In the frontal region neurons were highly vowel-selective while in the temporal region (on the right) the code was broad and non-selective.
Technion and UCLA researchers have directly decoded vowels from the neural activity which leads to their articulation – a finding which could allow individuals who are paralyzed to “speak” to the people around them through a direct brain-computer interface. 

Prof. Shy Shoham and Dr. Ariel Tankus of the Technion Department of Biomedical Engineering, together with Prof. Itzhak Fried of the University of California Los Angeles (UCLA) and the Tel Aviv University and Medical Center departments of Neurosurgery, describe the way in which neurons in different areas of the human brain encode different speech segments (vowels) during their articulation in the scientific journal Nature Communications

The discovery will indirectly enable scientists to decode the content of the subjects’ speech based on brain activity alone. One of the possible applications of speech decoding from brain activity is the creation of a brain-computer interface that can restore speech faculties in paralyzed individuals who have lost them.

“There are diseases in which the patient’s entire body is paralyzed, he is effectively ‘locked in’ (locked-in syndrome) and is unable to communicate with the environment, but his mind still functions”, explains Prof. Shoham, Head of the Neural Interface Engineering Laboratory in the Technion Department of Biomedical Engineering. “Our long term goal is to restore these patients’ ability to speak using systems that will include implanting electrodes in their brains, decoding the neural activity that encodes speech, and sounding artificial speech sounds. For this purpose, we wanted to first understand how the information about the articulated syllable is encoded in the electrical activity of an individual brain neuron and of a neuron population. In our experiments we identified cell populations that distinctly participate in the representation. For example, cells we registered in an area in the medial frontal lobe that includes the anterior cingulate cortex, surprised us in the manner in which they ‘sharply’ represented certain vowels but not others, even though the area is not necessarily known as having a major role in the speech generation process”.

The experiments were conducted in the UCLA Medical Center with the participation of epilepsy patients, in whose brain Prof. Fried and his team implanted depth electrodes. The objective of the implant is to locate the epileptic focus, which is the area in the brain where epileptic seizures begin. After the surgery, the patients were hospitalized for a week or two with the electrodes in their brain, and waited for the occurrence of spontaneous seizures. 

During that time, Dr. Tankus, who was a post-doctoral fellow at UCLA and is now a researcher at the Technion, conducted experiments in which he asked patients to articulate vowels as well as syllables comprising a consonant and a vowel, and recorded the resulting neuronal activity in their brain. 

The researchers discovered two neuron populations that encode the information about the vowel articulated in an entirely different way. In the first population, identified in the medial frontal lobe, each neuron encodes only one or two vowels by changing its firing rate, but does not change its activity when other vowels are articulated. However, in the second population, located in the superior temporal gyrus, each neuron reacts to all vowels tested, but the cell’s reaction strength changes gradually between vowels. 

Moreover, the researchers were able to deduce a mathematical arrangement of the manner in which the vowels are represented in the brain, showing it to match the phonetic vowel trapezoid, which is built according to the location of the highest point of the tongue during articulation. Thus, the researchers succeeded in connecting the brain representation with the anatomy and physiology of vowel articulation.

Understanding brain representation of speech generation constitutes a significant step on the road to decoding cellular activity using a computer, as Dr. Tankus explains: “We have developed a new algorithm that improves the ability to identify from brain activity which syllable was articulated, and this algorithm has allowed us to obtain very high identification rates. Based on the present findings, we are currently conducting experiments toward the creation of a brain-machine interface that will restore  speech faculties”.

The Technion Jumps in the Shanghai Ranking – to 78th Place Worldwide

Advances to 29th in chemistry and 42nd in engineering; Technion ranks 18th in computer sciences, above all European universities

The Technion jumped to the 78th place in the Shanghai ranking of universities, considered the world’s most reliable and comprehensive ranking system. Last year the Technion was in the 101-150 category.

In natural sciences the Technion jumped to 39th place (51-75 in 2011), and in engineering it maintained its 42nd place. In computer sciences, the Technion is ranked #18 worldwide, higher than all European universities; in chemistry the Technion jumped to 29th place (51-75 last year), in part a result of the Nobel Prize in Chemistry awarded to Distinguished Professor Dan Shechtman.

Technion President Professor Peretz Lavie attributed these rankings to the outstanding achievements of the past year, including Prof. Shechtman’s Nobel Prize and the selection of the Technion and Cornell University to establish a new applied science and engineering campus in New York City. He noted that in a recent survey by Business Insider, the Technion placed 25th   among engineering universities worldwide.

“These achievements are the result of the uncompromising excellence of the Technion, that is celebrating 100 years since the laying of the cornerstone for our historic building at Hadar Hacarmel”, he emphasized. “Our outstanding faculty, researchers and staff will continue to nurture and train the students of the Technion, the future generation of Israel, the ‘start-up nation.’ “

The Academic Ranking of World Universities, conducted by researchers at the Center for World-Class Universities of Shanghai Jiao Tong University, is  highly respected by university heads worldwide, and its editors have agreed to publish it on the internet, thus complying with many requests from around the world. Since it launched in 2003, the website has recorded 2000 visits daily. Among the ranking criteria are the number of Nobel laureates in the university, and the number of its scientific publications and their quality. The list of 500 leading universities is headed once again by U.S. universities – Harvard, Stanford, MIT and Berkeley.

Fighting Superbugs

By Vincent Zurawski, PhD

Iron Chelator-Based Neurodegenerative Drug for Treatment of Wound Infections and MDR BacteriaOccasionally, a chance alignment of circumstances can lead to an unexpected and highly productive outcome, one with the potential to induce a sea change in a particular field of endeavor. A novel small molecule called VK28, developed by Prof. Emeritus Moussa Youdim and his colleagues at the Technion and the late Prof. Abraham Warshawsky at the Weizmann Institute of Science, found its way to an unexpected collaborative research program involving Clinical Research Management contract researchers at the Walter Reed Army Institute of Research (WRAIR) in Silver Spring, Maryland, and at Varinel, Inc., the company to which VK28 had been licensed for commercial development.

VK28 – also known as VAR10100 – was originally developed by Youdim, Varinel’s scientific founder, as a brain-selective and brain-permeable iron chelator, a chemical entity with the ability to bind up free iron. VK28 was designed to target treatment of neurodegenerative diseases such as Alzheimer’s, Parkinson’s and Huntington’s disease. An iron chelator penetrating the blood-brain barrier might be expected to bind up and remove free iron from brain cells, providing a neuroprotective effect by eliminating an important source of tissue-damaging free radicals that can be stimulated by the presence of iron. Free radicals are highly reactive and can severely damage and even kill the very cells we use to think. Indeed, Youdim and colleagues showed that, in two animal models of Parkinson’s disease, VK28 was not only neuroprotective, but also neurorestorative and lowered the toxic brain iron that accumulates.

“The emergence of MDR bacterial strains has become a significant challenge for clinicians and caregivers.” During the product development process aimed at advancing VK28 to clinical trials, Varinel also developed a VK28 derivative called VAR10103 with potential as a drug candidate.

As luck would have it, my son, Daniel Zurawski, is a principal investigator and contracted at WRAIR to develop new therapies and preventive medicines for wound infections, especially those involving multidrug-resistant (MDR) bacteria. The emergence of MDR bacterial strains has become a significant challenge for clinicians and caregivers of the U.S. military as wounded soldiers returning from Iraq and Afghanistan are often infected with bacteria that are resistant to most, if not all, current antibiotic treatment.

Because iron is an essential nutrient for all bacteria, including MDR bacteria, it was thought that treatment of wounds before or after a bacterial infection with an iron chelator alone or in conjunction with antibiotic therapy might provide the required antibacterial effect to keep infections in check. Sure enough, under the auspices of a Cooperative Research and Development Agreement (CRADA) between Varinel and the U.S. Army and Department of Defense, both VK28 and VAR10103 have proven effective, in the laboratory, at stopping bacteria in their tracks, and they have proven to be synergistic with certain antibiotics in targeting MDR-resistant organisms. These results were presented at the 2011 ICAAC Meeting in Chicago. At WRAIR, both compounds are now being tested in animal models of infection along with some other iron chelators that showed promise in vitro.

The same chemical entities with the potential to protect brain cells may also provide the means to deliver a knock-out punch to bacteria that would otherwise elude treatment. It all begins with the kind of solid scientific effort that is traditional at the Technion, and which has led to its world recognition.

Dr Vincent Zurawski is Founding President of Varinel and its Chief Scientific Officer.

Disclaimer: The findings and opinions expressed herein belong to the authors and do not necessarily reflect the official views of the WRAIR, the U.S. Army, or the Department of Defense.

An optical spin – the nanoscience of electrons

Prof. Erez Hasman, Technion.

The spin Hall effect – the impact of the intrinsic spin on the particle trajectory, which produces transverse deflection of the particle – is a central tenet in the field of spintronics regarding particles of electrons. Now, its optical equivalent has been observed.

The Magnus effect is seen in a wide range of systems. For example, it describes the sideways force applied to a spinning ball as it travels through the air explains Prof. Erez Hasman, head of the Micro- and Nanooptics Laboratory and an avid tennis player.

Light waves, comprising mass-less particles called photons, also demonstrate spin. Light’s spin is determined by its polarization: whether the wave vibration rotates in one direction or the opposite as it travels. Hasman, together with his PhD student Avi Niv, Dr Vladimir Kleiner – a senior scientist in the lab – and Ukrainian visiting scientist Dr Konstantin Bliokh, were the first to observe the effect of spin on the trajectories of polarized light beams.

The researchers launched a laser beam at a sliding angle to the internal surface of a glass cylinder. Once inside the cylinder the beam traveled in a helical trajectory along the glass-air interface, and was collected and analyzed at the far end using polarization optics and a camera. They observed a transverse spin-dependent deflection of the optical beam. These results have promising applications in nano-optics leading to much faster and more accurate computational data processing.

Physics Prof. Mordechai (Moti) Segev, a world leader in the area of Nonlinear Optics, comments, “Nanophotonics is a field where light is manipulated and controlled on a scale that is smaller than the optical wavelength. Erez Hasman has written a series of important papers in this area, leading to a new branch in optics – spinoptics. His discoveries offer an unprecedented ability to control light and its polarization state in nanometer-scale optical devices, thereby facilitating a variety of applications related to nanophotonics.”

Applied to other areas Hasman says, “There are a number of systems where the spin of a particle couples with its trajectory in high-energy and condensed matter physics. The math is the same in all cases, but experimentally it’s hard to understand what’s going on. Our experimental system offers a new way to get at some of these fundamental questions clearly and precisely.”

What is Photonics?

Photonics is the science of generating, controlling, and detecting photons. Photonics researchers investigate the emission, transmission, amplification, detection, and modulation of light. Applications include laser manufacturing, biological and chemical sensing, medical diagnostics and therapy, display technology, and optical computing.

Spinoptics: The Magnus effect for light, also called the optical spin Hall effect, causes the light to deflect due to the interaction between the intrinsic spin of the photons and the shape of the light’s trajectory.

All eyes on CORTICA as it raises +7 million

Cortica gets $7M to bestow computers with the power of sight.

Israeli startup Cortica raised $7 in a second round of funding. The investment was led by Horizons Ventures, owned by the Hong Kong billionaire Li Ka-Shing. Venture capitalist Ynon Kreiz also participated in the round, as did Ynon Kreiz, the former Chairman & CEO of the Endemol Group, the world’s largest independent television production company. The company had already raised $4 million from a group of high quality angel investors. Mr. Kreiz joins Cortica as its Chairman; founder Technion graduate Igal Raichelgauz continues as CEO.

Cortica’s image recognition technology fuses neuroscience and computer science by imbuing computers the ability to comprehend visual content on the web in real-time. The core technology was developed at Technion, in Haifa, Israel, by a team of neuroscientists and digital media experts. The technology functions similarly to the human cortex and can identify patterns, and make classifications.

Cortica was conceived in 2006 with a vision to fundamentally revolutionize the way computers understand images and video. The essence of Cortica’s Image2Text™ technology lies in its ability to automatically extract the core concepts in images and video, and map these concepts to key-words and textual taxonomies.

By virtue of their ability to simulate the appearance of the physical world, pictures drive interest, sentiment and commercial intent. Cortica’s product reads and automatically associates images with relevant ads. This groundbreaking model gives publishers a completely new monetization stream and provides brands and marketers the opportunity to reach highly targeted mass audiences.

Cortica was founded by Technion researchers, Prof. Yehoshua (Josh) Zeevi (Faculty of Electrical Engineering); Karina Odinaev; and engineer Igal Raichelgauz, who assembled a multi-disciplinary team of neuroscience researchers, digital multimedia experts and veterans of Israeli military intelligence to develop and commercialize the technology. The underlying technology was derived from scientific research focused on understanding how neural networks of the human cortex perform complex computational tasks, such as identifying patterns, classifying natural signals and understanding concepts.

Cortica’s commercial team will relocate to the US and will be based in NY with an office in Silicon Valley. The first commercial applications of the technology will transform advertising, search and image analytics

The Technion has an unmatched position within (and outside) Israel in neuro-engineering, and is recognized as a leader in the application of engineering methods and principles to the study of neural systems. Indeed, in addition to Cortica, several successful companies have stemmed from the Technion’s activity in this field (e.g., “BrainsGate”, “Elminda”, “GeneGrafts”, “Neurovibes”).

Cortica