Some Earth Day events involve volunteer clean-ups, planting flower beds or unplugging your gadgets for a day. The Technion-Israel Institute of Technology, however, challenged high school students worldwide to build an Earth Day-themed Rube Goldberg Machine—and three schools came through with flying colors (all shades of green, of course).
For the uninitiated, a Rube Goldberg Machine is a wacky contraption that is deliberately over-engineered to perform a simple task by setting off a comical chain reaction. Katz Yeshiva High School of South Florida (whose team of students ranged from 9th through 11th grades), placed first in this fun but difficult challenge, winning a one-year full scholarship to the Technion.
“When I saw our school’s name appear on the screen, I was overcome with emotions of comradery and school spirit. Tens of hours of hard work had finally paid off,” said 11th grader Tani Loskove. Teammate Ty Kay added: “As a high schooler pursuing dreams of becoming an engineer, Technion’s Rube Goldberg Earth Day Challenge was a great stepping stone for me. It was my first real engineering ‘project,’ and it taught me about teamwork, advanced preparation and the differences between theoretical and applied sciences.” Other students include: Noah and Joshua Bernten, Michal Amar and Max Davis.
Working out of a student’s garage, the winning team concocted a nearly one-minute chain reaction in which Coca Cola bottles activated a toy car, sent a ball down a winding slide worthy of a Water Park, releasing liquid gallium to complete a circuit, which eventually set off a stream of water that spun a home-built Ferris wheel, knocked down popcorn boxes that activated a fan—illustrating recycling (plastic, metal and paper) and alternative energy sources (hydro, solar and wind power).
Some 24 high school teams around the world participated in the challenge. A team of judges led by Prof. Alon Wolf, Director of the Biorobotics and Biomechanics Lab at the Technion’s Faculty of Mechanical Engineering, selected the winners based both on their creative renditions of Earth Day themes and the complexity of the energy transfers from one action to the next. Le Hong Phong High School for the Gifted in Ho Chi Minh City, Vietnam placed second, while the International Bilingual School at Hsinchu-Science-Park in Taiwan came in third. (Prof. Wolf is no relation to Dr. Wolf).
“It’s very exciting for us,” says Dr. Yosef Wolf, who heads up Katz Yeshiva’s STEM courses and started a robotics club at the school. “It’s the first engineering contest that we have ever entered, and we came in 1st place! We’ve been growing our engineering opportunities recently and we have plans to further expand our STEM offerings next year. This result has further helped to increase our students’ excitement of the upcoming initiatives.”
Note: the contest is not over yet! May 1st is the deadline for the Most Popular Clip Contest. Winners will be announced May 3rd.
“Technology is an integral part of the medical world”
The Ophek VR team won first place in the 3DS Conference at the Technion – after 48 hours of linking entrepreneurship, research, engineering and medicine
By Keren-Or Grinberg
The corridors of the Rappaport Faculty of Medicine at the Technion were transformed into a miniature technological accelerator during the three days of the 3DS Conference. On the last day of the conference, students presented their developed ideas to the judges to compete for the top three places. The winning team, Ophek VR, received a ticket to the annual Entrepreneurship Conference in Boston. Teams placing 1st and 2nd automatically enter the BizTEC Entrepreneurship Contes; and all three winning teams receive USD 50,000 from the AMIT Incubator at Technion.
This is the fourth year of the conference, which is supported by the Faculty of Medicine and the Technion Faculty of Biomedical Engineering, under the auspices of the joint MED2E Center. The purpose of the conference is to foster cooperation between medical students, interns and residents, as well as students from other Technion faculties and from the Faculty of Biomedical Engineering in particular. The participants have to find a technological and business solution to significant problems in the medical world, within just 48 hours.
This year, 150 students applied to the competition, and 80 were selected. The students were mentored by experts from the medical, industrial and academic sectors. The mentors included Clinical Assistant Professor Ronen Jaffe, Director of the Catheterization Unit, Department of Cardiology, Carmel Medical Center; Lena Levin, founder and CFO of Via Surgical Ltd; Dr. Yoav Medan, technological director at Ninispeech Ltd; the organizer of Haifa Digital Health and many others.
The conference opened at the Faculty of Biomedical Engineering, with the participation of the Dean of the Rappaport Faculty of Medicine Prof. Shimon Marom, and the Dean of the Faculty of Biomedical Engineering Prof. Shulamit Levenberg. Prof. Yaron Har-Shai, Deputy Dean of Strategic Development at the Faculty of Medicine, and Prof. Amir Landesberg, Director of Entrepreneurship and Industrial Relations at the Faculty of Biomedical Engineering, also played an active role. Edna Lazar, Business Development Manager at the MEDX Incubator, spoke during the evening and offering a variety of tips and advice on the process, from the concept stage to product development.
On the second day, once the teams had been formed and the concepts formulated, the participants visited the various departments of Rambam Healthcare Campus and Carmel and Bnei Zion hospitals. “This is an amazing experience that we do not have in our engineering studies” said Yonatan, a member of LaborLane. “You walk around the hospital and learn from the doctors about problems that can be solved by technological means.”
On the last day, the final 11 ideas were presented to the judges, including EasyVisit – a navigation compass and information system for patients in a hospital emergency room; RedFlag and Digitest – methods for relatively friendly gastrointestinal examination and colorectal cancer diagnosis; DentUS – a unique combination of ultrasound and acid for root canal treatment; TipOff – an ear examination for children with earwax buildup; Heart ‘n’ Sole – daily self-monitoring of heart failure; and MamaRoo – a sensor belt that learns the mother’s heart rate and conveys the sensation to the baby in the mother’s absence.
The winning team, Ophek VR, developed a system for eye examinations for children using virtual reality and an image analysis algorithm. The system was born from the personal experience of one of the team members, Igor Weiner, a medical student who works as an intern in Rambam’s Department of Ophthalmology. “We never imagined that we’d win,” he said, “We came for the experience.”
Second place went to Soundfit – a product that will enable a radiologist to hear in real time what patients with hearing problems hear, for optimal and fast fitting of hearing aids. The product was presented by Dalia Orbach, a hard-of-hearing graduate student at the Viterbi Faculty of Electrical Engineering.
Third place went to LaborLane – a pressure sensor belt that enables the identification of prenatal labor. The idea, which arose during a visit to the Department of Gynecology at Rambam, was designed to reduce the unnecessary use of drugs to induce labor. “I see technology as an integral part of medicine, and as a medical student it was an experience to get to know the entrepreneurial side of the competition,” said Nitzan Halamish, who participated in the team together with her husband Omer, a mechanical engineering student at the Technion.
Technion 3DS 2017
Prof. Wayne Kaplan, Technion Executive Vice President for Research, said that judging the competition was difficult because of the high quality of the products. He said: “We saw promising ideas that aroused great enthusiasm. The Technion as a research institute believes that science is the future and without it there will be no technology 10 years from now. We must invest in human resources because the State of Israel does not have oil and water resources. Our goal is to provide researchers and students with tools for creating an industry with ideas from researchers and students, through processes such as BizTEC and accelerators such as AMIT and The Drive.”
Prof. Kaplan’s colleagues on the judging panel were Mor Research Applications Ltd CEO Pini Ben-Elazar; entrepreneur and physician Dr. Dalia Megiddo; Yael Wiesel, founder and CEO of the startup Zikit; Gsap Ltd founder Dr. Sigalit Ariely Portnoy; Dr. Orna Blondheim, Director of Haemek Medical Center in Afula; and Prof. Amir Landesberg, outgoing Dean of the Faculty of Biomedical Engineering.
This year, the conference was organized by medical students Shirel Levanon, a fourth year student at the Faculty of Medicine, and Guy Barshadsky, who participated in the conference last year. The two follow in the footsteps of Bar Rinot and Yuval Barak, who brought the competition to the Technion in 2013.
The Strage Foundation and Ben-Gurion University of the Negev are pleased to announce this year’s winner of the Strage-BGU Award for Excellence in Environmental Sciences: Prof. Matthew Suss of the Technion, Israel Institute of Technology
Prof. Matthew Suss of the Technion, Israel Institute of Technology
The Award is given for the article: “Water desalination via capacitive deionization: what is it and what can we expect from it?”, published in the journal Energy and Environmental Science.
The winning article critically reviews and evaluates the current state-of-the-art of Capacitive Deionization (CDI) technology and provides definitions and performance metric nomenclature in an effort to unify the fast-growing CDI community. It provides an outlook on the emerging trends in CDI and proposes future research and development directions.
The Strage Foundation and Ben-Gurion University of the Negev founded the Strage-BGU Award for Excellence in Environmental Sciences.
The award, of $5,000, is granted to israeli scientists and environmental practitioners for an outstanding and influential publication (a paper in a peer-reviewed journal) in any of the diverse fields of environmental sciences (e.g., ecology, hydrology, sustainability, environmental economics, environmental engineering, environmental education, environmental epidemiology) published in the in the past year. Special emphasis is given to the paper’s potential to make a long-term impact on protecting our environment and ecosystem.
The Award committee:
Prof. Orit Ben-Zvi Assaraf, Faculty of Humanities and Social Sciences
Prof. Asher Brenner, Faculty of Engineering Sciences
Dr. Rachel Golan, Faculty of Health Sciences
Dr. Yodan Rofè, Jacob Blaustein Institutes for Desert Research
Dr. Anat Tchetchik, Guilford Glazer Faculty of Business & Management
Prof. Yaron Ziv, Faculty of Natural Sciences (Chairperson)
An Israeli – Japanese conference on regenerative medicine and medical uses of stem cells was held at the Technion on Tuesday, March 14th. The conference took place in the Rappaport Faculty of Medicine and was aimed at strengthening the relations between the Technion and Kyoto University.
Professor Kenji Osafune from the CiRA Center of Kyoto University.
The conference opened with a lecture by Professor Kenji Osafune from the CiRA Center of Kyoto University. Professor Osafune, who completed his PhD In medicine at Kyoto University and his post-doctorate at Harvard, explained that this was his first visit to the Technion and that he hoped the conference would bring about cooperation between the Technion and Kyoto University.
More than a quarter of a million people in Japan are currently on dialysis, and the total cost of treating chronic kidney diseases in Japan amounts to 13 million dollars annually. Professor Osafune is therefore developing methods for growing tissues from pluripotent stem cells, or hiPSCs. This approach involves the production of mature tissues from embryonic stem cells derived from available mature cells in the patient’s body such as skin cells. In other words, cells that are taken from the patient’s body are transformed into embryonic stem cells that can potentially transdifferentiate into any type of tissue. Professor Osafune is transforming these cells into kidney cells that can be transplanted. Since the donor and recipient are the same person, this approach does not result in rejection of the cells. Professor Osafune explained: “We can currently produce a few cells, and soon we will be able to produce tissue. My dream is to produce in the laboratory an entire kidney that can be transplanted.”
Professor Lior Gepstein of the Technion explained that the Technion is a leader in research in the field of hiPSCs, and that this area could provide common ground for ties with Kyoto University. Professor Gepstein, a faculty member of the Rappaport Faculty of Medicine in the Technion and Director of the Cardiology Department at Rambam Health Care Campus, presented before Professor Osafune and the other conference participants the latest developments from his laboratory in the field of cardiac rehabilitation. His work is also based on the hiPSCs approach in which body cells are transformed into embryonic stem cells from which heart cells are produced.
The heart is an organ in which cells do not regenerate if the heart is damaged, and an incident such as a myocardial infarction is liable to destroy a billion cells – a quarter of the heart’s total cell population. Most of the technologies that Professor Gepstein is developing are based upon the following process: Available cells, such as skin cells, are taken from the patient. These cells are transformed in the laboratory into unique stem cells known as induced stem cells. These cells are used to produce the designated heart cells that are required (myocardial cells, pacemaker cells etc.), which are then transplanted in the patient’s heart. Since these heart cells come from the patient, the method eliminates the problem of organ rejection that is characteristic of transplants. Professor Gepstein has successfully demonstrated this technique in several channels, including transplanting pacemaker cells in patients suffering from cardiac arrhythmias and transplanting atrial tissue to treat patients with ventricular fibrillation. This approach, which is based on research that earned the Japanese researcher Shinya Yamanaka a Nobel Prize, holds immense potential from the standpoint of treatment as well as cardiac research. One of the latest developments in Gepstein’s laboratory is synchronization of cardiac rhythm using light to control cardiac cells (optogenetics).
Other researchers from the Technion, including Dr. Yuval Avni, Professor Ofer Binah, Professor Shulamit Levenberg, and Assistant-Professor Ruby Shalom- Feuerstein, also presented their research at the conference.
Technion researchers have helped the Ministry of Environmental Protection reduce fuel leaks into the ground by tens of thousands of cubic meters. This represents a dramatic reduction of the health risk inherent in the seepage of fuel into groundwater.
תחנת דלק
Fuel is an essential resource for the existence of modern life, yet it is also a dangerous poison; its penetration into the body can be hazardous to health in various ways, including cancer and damage to the nervous system, the immune system, fertility, the liver, kidneys and red blood cells. Therefore, Israel, like other countries, has set limits on the quantity of fuel components in drinking water.
Fuel components reach drinking water as a result of leaks from damaged pipes and fuel tanks. Fuel seeping into the soil and groundwater (aquifer) is liable to reach the human body not only from drinking water but also from the inhalation of fuel vapor in the soil and from contact with contaminated soil. Benzene, one of the hazardous materials in fuel, does not decompose quickly and is liable to migrate in the soil up to a distance of a few dozen meters, thereby reaching open public spaces and even private gardens.
Because of these risks, the Ministry of Environmental Protection promulgated the Water Regulations in 1997, which focus on the prevention of water pollution by fuel. In addition, the Ministry contacted experts from the Davidson Faculty of Industrial Engineering and Management at the Technion, requesting the following services: monitoring the impermeability of the underground infrastructure (fuel tanks and pipes) at gas stations and ensuring that they are in good condition, monitoring the repair and replacement of facilities and devices found to be leaking, and monitoring and providing engineering solutions with regard to the various testing methods. Indeed, over the past 16 years, following the implementation of the Technion experts’ recommendations, there has been a dramatic reduction in the total volume of fuel leaks in Israel, mostly leaks from gas stations. The Ministry estimated that thanks to these measures, the total leakage was reduced by 3,500 cubic meters per year at the very least (over the past 12 years), thanks to the monitoring and servicing of underground tanks and underground pipes at gas stations. In other words, within 12 years, leaks were reduced by a total of more than 42,000 cubic meters.
The Technion research team, whose members include Prof. Dov Ingman (team leader), Dr. Chaim Michlin, and Ms. Yelena Leschenko, recently submitted a final report reviewing the developments over the past 16 years. According to the findings, despite the considerable improvement there are still leaks that jeopardize the soil and groundwater. Therefore, the researchers recommend expanding the activity to include the national fuel pipeline and particularly large overhead tanks.
The monitoring system developed at the Technion includes fuel leakage monitoring, data collection and analysis, monitoring the reliability and integrity of tanks and pipelines, and employing statistical means to predict future defects in the infrastructure. The ongoing study has already led to the publication of eight articles in professional journals, and the information accumulated is already being used in courses taught by Dr. Michlin (Reliability Management and Reliability Engineering) at the Technion, and in other studies conducted by the research team. Inter alia, innovative testing methods developed on the basis of the study as part of Ofer Shaham’s doctoral work have been implemented in the national standard, and an international committee is currently deliberating the implementation of the methods in the ISO (International Standard Organization) standards.
Now, in light of the above interim study and in preparation for the expansion of the study in the years ahead, the Ministry of Environmental Protection sent a letter to Prof. Wayne Kaplan, Executive Vice President for Research at the Technion. The letter, signed by Dr. Arie Pistiner, the official in charge of the prevention of water pollution from fuel, stated that: “The work done so far by the Technion team has been very helpful to our Ministry in promoting the issue of preventing leaks from fuel pipelines and tanks into soil and groundwater, and has contributed greatly to environmental protection in Israel.”
Dr. Pistiner, an alumnus of the Technion Faculty of Civil And Environmental Engineering, did his doctorate on precisely this subject: Migration of Fuel Pollutants in Groundwater.
The study was supported in its early stages by the Chief Scientist of the Ministry of Environmental Protection, and later by the Division of Industrial Waste, Fuel and Polluted Soil at the Ministry.
An international group of researchers from the United States, the Technion, Italy and Greece presents a new explanation for the emission of energy and winds from black holes.
Nature Astronomy recently published a new model, which explains the typical phenomena surrounding black holes: plasmatic outflows (winds). The article has been submitted by researchers from the US, Israel, Italy, and Greece.
Prof. Ehud Behar
The existence of black holes as a consequence of enormous gravity that prevents light from escaping had already been sighted at the end of the 18th century by the clergyman and philosopher John Mitchell and the mathematician Pierre-Simon Laplace. But it was only Albert Einstein’s theory of general relativity laid an ordered foundation for the phenomenon. At the beginning of the 20th century the physicist Karl Schwarzschild discovered the solution for the black hole in Einstein’s equations, though Einstein himself refused to believe in the existence of black holes in nature. Since then, there have been dramatic developments in this area, particularly a burgeoning of observable, clear and conclusive evidence for the existence of black holes.
One of the surprising phenomena observed in black holes is the strong winds that blow in their vicinity. The nature of these winds and the power, which drives them are explained in many ways, one of which describes them as a result of strong and regulated magnetic fields. The present article shows that the model of magnetic winds explains not only winds from super-massive black holes at the centers of galaxies, but also winds deriving from small black holes, whose mass resembles that of stars.
In popular culture, black holes are described as all-consuming creatures: anything that approaches them, even light, is swallowed up and gone forever. Although that is their nature, the astrophysical truth is more complex. It turns out that black holes do not only swallow anything and everything in their path, but also emit radiation and plasmatic winds (physical plasma is hot gas whose electrons have been torn out of their atoms). Plasmatic winds, which can travel at speeds ranging from 100 km per second to a fraction of the speed of light (over 30,000 km per second), dramatically affect the surroundings of the black hole and the entire galaxy.
What are black holes?
Black holes are created when a relatively heavy star (a few solar masses) collapses into itself as a result of the loss of its nuclear fuel and the prevailing of self gravity. Remaining at the end of this collapse is a “singularity” – a tiny dot of tremendous mass. This gluttonous being feeds on gas which it pulls from neighboring stars, and since even the light it swallows cannot escape it, it is called “a black hole”.
Separating the black hole from its surroundings is a [nearly] spherical boundary called an “event horizon”. Anything that passes the event horizon towards the center of the black hole is indeed swallowed forever, yet outside the event horizon, part of the “stolen gas” taken from the neighboring stars moves inward in the shape of a disk. This “accretion disk” emits a large quantity of light, especially in the form of X-rays – which is why these systems are sometimes called X-Ray-Binaries – as well as strong plasmatic winds and jet streams. This process in giant black holes at the centers of galaxies shapes the largest structures in the universe – galaxies and galaxy clusters – and generates a significant part of the ionizing radiation in the universe.
In an article that is currently being published by Prof. Ehud Behar from the Technion’s Faculty of Physics, along with Prof. Keigo Fukumura from James Madison University and their associates, the researchers present evidence found by observation that the magnetic field created around the black hole fills a crucial role in the creation of the accretion disk and the winds it diffuses. The article attributes the magnetic-hydrodynamic model developed by the researchers to all black holes, including massive ones. The fact that the same description matches black holes of different sizes, be it one solar mass or billions of solar masses, attests to a general fundamental magnetic structure which exists around black holes.
The model described in the article has been examined in great detail by mapping the absorption spectrum of X-rays emitted by various atoms found in the plasmatic wind. Prof. Behar, who led the study’s spectroscopic analysis, explains that “absorption spectroscopy, which we map according to its kinetic features (shift to blue), provides us with extensive, in-depth physical properties regarding the composition of the wind and the energy surrounding the black hole. These allow us to quantitatively map the density of the wind, its level of ionization, its energy and its speed. The fact that a relatively simple model of magnetic wind describes the observed wind, with all its complexity, and better than any other outflow theory makes it a highly successful model.”
In summary, the article published in Nature Astronomy sheds light on the “behavior” of black holes, and particularly on the mechanisms of their influence on their surroundings. The magnetic hydrodynamic model is found to be valid for black holes in every size-scale – from 10M☉ to 109M☉ (M☉ = solar mass).
Figure 1: Illustration of wind blowing from accretion disk in GRO J1655-40 (Credit: NASA/CXC/A.HOBART) Mass moves from a neighboring star to the black hole in the accretion disk which emits X-rays. During the accretion towards the black hole some of the mass is lost in the wind. The model developed by Fukumura, Behar, and others in the past explains these processes by a magnetic model. Credit: “An artist’s impression of the “magnetically-driven disk-wind made by K. Fukumura using the BINSIM visualization code by R. Hynes (LSU)”
Researchers from the Technion Computer Science Department introduce unprecedented theoretical foundation to one of the hottest scientific fields today – deep learning
In a recent article, Prof. Elad and his PhD students, Vardan Papyan and Yaniv Romano introduce a broad theory explaining many of the important aspects of multi-layered neural networks, which are the essence of deep learning.
Initial seed ideas in the 1940s and 1950s, elementary applications in the 1960s, promising signs in the 1980s, a massive decline and stagnation in the 1990s, followed by dramatic awakening development in the past decade. This, in a nutshell, is the story of one of the hottest scientific fields in data sciences – neural networks, and more specifically, deep learning.
Deep learning fascinates major companies including Google, Facebook, Microsoft, LinkedIn, IBM and Mobileye. According to Technion Professor Michael Elad, this area came to life in the past decade following a series of impressive breakthroughs. However, “while empirical research charged full speed ahead and made surprising achievements, the required theoretical analysis trailed behind and has not, until now, managed to catch up with the rapid development in the field. Now I am happy to announce that we have highly significant results in this area that close this gap.”
In a recent article, Prof. Elad and his PhD students, Vardan Papyan and Yaniv Romano, present, for the first time, a broad theory that explains many of the important aspects of multi-layered neural networks, which are the essence of deep learning.
“One could say that up to now, we have been working with a black box called a neural network,” Elad explains. “This box has been serving us very well , but no one was able to identify the reasons and conditions for its success. In our study, we managed to open it up, analyze it and provide a theoretical explanation for the origins of its success. Now, armed with this new perspective, we can answer fundamental questions such as failure modes in this system and ways to overcome them. We believe that the proposed analysis will lead to major breakthroughs in the coming few years.”
But first a brief background explanation.
*(Multi-layered) Neural Networks
Convolutional neural networks, and more broadly, multi-layered neural networks, pose an engineering approach that provides the computer with a potential for learning that brings it close to human reasoning. Ray Kurzweil, Google’s chief futurist in this field, believes that by 2029 computerized systems will be able to demonstrate not only impressive cognitive abilities, but even genuine emotional intelligence, such as understanding a sense of humor and human emotions. Deloitte has reported that the field of deep learning is growing at a dizzying rate of 25% per year, and is expected to become a 43 billion USD industry per year by 2020.
Neural networks, mainly those with a feed-forward structure that are currently at the forefront of research in the fields of machine learning and artificial intelligence, are systems that perform rapid, efficient and accurate cataloging of data. To some extent, these artificial systems are reminiscent of the human brain and, like the brain, they are made up of layers of neurons interconnected by synapses. The first layer of the network receives the input and “filters” it for the second, deeper layer, which performs additional filtering, and so on and so forth. Thus the information is diffused through a deep and intricate artificial network, at the end of which the desired output is obtained.
If, for example, the task is to identify faces, the first layers will take the initial information and extract basic features such as the boundaries between the different areas in the face image; the next layers will identify more specific elements such as eyebrows, pupils and eyelids; while the deeper layers of the network will identify more complex parts of the face, such as the eyes; the end result will be the identification of a particular face, i.e., of a specific person. “Obviously the process is far more complex, but this is the principle: each layer is a sort of filter that transmits processed information to the next layer at an increasing level of abstraction. In this context, the term ‘deep learning’ refers to the multiple layers in the neural network, a structure that has been empirically found to be especially effective for identification tasks.
The hierarchical structure of these networks enables them to analyze complex information, identify patterns in this information, categorize it, and more. Their greatness lies in the fact that they can learn from examples, i.e. if we feed them millions of tagged images of people, cats, dogs and trees, the network can learn to identify the various categories in new images, and do so at unprecedented levels of accuracy, in comparison with previous approaches in machine learning.”
The first artificial neural network was presented by McCulloch and Pitts in 1943. In the 1960s, Frank Rosenblatt from Cornell University introduced the first learning algorithm for which convergence could be proven. In the 1980s, important empirical achievements were added to this development.
It was clear to all the scientists engaged in this field in those years that there is a great potential here, but they were utterly discouraged by the many failures and the field went into a long period of hibernation. Then, less than a decade ago, there was a great revival. Why? “Because of the dramatic surge in computing capabilities, making it possible to run more daring algorithms on far more data. Suddenly, these networks succeeded in highly complex tasks: identifying handwritten digits (with accuracy of 99% and above), identifying emotions such as sadness, humor and anger in a given text and more.” One of the key figures in this revival was Yann LeCun, a professor from NYU who insisted on studying these networks, even at times when the task seemed hopeless. Prof. LeCun, together with Prof. Geoffrey Hinton and Prof. Yoshua Bengio from Canada, are the founding fathers of this revolutionary technology.
Real Time Translation
In November 2012, Rick Rashid, director of research at Microsoft, introduced the simultaneous translation system developed by the company on the basis of deep learning. At a lecture in China, Rashid spoke in English and his words underwent a computerized process of translation, so that the Chinese audience would hear the lecture in their own language in real time. The mistakes in the process were few – one mistake per 14 words on average. This is in comparison with a rate of 1:4, which was considered acceptable and even successful several years earlier. This translation process is used today by Skype, among others, and in Microsoft’s various products.
Beating the World Champion
Google did not sit idly by. It recruited the best minds in the field, including the aforementioned Geoffrey Hinton, and has actually become one of the leading research centers in this regard. The Google Brain project was established on a system of unprecedented size and power, based on 16,000 computer cores producing around 100 trillion inter-neuronal interactions. This project, which was established for the purpose of image content analysis, quickly spread to the rest of the technologies used by Google. Google’s AlphaGo system, which is based on a convolutional neural network, managed to beat the world champion at the game of Go. The young Facebook, with the help of the aforementioned Yann LeCun, has already made significant inroads into the field of deep learning, with extremely impressive achievements such as identifying people in photos. The objective, according to Facebook CEO Mark Zuckerberg, is to create computerized systems that will be superior to human beings in terms of vision, hearing, language and thinking.
Today, no one doubts that deep learning is a dramatic revolution when it comes to speed of calculation and processing huge amounts of data with a high level of accuracy. Moreover, the applications of this revolution are already being used in a huge variety of areas: encryption, intelligence, autonomous vehicles (Mobileye’s solution is based on this technology), object recognition in stills and video, speech recognition and more.
Back to the Foundations
Surprisingly enough, however, the great progress described above has not included a basic theoretical understanding that explains the source of these networks’ effectiveness. Theory, as in many other cases in the history of technology, has lagged behind practice.
This is where Prof. Elad’s group enters the picture, with a new article that presents a basic and in-depth theoretical explanation for deep learning. The people responsible for the discovery are Prof. Elad and his three doctoral students: Vardan Papyan, Jeremias Sulam and Yaniv Romano. Surprisingly, this team came to this field almost by accident, from research in a different arena: sparse representations. Sparse representations are a universal information model that describes data as molecules formed from the combination of a small number of atoms (hence the term ‘sparse’). This model has been tremendously successful over the past two decades and has led to significant breakthroughs in signal and image processing, machine learning, and other fields.
So, how does this model relates to deep neural networks? It turns out that the principle of sparseness continues to play a major role, and even more so in this case. “Simply put, in our study we propose a hierarchical mathematical model for the representation of the treated information, whereby atoms are connected to each other and form molecules, just as before, except that now the assembly process continues: molecules form cells, cells form tissues, which in turn form organs and, in the end, the complete body – a body of information – is formed. The neural network’s job is to break up the complete information into its components in order to understand the data and its origin.
Papyan and Sulam created the initial infrastructure in two articles completed in June 2016, while in the follow-up work Papyan and Romano diverted the discussion to deep learning and neural networks. The final article, as noted, puts forward the theoretical infrastructure that explains the operating principles of deep neural networks and their success in learning tasks.
“We can illustrate the significance of our discovery using an analogy to the world of physics,” says Prof. Elad. “Imagine an astrophysicist who monitors the movement of celestial objects in search of the trajectories of stars. To explain these trajectories, and even predict them, he will define a specific mathematical model. In order for the model to be in line with reality, he will find that it is necessary to add complementary elements to it – black holes and antimatter, which will be investigated later using experimental tools.
“We took the same path: We started from the real scenario of data being processed by a multi-layered neural network, and formulated a mathematical model for the data to be processed. This model enabled us to show that one possible way to decompose the data into its building blocks is the feed-forward neural network, but this could now be accompanied by an accurate prediction of its performance. Here, however, and unlike the astrophysical analogy, we can not only analyze and predict reality but also improve the studied systems, since they are under our control.”
Prof. Elad’s emphasizes that “our expertise in this context is related to handling signals and images, but the theoretical paradigm that we present in the article could be relevant to any field, from cyberspace to autonomous navigation, from deciphering emotion in a text to speech recognition. The field of deep learning has made huge advances even without us, but the theoretical infrastructure that we are providing here closes much of the enormous gap between theory and practice that existed in this field, and I have no doubt that our work will provide a huge boost to the practical aspects of deep learning.”
About the Doctoral Students
When Vardan Papyan completed his master’s degree, supervised by Prof. Elad, he didn’t intend to continue studying towards a PhD. However, during the final MSc exam, the examiners determined that his work was almost a complete doctoral thesis. After consulting with the Dean of the Computer Science Faculty and the Dean of the Technion’s Graduate School, it was decided to admit him to the direct Ph.D. track with the understanding that he would complete his doctorate within less than a year.
Yaniv Romano, a student in the direct Ph.D. track, has already won several prestigious awards. In the summer of 2015, he spent several months as an intern at Google Mountain View, USA, and left an unforgettable impression with his original solution to the single-image super-resolution problem, which is being considered for several of Google’s products.
For the uninitiated, a Rube Goldberg Machine is a wacky contraption that is deliberately over-engineered to perform a simple task by setting off a comical chain reaction. Katz Yeshiva High School of South Florida (whose team of students ranged from 9th through 11th grades), placed first in this fun but difficult challenge, winning a one-year full scholarship to the Technion.
