Mobile Phone Application for the Blind

 Seeing by Hearing

Technion Students Develop Seeing Application for the Blind

The invention is based on a Kinect camera, mobile phone application and headphones

Three undergraduate students from the Faculty of Electrical Engineering at the Technion developed a seeing application for the blind, based on a Kinect camera, mobile phone application and headphones.

The application is based on images from the surrounding area obtained from the depth camera, which helps the blind navigate inside a building, warning them of obstacles through voice indication, while identifying and directing them to studied objects in the room.

The idea for the project was instigated by an undergraduate student, Tzahi Simkin. “The idea came to me while I was driving, where right before me I saw a blind man having trouble crossing the road. I thought to myself that if I could only describe to him, through technological means, a snapshot of the surrounding area, I would make it much easier for him and build his confidence in getting better oriented with his surroundings. I wanted to combine technological development with social assistance, and this is how this product was born; it connects a depth camera and cellular application, and integrates two different technological systems.”

Koby Kohai with students Tzahi Simkin and Gal Dalal

Tzahi partnered up with two undergrads, Gal Dalal and Danny Zilber, and together the trio began working on the project. “The technological advantage of the Kinect camera lies in its ability to take very good depth images and that it is relatively cheap,” added Tzahi. “This field is continually evolving, with cameras becoming smaller and less expensive all the time. Our project connected the depth images received from a smart phone application, to guide the blind within a given space.”

“The camera sits on a belt and takes depth images of the surrounding area,” explains Gal. “The wireless device processes the information received from the camera and gives a voice indication to the user through the application. The application we developed helps blind people navigate inside a building, warning them of obstacles through voice indication that identifies studies objects and directs the user to them. Studied objects refer to items such as keys or handbag that the application is previously programmed to recognize. In other words, there is an element of recognition and learning.”

“When there is an obstacle before someone who is blind, the application warns him/her to stop and directs them right or left to bypass the barrier,” says Tzahi.” We haven’t yet tested the product with blind people, but we tested it ourselves when blindfolded, and it worked. Recently, we contacted the Association for the Blind in Haifa, so that we could test the application on site from people who are blind, our end users, and obtain feedback for needed improvements.”

Koby Kohai, who heads the Control Robotics & Machine Learning Laboratory at the Faculty of Electrical Engineering, guided and mentored the students throughout the project. “The project received a grade of 100 and has been submitted for a competition for outstanding faculty projects,” said Kohai. “The project was initiated by students, and I instructed and steered them towards technologies currently available on the market. The concept of the project was to test a technological concept that could in the future integrate from a technological standpoint, developing hardware into something more advanced. Every year we suggest ideas for project development to our undergraduate students, coming from industry or research of graduate students at the Faculty. We do our best to provide students with a broad space with which to encourage their creativity and their ideas in their chosen projects.”

“From my point of view, we succeeded with this project,” summarizes Tzahi. “I am interested in continuing to develop the product, so that it can be used by the blind one day. Our motivation for the project was to help those who needed help. There are over 150 million blind and visually impaired people in the world, yet the number of technological solutions offered to them today is very limited. Despite advances in technology, the best means of guidance remains through a Seeing Eye cane or Guide Dog. Our product is not yet perfected, and we intend to continue to develop it.”

Pictures: Koby Kohai with students Tzahi Simkin and Gal Dalal.

Photographed by: Shiatzo Photography Services, Technion’s Spokesperson’s Office