Low Vision Assistive Device by OrCam

A new low vision assistive device for those with macular degeneration and other eye conditions has been developed by a company called OrCam. Unlike any other low vision technology that's available, this device is mounted onto one's existing prescription glasses. It works by sensing what is in front of the user and then speaks to them. Find out how this device works and who can benefit from it by reading our e-interview with OrCam.

1. Tell us about OrCam and how you came to develop these “high tech glasses.” Take me from idea to product. How do these "talking glasses” work?

OrCam is an intuitive portable device with a smart camera mounted on the frames of the user’s eyeglasses that harnesses the power of Artificial Vision to assist the visually impaired. The OrCam device recognizes text and products, and speaks to the user through a bone-conduction earpiece.

The device can be mounted on most eyeglass frames, and utilizes bone-conduction technology to convey sound without blocking the user’s ears. We combined a standard camera module with a powerful image processor, but it’s the brains behind the device that really make it work. We are able to analyze the images, figure out what’s going on and verbalize it to the user instantly.

It’s powered by the ShareBoost proprietary algorithm which was developed by our founder, Prof. Amnon Shashua, our Senior VP of R&D, Dr. Yonatan Wexler and their colleague, Dr. Shai Shalev-Shwartz. The ShareBoost method offers a reasonable trade-off between recognition accuracy and speed by actually minimizing the amount of additional computer power required with each new product it learns to recognize.

The OrCam device is small and portable, so our challenge became a real-world challenge, where text and products are three-dimensional. Lighting conditions are constantly changing, surfaces are curved, bendable, movable… but we can still read off of them in about 2 seconds. OrCam works in the real world. Instantly.

2. Please explain how the device is personalized – what is so revolutionary about the ability to teach it to recognize a product?

We thought it was important to provide the user with a personalized experience. The OrCam device comes pre-installed with the ability to read.

It can recognize money notes and some other basic items. You can also teach it new things that are important to you, such as the contents of your pantry.

With facial recognition, the device won’t have to store 7 billion faces in its memory, rather, the user will teach it the people they care about - family, friends, neighbors, doctors, etc.

Today you can teach it dozens of products, and we’re working on the public transportation function which will identify the number of an approaching bus. Our vision is that OrCam will help you make it to the supermarket by finding the street, finding the right bus, locating the product you want and paying for your purchase. You teach it the things you care about. Unlike some other technological products which utilize cameras, many of which rely on scanning the internet for information, with OrCam you don’t need to search through every bit of information out there.

We designed it to be robust, to last the whole day, and not to be dependent on anything external. Because everything happens inside, it’s very quick. It doesn’t overload you with unnecessary information - it waits for you to tell it when you want it to recognize something.

3. Give us examples of some of the different ways one would benefit from using the OrCam device.

We met a woman in New York who is blind, and she loves to cook. It’s one of the things she can still do despite her vision loss. She had a problem where she would accidentally open up the same can of soup two or three times in a row. She could hire someone to help her all the time, but then what would her contribution be to the family? She told us this and began to cry.

Throughout the course of our research, we met a guy who said all he does since he lost his sight is to listen to the radio. Once a week, a friend comes over to help him shop. He looks forward to grocery shopping all week, and for this task he needs someone to help him. We thought: if we can help him be more independent, go to the supermarket and know that the product he bought is the one his wife asked for - that’s a major step up in his lifestyle.

The OrCam device allows users to read their morning newspapers and books without asking for assistance or searching for audio books, it reads text on computer or smartphone screens at home or on the go, and so much more…

4. What are some activities that the device can be used for?  In what activities is it not helpful?

The OrCam device is geared to give you information. In that way, it’s quite different than a white cane, which can give you mobility. The cane extends your hand so you can feel the floor while standing up. Our device is more informative – it reaches further. You can read the menu at the restaurant on your own. You can read a book while holding it – that’s not a trivial matter. To those who have lost their vision, the ability to hold a book and independently read in the natural position which they are used to is extremely rewarding.

The ability to trigger the OrCam device, or designate a region of interest, by using a gesture as simple as pointing your finger makes it especially easy to use - no need to learn new software and tools. The battery is designed to last all day and charge overnight, as you would do with a mobile phone.

You will be able to go out and meet a friend - first find the meeting spot, then recognize the friend. It will be able to tell you what street corner you are standing at by looking in the general direction of the street sign and reading the information about where you are. The OrCam device is intended to enhance the day-to-day lives of people with visual impairments. It should not be solely relied upon in circumstances which are potentially life-threatening, such as crossing a street, or that which could lead to financial loss.

5. How is this product different from other “smart glasses”?

For starters, the OrCam device has no screen, obviously. The OrCam device’s main added value is a core improvement in daily life – our goal is not to augment reality, it is to translate visual information into audio. It’s designed to work all day and be dependable; you can use it all day without recharging, and then recharge it overnight like you would your smartphone. Our device is also designed to be discreet, which is a major issue for the blind and low vision community.

We understand that eyeglasses are important and unique; they are a part of your look and not just a visual aid. Hence, the OrCam device was designed to be mounted on your existing eyeglass frames (almost all sorts are appropriate, including sunglasses). The device itself has two parts: a base unit which is about the size and weight of most smartphones, and a head unit which weighs less than an ounce (about 20 grams), connected by a thin cable.

The OrCam device is self-contained, which enables it to be very fast, as it does not need to search the internet for information. You don’t need a computer or internet connection to use it, so we also have no issue of privacy, nobody looking over your shoulder, monitoring your usage or targeting you with ads. 

6. Patients with what eye conditions would find the OrCam technology useful?

Since we don’t physically restore vision, everyone can use it. We have found that people with a knack for technology can start using it within a few minutes. Less techie people may need a little longer, but within two hours anyone can use it, and we have gained significant experience with this. In our experience it takes about two weeks for OrCam to become a natural part of daily life.

For all of our initial users, we provide a complimentary in-home training session with a certified OrCam Trainer. This one to two hour session has proven very successful in helping our new users get to know the OrCam device and in removing the initial trepidation from our less techie users.

7. How easy or hard is it to adapt OrCam for use in additional languages or regions?

We’re constantly working to expand and improve. OrCam is currently available in the United States, Australia, Germany, Switzerland and Austria. 

It is now programmed to work in English and German. We expect to have support for additional languages and regions in the future. The OrCam device comes pre-loaded with region-specific data, such as the way money or common consumer products look. This requires a significant effort on the part of the development team, to collect and input this information. We’re also subject to the standard regulatory processes in each country. OrCam believes in great customer service, which is why we are also working to develop the infrastructure to support users around the world.

8. What are your plans for the future?

There are about 300 million people waiting for help, and we want to get to each and every one of them. We’re constantly working to improve the OrCam device. We’re working with various agencies to remove all barriers between people who need our help and getting that help to them. We have plans to improve the device and cover more of the needs of the users out there so that we can help them be more independent and productive.

The Updated OrCam MyEye 2.0

You can find more information about OrCam online at www.orcam.com.

Go from Low Vision Orcam Device to Low Vision Technology

Return to WebRN Macular Degeneration Home