Seeing Is Communicating

Subheadline

Eye-tracking technology makes 'talking' possible for people who can’t

Communicating when traumatic brain injury, stroke, or disease has made speech impossible can be daunting. But specialized eye-tracking technology uses eye movement to enable people living with disabilities to connect one-on-one, over the phone, or via the internet.

The technology isn’t new, but it has become much more widely accessible, thanks in part to a collaboration between NASA’s Jet Propulsion Laboratory in Southern California and a Fairfax, Virginia-based company called Eyegaze Inc. In the late 1990s, the two entities worked together to miniaturize and improve the company’s Eyegaze Edge system, eliminating barriers to ownership of this communications technology.

Eye-tracking systems for computers pinpoint a person’s gaze — where the eye is looking on a screen — by reflecting infrared light off the cornea and capturing it with a camera, using image-processing software to determine the eye’s orientation. Looking at a specific button on the screen initiates an action. That could be a letter on a keyboard or a command to open a web browser. When the company built the first model in 1988, its computers were bulky, requiring three shipping boxes for equipment and a company staff member to set up the system. That cost limited access, and the learning process could be intimidating.

In 1998, Congress provided funding for NASA to make the hardware smaller, more portable, and more affordable without compromising efficiency. The public-private partnership accomplished that and much more. It reduced the weight of the original system by six times and its volume by nearly the same factor (Spinoff 2002). Other advancements served as a springboard for two more decades of development.

“Working with NASA, we were able to make the device less bulky,” said Preethi Vaidyanathan, an engineer with Eyegaze. “Since then, we integrated the external components into a small camera.” It mounts above or below a standard computer screen and requires less than 15 seconds to calibrate to an individual’s gaze.

Visual Surfing

As personal electronic devices and internet access became commonplace, Eyegaze customers wanted to do more than type. The company’s work with NASA and other government partners put it in a position to meet that demand.

The company’s Eyeworld software integrates with computers, mobile phones, and tablets, allowing the Eyegaze camera to function as an external mouse and keyboard. By enabling almost any computer function, it lets users chat online, post to social media, send email, text, or make phone calls. It’s also possible to change room lighting, adjust thermostat settings, and operate other environmental controls using Amazon Echo and Google Home via pages of specialized buttons with one-glance button controls.

During the COVID-19 pandemic, Eyegaze integrated communications programs such as Zoom and WhatsApp to allow people to connect with their family and friends. To help combat a sense of isolation, the company added meditation apps and Window Swap, which streams the views from different windows around the world. Virtual museums take the viewer on guided tours. And a music streaming service syncs to favorite music apps and offers the International Radio Garden app to sample music from other cultures. Access to online gaming communities provides another way to connect.

“We are constantly thinking about what the customers want. One thing is some freedom of movement, so controlling a wheelchair gives them that to some extent,” said Vaidyanathan. Eyegaze Edge integrates with the Ability Drive application used with motorized mobility devices. Looking at specific buttons enables hands-free control of a wheelchair’s motion.

The Right to Speak

Numerous medical conditions can lead to the kind of physical paralysis that makes it impossible to speak. Well-publicized brain and spinal cord injuries in athletes have led to wider awareness. But there are a host of other causes such as cerebral palsy, ALS, multiple sclerosis, muscular dystrophy, and spinal muscular atrophy. Strokes, infections, and other traumas can result in locked-in syndrome, which paralyzes all voluntary muscles except those that control eye movement. For all of these patients, eye-tracking software is a lifeline, making use of what may be their only remaining voluntary movement. But even that can be impeded.

Eyegaze Edge measures several eye features, making it possible for the tracker to work even when less of the pupil is visible. The company employs clinical specialists who can troubleshoot issues that could  prevent customers from using the system — eye conditions like cataracts, dry eyes, and eyelids that droop, partly covering the pupil.

The company is now serving individuals in 44 countries including India and Pakistan and nations in Africa. Adults, children as young as 18 months old, military veterans, and others are able to communicate using only their eyes. If only vertical eye movement is possible, Eyegaze Edge offers a special vertical keyboard screen. Pages of commonly used phrases communicate a statement with a single glance. Common medical needs are also connected to large, easy-to-use buttons that allow patients to communicate directly with healthcare providers and caregivers.

Clinicians use Eyegaze Edge and teach their patients how to communicate using the technology. Those same medical professionals are also able to advocate for their clients with insurance companies, documenting the need for them to own such an assistive device. In some cases, Vaidyanathan said, this technology even allows people to share their final thoughts with family and friends.

“NASA helped us get our technology to the size of a laptop — small and sleek. But we continue to investigate and design our solutions to meet ongoing needs. Communication is a key right, so we have to accommodate these different challenges,” she said.