How researchers hacked into Stephen Hawking's brain

Ted S. Warren / AP

To produce the words for text or speech, British physicist Stephen Hawking currently uses an infrared sensor mounted on his eyeglasses, visible here during an appearance this month in Seattle. The sensor picks up twitches from his cheek, which are translated into the desired letters or words. Hawking and neuroscientist Philip Low are experimenting with a system that can translate brain waves directly into text and speech.


After months of tweaking, researchers are finally ready to show off a high-tech headband that can translate Stephen Hawking's brain waves into speech — providing what could eventually become an easier avenue for the paralyzed British physicist and many others to share their deep thoughts.

The system, developed by San Diego-based NeuroVigil and known as iBrain, uses a head-mounted receiver the size of a matchbox to pick up different types of brain waves. iBrain employs a computer algorithm called SPEARS to analyze the brain emanations and encode them for a text-based speech reader. Philip Low, NeuroVigil's founder, chairman and CEO, is to present the latest results from his work with Hawking on July 7 at a Cambridge conference on consciousness.


"I haven't discussed doing a demonstration with Stephen, but we could do that, of course," Low told me today. During the conference, Low will be showing video clips of Hawking using the iBrain to communicate.

For decades, Hawking has been coping with amyotrophic lateral sclerosis, a degenerative nerve disease that has left the theoretical physicist confined to a wheelchair and unable to move even his fingers. To write or speak, he currently uses an infrared sensor system mounted on his eyeglasses: His cheek twitches are read by the sensor to control a wheelchair-mounted computer system that slowly encodes the patterns of those twitches. It can take a half-hour for Hawking to twitch out a couple of sentences in response to a question.

In an abstract prepared for next month's presentation, Low and Hawking describe how they worked out their technique for the iBrain system. Hawking (who is described as a "high-functioning 70-year-old ALS patient" in the abstract) was told to try moving one of his hands or feet — for example, flexing his foot or scrunching his hand into a ball. The limbs didn't move, of course, but just thinking about trying to move them generated readable brain-wave patterns.

"The subject's brain activity demonstrated distinct broad-spectrum pulses extending to the gamma and ultra-high gamma ranges," the researchers wrote. "Such pulses were present in the absence of actual movement, and absent when the subject was not attempting motion."

The abstract said Hawking's brain also buzzed with alpha brain waves when he closed his eyes, as expected. Alpha waves are associated with wakeful relaxation, and are probably familiar to anyone who's undergone biofeedback training. Gamma waves, in contrast, are associated with increased attention — and in the past have been linked to activities ranging from running to learning.

Lots of possibilities
The fact that Hawking's brain signals could be read reliably is a good sign, not only for one of the world's best-known scientists but for hundreds of thousands of others around the world. Low and Hawking say their work "opens the possibility to link intended movements to a library of words and convert them into speech, thus providing ALS sufferers with communication tools more dependent on the brain than on the body."

Low told me that the brainwave-reading device could be used to control prosthetic devices "to give ALS sufferers mobility" — sort of like a real-life version of the Stephen Hawking robotic exoskeleton proposed in an Onion parody 15 years ago.

The iBrain device could have other applications, such as diagnosing sleep apnea, studying autism and monitoring other brain conditions. It's already been used in a clinical trial to monitor the effects of experimental drugs on brain activity. The U.S. military is also looking into how the device can help treat traumatic brain injury and post-traumatic stress disorder, which are big issues for combat veterans. Direct brain-to-speech communication, however, represents the highest-profile application, particularly when Stephen Hawking is involved.

"We'd like to find a way to bypass his body, pretty much hack his brain," The Telegraph quoted Low as saying.

The key question for Hawking is whether iBrain represents an improvement over the system he currently has. Back in April, the professor told The New York Times that the project hadn't quite reached that point. "At the moment I think my cheek switch is faster ... but should the position change I will try Philip Low's system," he wrote in an email sent by an assistant.

In that quote, Low said Hawking was talking about brain-computer interfaces in general, rather than specifically about iBrain. "What we are seeing is in fact an immediate response, so the question is going to be to productize this, so that he can communicate reliably should he lose control of his cheek muscles," he said.

TEDMED via YouTube

Neuroscientist Philip Low (at right) demonstrates how the iBrain device can send brain-wave readings to a cellphone with an subject who's wearing the headband (at left) during a TEDMED 2009 presentation. Click on the image to watch the YouTube clip.

Personal quest
Low said the iBrain project was already moving on to Version 2.0, and the iBrain 3 device is due to be built next year. "That will be about the size of a U.S. quarter," he told me. "People will be able to check their brain activity much like you or I can check our blood pressure."

The 32-year-old, Vienna-born researcher's company has come a long way since its founding, which Low says he initially financed by putting $240,000 on his credit card. Someday, he hopes brain-monitoring systems will be used to pick up the signs of neurological problems early enough to do something about them. For Low, this is not just business. It's personal.

"I would have loved to see this 20 years ago, when my father suffered from a side effect of a commonly used sleep drug," he told me. "He threatened someone with a weapon ... a gun, actually. And it destroyed our family."

His father was eventually pardoned, but it took a long time to put everything back together. That experience led Low to look into the neurological basis of sleep, including experiments with bird brains. That was what led him to come up with the SPEARS algorithm in the first place.

"It's very ironic that an algorithm I initially developed to analyze the brain patterns of birds has found its way to dealing with Stephen Hawking's brain patterns, the U.S. military and autistic children," he told me. 

More about Stephen Hawking:


Alan Boyle is msnbc.com's science editor. Connect with the Cosmic Log community by "liking" the log's Facebook page, following @b0yle on Twitter and adding the Cosmic Log page to your Google+ presence. You can also check out "The Case for Pluto," my book about the controversial dwarf planet and the search for new worlds.

Discuss this post

Jump to discussion page: 1 2 3

I smell a lawsuit from those greedy tards at Apple coming... Since they sue over people using an apple (even if it looks completely different) in their logo, I wouldn't be surprised at all, and in fact expect them to sue over "iBrain."

  • 1 vote
Reply#21 - Mon Jun 25, 2012 7:42 PM EDT

I never really understood apple's ability to get a patent on the two-touch technology, since things like the movie "Minority Report" proved that the idea was public domain long before Apple could have filed for a patent.....

    #21.1 - Mon Jun 25, 2012 8:50 PM EDT
    Reply

    What a great device.

    The first people it should be used on are politicians to determine if they are fibbing about their political campaigns and what they will do if elected.

    The next group of people should then be the criminals like Zimmerman who would paint a picture of the events surrounding the night he killed Trayvon Martin.

    THERE IS NO ESCAPING THE TRUTH NOW! The devil has been caught.

    • 2 votes
    Reply#22 - Mon Jun 25, 2012 8:13 PM EDT

    You're a moron.

    • 1 vote
    #22.1 - Mon Jun 25, 2012 8:26 PM EDT
    Reply

    One has to feel sympathy for his condition but he has been vastly over rated in science.

    • 1 vote
    Reply#23 - Mon Jun 25, 2012 8:25 PM EDT

    Absolutely Correct. I do research in the fields of physics and astrophysics. Hawking's condition is indeed tragic and not what I would wish on anyone. But without this condition and having to speak through a computer, I and many of my associates do not believe that people would make much of a fuss about Hawking and his wild theories. But the popular press puts Hawking on a pedestal that he has not truly earned. Most all of his theories and contributions to science have not been proven except mathematically. They all may in fact fall flat on their face once all is known.

    Most people do not want to say anything bad about Hawking though due to his condition. But he is highly over rated for certain. Brilliant at math, but not so much in other areas.

      #23.1 - Mon Jun 25, 2012 10:13 PM EDT
      Reply

      There is, of course, another, more sinister application: Mind reading, use on prisoners, terrorists, etc. Perhaps Mel Gibson was on to something after all when he used a similar process to confirm "What Women Want"......

      • 1 vote
      Reply#24 - Mon Jun 25, 2012 8:29 PM EDT

      For me, this is far more "cool" than "scary" but consider: If we have the technological capacity to have a device which can turn our thoughts into speech, how long until someone figures out how to do this on an involuntary basis.... ?

      • 1 vote
      Reply#25 - Mon Jun 25, 2012 8:40 PM EDT

      Exactly - Scientists are like little kids with a new toy: They immediately think if they CAN do something, then they SHOULD do it (logically unsound reasoning) and as often as possible (the prepubescent thing) whether others like it or not or are even aware of it (Unethical at best). Great Invention for a genius like Hawking to continue to contribute to humanity, frightening that humanity will somehow pervert it....

        #25.1 - Mon Jun 25, 2012 8:54 PM EDT
        Reply

        This new technology could allow us to hear Stephen Hawking's scientific description , analyzation , atomic physical make up and cosmic significance of a beautiful woman , when observed by him ....

        • 2 votes
        Reply#26 - Mon Jun 25, 2012 8:51 PM EDT

        First of all, thanks for the article, Alan. I don't know how you put up with all these smartass comments day in and day out.

        Second, Stephen Hawking is incredibly brilliant. Most people who seriously appreciate science probably think so too. Hopefully Mr. Low can teak the iBrain system and not only help an amazing scientist like Hawking but eventually help others with ALS ,children with autism and combat soldiers with neurological injuries. I'm sure the list is endless.

        It will be interesting to see how the iBrain system comes along in the next year.

        Keep us informed. Thanks again.

        Hi, Ben. I saw your comment just as I was about to post mine. :-)

        • 3 votes
        Reply#27 - Mon Jun 25, 2012 8:57 PM EDT

        I saw what you saw Darrah , on page one with Alan ....

        But I knew that the following responses would be fun to read ....

        I too like this new technology and its possible uses to help many ....

        We're surely living in an amazing time Darrah ....

        Always good to see you ....

        • 2 votes
        #27.1 - Mon Jun 25, 2012 9:18 PM EDT

        Mr. Low can teak tweak the iBrain system and not only help an amazing scientist like Hawking but eventually help others with ALS ,children with autism and combat soldiers with neurological injuries. I'm sure the list is endless.

        • 2 votes
        #27.2 - Tue Jun 26, 2012 7:28 PM EDT

        From Wikipedia, the free encyclopedia

        The human brain is the center of the human nervous system. It has the same general structure as the brains of other mammals, but is larger than expected on the basis of body size among other primates. Estimates for the number of neurons (nerve cells) in the human brain range from 80 to 120 billion. Most of the expansion comes from the cerebral cortex, especially the frontal lobes, which are associated with executive functionssuch as self-control, planning, reasoning, and abstract thought. The portion of the cerebral cortex devoted to vision is also greatly enlarged in human beings, and several cortical areas play specific roles in language, a skill that is unique to humans ....

        Up 120 billion nerve cells Darrah ....

        • 1 vote
        #27.3 - Wed Jun 27, 2012 2:04 AM EDT

        Al,

        I think my brain cells are shy of a couple of billion. lol

        The human brain is really brilliant to say the least. The brain is always processing new information too. Just the act of you getting that info and my receiving it has added a few brain cells.

        Thanks!

        I wish people would give Stephen Hawking a break. They can't even admit that the man is extremely brilliant. If they had a brain like his, they wouldn't be making fun of other people. I guess we'll never know though since they (CL nasties) don't have a brain like his.

        • 1 vote
        #27.4 - Wed Jun 27, 2012 11:23 AM EDT

        Al Ben. God, where did that come from!

        That just goes to show you why I lack a few brain cells. lol

        • 1 vote
        #27.5 - Wed Jun 27, 2012 11:30 AM EDT
        Reply

        Hasn't this religious nut-job of the pop-Atheism cult already told us everything he thinks? Actually, Star Trek isn't real and Stephen ain't no Albert and he ain't no Isaac. My barber has deeper thoughts than this guy.

        • 1 vote
        Reply#28 - Mon Jun 25, 2012 9:13 PM EDT

        Your barber should probably put down the scissors and pick up a pen, then....

          #28.1 - Wed Jun 27, 2012 4:57 PM EDT
          Reply

          Where do you get a credit card with a $240,000 limit?

            Reply#29 - Mon Jun 25, 2012 9:18 PM EDT

            I once heard that Larry Ellison had a $110 million credit card bill one month. He brought a boat, I think. [lol]

            I would guess that this guy had a bunch of credit cards and maxed out each to the credit card limit.

            BTW: One my credit cards has a $35k limit. Another has a $22k limit. So right there, I could put together a quick $55k. Many people with good credit have high credit limits on their cards.

              #29.1 - Mon Jun 25, 2012 11:31 PM EDT
              Reply

              When worst comes to worse, if these iBrains become as popular as iwhatevers, we may discover that like many phone conversations, twitters and newsvine comments that there is no one on the other end.

              We may learn that many, many brains are infact empty of tought.

                Reply#30 - Mon Jun 25, 2012 9:24 PM EDT

                What would you suggest be done for his teeth? Nothing painful or stressful I hope.

                  Reply#31 - Mon Jun 25, 2012 9:28 PM EDT

                  LOL you can't un-do being English...

                    #31.1 - Wed Jun 27, 2012 4:58 PM EDT
                    Reply

                    Sounds and looks alot like Mattel's Mindflex game which you can buy on Amazon for 58 bucks. Actually, there are quite a few of these floating around under different names for various platforms.

                    • 1 vote
                    Reply#32 - Mon Jun 25, 2012 9:42 PM EDT

                    Seems simpler for someone to learn the Vulcan Mind Meld.

                      Reply#33 - Mon Jun 25, 2012 10:17 PM EDT

                      so will this work with people in comas or on life support ????????????

                        Reply#34 - Mon Jun 25, 2012 10:29 PM EDT

                        No. It reads brainwaves caused by the user voluntarily attempting to move a certain part of his body. The device can interpret these voluntary movements and assign them to a letter, word, or phrase. People in comas and on life support can not voluntarily attempt to move their bodies. This device does 'not' read the mind.

                        • 1 vote
                        #34.1 - Mon Jun 25, 2012 11:33 PM EDT
                        Reply

                        gee you people think he is always thinking about science..lol

                        • 1 vote
                        Reply#35 - Mon Jun 25, 2012 11:02 PM EDT

                        Did you folks really have to use a photo so unflattering??? Was that the only photo available?

                          Reply#36 - Mon Jun 25, 2012 11:09 PM EDT

                          Sorry, I'll take another look, but that was the only photo I had access to that was close-up enough on Dr. Hawking's face to show the infrared sensor in detail.

                          • 2 votes
                          #36.1 - Mon Jun 25, 2012 11:18 PM EDT

                          OK, I've cropped another photo that may be a little less unflattering ... that is, a little more flattering.

                          • 2 votes
                          #36.2 - Mon Jun 25, 2012 11:38 PM EDT

                          Look, the guy has very little use of any body parts. Exactly what do you expect? Sometimes reality isn't pretty. For somebody in Hawkins state the photos are very good. Try volunteering in a nursing home some time.

                          • 2 votes
                          #36.3 - Tue Jun 26, 2012 12:01 PM EDT
                          Reply

                          If this were any other invention for any other purpose I predict Apple would sue. Or buy the company outright and claim they invented this.

                          Either way, this would be neat if they combined it with that company that created the synthesized Roger Ebert voice for him.

                            Reply#37 - Mon Jun 25, 2012 11:17 PM EDT

                            Wow! If they can get this working reliably, they could extract Hawking's brain from his nearly useless body. Then he could think for all of humanity w/o the distractions of the disease he suffers under.

                            Would the brain age, outside the body? Assuming the correct support system, could a live brain survive external to a body? Have any experiments with animals been done yet?

                            And if his brain could live w/o a body, one day when we have perfected human cloning, we might be able to drop his brain into a human body once again.

                            Even if none of this appears possible, it would make for an interesting SF story!

                              Reply#38 - Mon Jun 25, 2012 11:22 PM EDT

                              I wonder if it would be possible to recognize the brains attempts to move the hands and arms for american sign language? Or perhaps the sign language manual alphabet.

                              Could they possibly recognize the brain to finger commands for typing on a typewriter? It wouldn't even have to be QWERTY. There have been a number of one and two handed keyboards devised in the past.

                              I wonder how adaptable the brain might prove to be? If some system could be found to read sign or typing intent from the brain, might a person over time start to think in that movement language, like a person who is fluent in a foreign language might? They she/he wouldn't have to be constantly translating from words to movement equivalents.

                              It's fascinating to think about.

                              • 1 vote
                              Reply#39 - Mon Jun 25, 2012 11:35 PM EDT

                              I had the sign language thought on the next page... I think you beat me to it though. I commented before I read all the other comments. I must commend you, though, on apparently reading the entire article. I think most people think that the device just reads your speech center and translates it to text.

                                #39.1 - Wed Jun 27, 2012 5:01 PM EDT
                                Reply

                                Amazing technology. But it won't be long before it is used as a weapon. That's what always happens with new technology. A thought scanner, perhaps. "You were arrested because our scanner indicated you were thinking about committing a crime." Not as far fetched as it sounds, and we may already be to that point.

                                • 1 vote
                                Reply#40 - Tue Jun 26, 2012 1:58 AM EDT
                                Jump to discussion page: 1 2 3
                                You're in Easy Mode. If you prefer, you can use XHTML Mode instead.
                                As a new user, you may notice a few temporary content restrictions. Click here for more info.