E-mail updates
Follow on Twitter
Subscribe to RSS
Ted S. Warren / AP
To produce the words for text or speech, British physicist Stephen Hawking currently uses an infrared sensor mounted on his eyeglasses, visible here during an appearance this month in Seattle. The sensor picks up twitches from his cheek, which are translated into the desired letters or words. Hawking and neuroscientist Philip Low are experimenting with a system that can translate brain waves directly into text and speech.
After months of tweaking, researchers are finally ready to show off a high-tech headband that can translate Stephen Hawking's brain waves into speech — providing what could eventually become an easier avenue for the paralyzed British physicist and many others to share their deep thoughts.
The system, developed by San Diego-based NeuroVigil and known as iBrain, uses a head-mounted receiver the size of a matchbox to pick up different types of brain waves. iBrain employs a computer algorithm called SPEARS to analyze the brain emanations and encode them for a text-based speech reader. Philip Low, NeuroVigil's founder, chairman and CEO, is to present the latest results from his work with Hawking on July 7 at a Cambridge conference on consciousness.
"I haven't discussed doing a demonstration with Stephen, but we could do that, of course," Low told me today. During the conference, Low will be showing video clips of Hawking using the iBrain to communicate.
For decades, Hawking has been coping with amyotrophic lateral sclerosis, a degenerative nerve disease that has left the theoretical physicist confined to a wheelchair and unable to move even his fingers. To write or speak, he currently uses an infrared sensor system mounted on his eyeglasses: His cheek twitches are read by the sensor to control a wheelchair-mounted computer system that slowly encodes the patterns of those twitches. It can take a half-hour for Hawking to twitch out a couple of sentences in response to a question.
In an abstract prepared for next month's presentation, Low and Hawking describe how they worked out their technique for the iBrain system. Hawking (who is described as a "high-functioning 70-year-old ALS patient" in the abstract) was told to try moving one of his hands or feet — for example, flexing his foot or scrunching his hand into a ball. The limbs didn't move, of course, but just thinking about trying to move them generated readable brain-wave patterns.
"The subject's brain activity demonstrated distinct broad-spectrum pulses extending to the gamma and ultra-high gamma ranges," the researchers wrote. "Such pulses were present in the absence of actual movement, and absent when the subject was not attempting motion."
The abstract said Hawking's brain also buzzed with alpha brain waves when he closed his eyes, as expected. Alpha waves are associated with wakeful relaxation, and are probably familiar to anyone who's undergone biofeedback training. Gamma waves, in contrast, are associated with increased attention — and in the past have been linked to activities ranging from running to learning.
Lots of possibilities
The fact that Hawking's brain signals could be read reliably is a good sign, not only for one of the world's best-known scientists but for hundreds of thousands of others around the world. Low and Hawking say their work "opens the possibility to link intended movements to a library of words and convert them into speech, thus providing ALS sufferers with communication tools more dependent on the brain than on the body."
Low told me that the brainwave-reading device could be used to control prosthetic devices "to give ALS sufferers mobility" — sort of like a real-life version of the Stephen Hawking robotic exoskeleton proposed in an Onion parody 15 years ago.
The iBrain device could have other applications, such as diagnosing sleep apnea, studying autism and monitoring other brain conditions. It's already been used in a clinical trial to monitor the effects of experimental drugs on brain activity. The U.S. military is also looking into how the device can help treat traumatic brain injury and post-traumatic stress disorder, which are big issues for combat veterans. Direct brain-to-speech communication, however, represents the highest-profile application, particularly when Stephen Hawking is involved.
"We'd like to find a way to bypass his body, pretty much hack his brain," The Telegraph quoted Low as saying.
The key question for Hawking is whether iBrain represents an improvement over the system he currently has. Back in April, the professor told The New York Times that the project hadn't quite reached that point. "At the moment I think my cheek switch is faster ... but should the position change I will try Philip Low's system," he wrote in an email sent by an assistant.
In that quote, Low said Hawking was talking about brain-computer interfaces in general, rather than specifically about iBrain. "What we are seeing is in fact an immediate response, so the question is going to be to productize this, so that he can communicate reliably should he lose control of his cheek muscles," he said.
TEDMED via YouTube
Neuroscientist Philip Low (at right) demonstrates how the iBrain device can send brain-wave readings to a cellphone with an subject who's wearing the headband (at left) during a TEDMED 2009 presentation. Click on the image to watch the YouTube clip.
Personal quest
Low said the iBrain project was already moving on to Version 2.0, and the iBrain 3 device is due to be built next year. "That will be about the size of a U.S. quarter," he told me. "People will be able to check their brain activity much like you or I can check our blood pressure."
The 32-year-old, Vienna-born researcher's company has come a long way since its founding, which Low says he initially financed by putting $240,000 on his credit card. Someday, he hopes brain-monitoring systems will be used to pick up the signs of neurological problems early enough to do something about them. For Low, this is not just business. It's personal.
"I would have loved to see this 20 years ago, when my father suffered from a side effect of a commonly used sleep drug," he told me. "He threatened someone with a weapon ... a gun, actually. And it destroyed our family."
His father was eventually pardoned, but it took a long time to put everything back together. That experience led Low to look into the neurological basis of sleep, including experiments with bird brains. That was what led him to come up with the SPEARS algorithm in the first place.
"It's very ironic that an algorithm I initially developed to analyze the brain patterns of birds has found its way to dealing with Stephen Hawking's brain patterns, the U.S. military and autistic children," he told me.
More about Stephen Hawking:
- Stephen Hawking keeps his eyes on the prize
- What's on Dr. Hawking's desk? A guided tour
- Stephen Hawking's biggest mystery? Women
- Hawking says God's not needed. So?
Alan Boyle is msnbc.com's science editor. Connect with the Cosmic Log community by "liking" the log's Facebook page, following @b0yle on Twitter and adding the Cosmic Log page to your Google+ presence. You can also check out "The Case for Pluto," my book about the controversial dwarf planet and the search for new worlds.
Leave a Comment:
Be careful what you think Mr. Hawking
Yeah, really,I hope he only has clean thoughts,I wouldn't be able to relax just thinking about it.Kind of funny.
It won't read out the words he is thinking. Instead, they will program different thoughts about motion to mean different words (like programming the thought of pointing one's index finger towards oneself to be read as "me" or the thought of tapping the index finger as the word "the".) If it did verbalize all thoughts, the problem would quickly become creating a way to filter those thoughts.
Or moving an arm = dot, moving a leg = dash and communicate in Morse code. Or something similar with 0s and 1s and use ascii. Or use the brainwaves to replace his previous menu-driven typing system.
Whatever it is, it will have to be simple.
Football, porno, cheese burger...how hard is that...I'll be impressed when they can get the tech to work on Women...Then it will be scary
Only one word for Mr. Stephen Hawking, THANK YOU, Sir.
Mister Peter Jacobs,
I am unsure if you meant to do this or not, but thank you is two words.
-Septirath.
I'm so sorry. I accidentally posted that comment twice. Pardon me.
Yup! Poetic license intended to demonstrate ONE feeling of appreciation and awe for a person which inspires me in so many words. Thanks!
Super Cool...
I am hoping that within the next couple years, science makes a fully functional cyborg that they can just throw Mr. Hawking's brain into. We need to keep it around.
Yeah, there's not enough original Science Fiction. Maybe he can do a better job of it.
Fascinating stuff. I hope Hawking can last long enough to benefit from a system that could automatically translate his thoughts into language, both spoken and written. I cannot imagine how frustrating it must be for him in his current state. He seems to manage, he's certainly doing better than I would be doing if I were in his shoes. But it still must be very difficult for him. Hang on Stephen, help is on the way. I just hope it gets here in time.
Looks more like alot of exploitative advertising in the articles links coupled with a toying of anxieties, until we actually see the device work.
This picture is a 1st, I've never seen him drool before.
great work
S. Hawking greatest mind on the planet. MSNBC not so smart, at least not smart enough to know the difference between the left and right side of a man's face. It's really quite simple, consult an 8 year old next time.
(Sigh) ... Are you really appealing to the "my left" / "his right" issue? In this picture, it's on the left side. I appeal to the commenters: Am I off-base on this? But just so that I don't confuse the 8-year-olds, I'll rewrite the caption to avoid stating the obvious.
Well, to be honest, the caption does say "left side of his face" and not "left side of the picture". If someone were to tell me I had something on the left side of my face, I would go to MY left, not their left. However, I didn't pick up on it when I read it and it's pretty obvious what is meant. Someone is just being nit-picky.
It's semantics, to be sure.
Someday people on these blogs might actually be concerned with something that actually matters.
Gotta be tough, being under the microscope when the eye of perfection is peering at you. I didn't catch it, either, I just looked at the sensor...then the drool, which made me feel bad for Dr. Hawking. In any case, I guess I have to say it. My name is Ken, and I am NOT smarter than an 8 year old.
Yes, I'm not happy about the drool, but that was the only recent picture at my disposal that clearly showed the sensor (on the left side of the picture). ;-)
Alan
I do my best to try an ignore all comments that are negative, Alan you are with friends an i agree with you.
For the article, Technology is forever on the move, this is amazing to say the least.
Stephen you have my best wishes, to you my friend.
Have a good day, Tom And Lyn
Another comment if I may.
All of his life, Dr Stephen Hawking, has been a tremendous, and overwhelming, roll-model, not just in physics, but also for those who have a disability.
Stephen keeps his head high, and never lets his disability keep him from his dreams or his work.
I take my hat off to him, and his great achievements and success.
Have a good day, Tom And Lyn.
When in doubt, I just assume the picture was reversed after the article was written. Certainly no big deal! The new technology is great news.
This is only the first step. The sky is not even the limit on this. I remember the first moon landing, and never considered a space station in my lifetime, but there it is. A device like this could some day help law enforcement in trying to get reliable lie detectors. It could help psychologists in treating mental illness by helping people gain insight into their own attitudes and thoughts by collecting thousands of mini reactions and giving real data readouts. Maybe it could help with our reaction time in traffic, or train better athletes. I really think the possibilities are only limited to the imagination.
Why can't the (loosely referred to as) authors on msn who really mess up stories read the comments and fix their articles? And notice how none of us corrected Rob's grammar. Because we know what he meant.
I've substituted a different picture that doesn't show the sensor quite as well but also is a bit more complimentary to Dr. Hawking.
I was never crazy about my drivers license pictures ....
But I think they do that on purpose at the D.M.V.'s .... "LOL"
Alan, dearie, I hope you know you're not in that sloppy category to which I referred. Keep up the good work.
Alan,
As a former journalist and Peabody Award recipient I feel your pain brother. Nit-pickers will drive you crazy. Funny, I didn't notice the left/right issue or the drool. I was more absorbed in the possibility that Hawking may be able to communicate more quickly and easily with the rest of us and what that would mean to us all. Oh well, it takes all kinds.
Keep up the good work.
Skip
Skip :-) I 1000% agree with you my friend, nice post.
Have a good day, Tom And Lyn.
Brain transparency. Lol. No thoughts would be guarded.
Potentially dangerous technology to let loose into the world as we know it..
Dangerous? How? From the descriptions given, I fail to see any danger. Is it dangerous to use it to analyze PTSD or Autism? Perhaps it's dangerous to be analyzing sleep apnea. Give someone the ability to speak once again? <sarcasm>Yes, that certainly is extremely dangerous.</sarcasm>
So many people find anything they do not understand to be dangerous and frightening.
Many likely read this article and stepped away afraid that some evil, godless scientists are making machines to read peoples minds. The whole article as basic and totally nontechnical as it is, went flying right over the top of their heads and now the sky is falling! It is pathetic.
I believe the danger comes in the form of unintended consequences. If you can read a persons thoughts to enable speech for those who cannot speak it is an amazing technology giving a huge leg up to someone facing an adversity. But why would it not be possible to use the same technology to read the thoughts of a person and use that information against them, say in an interrogation situation. We have laws that protect us against self incrimination. However in some way shape or form the introduction of lie detectors were allowed in police interrogations. Why not this technology. How reliable is a thought when the person is under stress.
It is not fear or paranoia to see the downside of a new technology applied in ways the inventors did not think. These scientists are searching for a remedy to a particular issue. The study of nuclear physics was not intended to build a nuclear bomb, yet the science was hijacked and now we live with the consequences.
Science and technology are amazing things. We live in extraordinary times. I believe the benefits far outweigh the down sides. However assuming the science is flying over someones head and inferring chicken little is running scared because of their limited ability to comprehend is really just looking at one side of the story and not giving equal time to consider the risks. They do exist and the possibility of misuse is very real as our history has demonstrated over and over.
Frankly, I find it thrilling. Two years ago, I sat next to my mother's bedside as she died from ALS. Although her mind was intact, her body had utterly failed her. She was no longer even able to utilize her voice synthesizer through eye movement at the end. The most horrible aspect was that there was absolutely no ability to communicate. We could only attempt to guess what she needed when she could no longer blink once for yes, twice for no. "I'm giving you more Morphine drops for the pain, Mom, we hope that is what you need." "I'll rub ice chips on your lips, Mom, is that what you need?"
If this scientific advancement can spare another family from that... I applaud it fiercely and with great vigor. Until you have endured the suffering of a loved one with helpless grief, unable to provide even minimal comfort due to an inability to communicate, perhaps you cannot grasp what this would mean to a host of patients with varying disabilities and disease, as well as the boon it presents to their caretakers. Someday, this advance may be available to many of those who need it most and present some increase in the quality of a waning life for those trapped in an unresponsive body.
Phil, you have just proved my point!
THIS TECHNOLOGY DOES NOT READ THOUGHTS!
It measures and reads brainwaves, basically turning your mind into a virtual keyboard. It absolutely does not translate human thoughts into words. Such technology would have obvious risks, but this is not it, this is nothing but positive!
The article states they are attempting to map the brains response to movements and a catalog of words that could be used to facilitate communication. Today he twitches his cheek in a particular manner and it spits out a letter or word. With this machine he thinks about moving a leg and that brain wave action is recognized and mapped to the word walk or something along those lines.
I get that. Really I do. I'm not here to jump to wild conclusions and tremble under a rock as technology marches on. My post was to point out the unintended consequences that often occur with scientific breakthroughs. The poster you were mocking as chicken little running around yelling the sky is falling is also not likely hiding under a rock. Just because someone wants to say hey, wait a minute, are we really thinking this through does not make them a techno phobe.
But just to be clear, the machine is measuring brain wave activity for known patterns to specific thoughts. These thoughts are being mapped to a catalog of words to facilitate speech. But at the root it is a measurement of activity of known patterns.
Why is it a leap to think that there could be known patterns of brain wave activity for a lie? Why could we not map brain wave activity for the entire English lexicon? Right now we are mapping activity for muscle movements but could we not move beyond that to the areas I just spoke about? There in lies the moral and ethical divide. Could this be the technology that supplants a lie detector? If we can begin mapping the extent of human experience then why would we not be able to pier into the mind of another in ways that could be self incriminating? It's a valid concern based on the technology this is driving. Maybe not the particular use outlined here but it is not a huge leap to make.
There is nothing dangerous in this technology. It appears this writer did not comprehend the article - later comments are just as outlandishly fearing what has not yet been created.
Looking into possible science fiction scenarios takes away from the reality of being able to communicate with a love one. (And overlooking 'pier')
I would have loved having one for my ALS-ridden mother after she lost her ability to speak.
AnotherTexan: Your comment: "There is nothing dangerous in this technology." Read the very last paragraph in the article and you will find the danger, my friend, in the words 'the military'! Anytime 'the military' gets hold of something you can BET that they are NOT in it for the good. They are trying to figure out how to use it as a weapon or to 'control' a subject. Sorry folks but there IS danger here. However, that being said, I am sure they already have it and we have no control. I am happy for Hawking and all the people that can use it now and in the future to help them. Technology can be used for good or evil whichever is in the eye of the beholder.
Diana-2844801, "the military" has already demonstrated the ability to fly planes using pilot's brain waves. This device is a step down from the tech they already posses.
Hmmmm.... maybe, just maybe, they are planning to see if soldiers that have been seriously wounded in battle might benefit?
If I remember correctly, there was a device featured in Time magazine last year that could accurately read brain waves while attached to other parts of the body. Was it the iBrain listed here? Is this the first specific application of the iBrain technology, or do they gave other working devices?
have had friends suffering from Als, They used an eye chart, much quicker than half hour. Problems was for the observer to learn to recognize where their eyes were looking. a corneal implant, small of course, would really facilitate the interchange. Science has got to be caught up by now enough to blend the two, Im not educated>> but a small visor could be their keyboard.. They finally got technology to make this now, Please??
Why don't we just get Yuri Geller to follow him around, read his thoughts, and be his voice box?
With the most brilliant brain in the universe, why doesn't someone just freeze-dry it? Seriously.
Because the spark of life didn't come from a laboratory.
Mr. Hawking may be a genius when it comes to physics and space theory, but can we just keep it to that, please? Also, why can't someone fix his teeth?
British National Health Care
Think about *that* this coming National Elections
“I wouldn’t be here today if it were not for the N.H.S. I have received a large amount of high-quality treatment without which I would not have survived.” Stephen Hawking on the British National Health Care system.
To what purpose? He can't chew or, as the article clearly states, even move his finger.
I guess a lifetime in a wheelchair isn't enough for you. You'd have him endure painful oral surgery and/or orthodontics, just to satisfy your twisted sense of aesthetics...
Chris-587843
“I wouldn’t be here today if it were not for the N.H.S.
Nice Health Care... If you can get it:
Death Panels:
www_telegraph_co_uk/health/healthnews/6127514/Sentenced-to-death-on-the-NHS.html
Cancerous Kidneys
www_guardian_co_uk/society/2012/mar/01/nhs-apologises-patients-kidneys-cancer
Bankrupt
www_cobdencentre_org/2010/08/the-uk-is-broke/
One recent Institute of Economic Affairs study by Nick Silver put this figure at 333% of GDP. Another, by Christian Hagist and his colleagues at Freiburg University, put the figure at 530% of GDP. Two different methodologies by reputable researchers, both painting a very bleak picture.
Don't forget to participate in the upcoming 2012 elections.
So if I have a device the size of a quarter that can read my brain waves and convert it to text then to speech I should be able to hook this up via bluetooth to my cellular phone call my buddy and have my thoughts sent to him over the airwaves. He gets the call and it goes right to his device which will then re-encode it back to brain waves he can interprut and vice versa. We have true, technology enabled telepathy.
Think about this now. We already communicate via voice less because we all text and tweet. Once we start communicating telepathicly the only thing we'll need our mouth for is food. So we could begin to evolve just like all those aliens with the super big eyes and tiny little mouths. I think I can see where this is going. Time travel, aliens are really just the future versions of ourselves traveling back in time to better understand the past. Basically just anthropology students from the future.
wow...
Hey, why stop there? Eventually we'll evolve into one contiguous thought with no bodies, no machines, no language.. then we'll be cooking with gas!
It gets even better.
Imagine instant recall of facts from vast stored databases and libraries.... directly into the brain.
Everybody the equivalent of a PhD.
"No more teachers, no more books, no more teachers dirty looks!"
And getting rid of all those useless unionized "teachers" that can't even impart basic arithmetic, despite their certificates and masters degrees!
Unfortunately, Anon, no amount of facts will turn the majority of us into geniuses. Look at how many deny facts at face value.
Personally I would love to see this developed more and priced were we could buy it. I have a son with cerebral palsy that is non-verbal and it takes him forever to type something out. This would truly be beneficial for him.
Remind the good Doctor once he's up & about that "No" still means "No", no special pass for being the 'Greatest Mind in the Universe'
Jump ahead a few years with this equipment and you could see it in the court system. " So Mister X did you do the crime?" Well no your honor. (head band says yes) Guilty! lol Just like Wonder Woman's golden rope.
Of the thousands of pictures of this man, they publish the one of him drooling. You guys are getting as bad as Yahoo News.
Here's hoping for a successful hacking into Hawking! ;-)
Gives hope not only to him (and the countless treasures stored in that magnificent mind of his), but also to other LS patients whom this could help. It's interesting how he refuses an upgrade to his older style voice synthesis device. In an interview I've read, he said he's become used to this voice and it's now 'his voice.'
Alan, do you have any info on this device? Have you moved on to the next article? Heloooo....
Hi, Ben... Yes, I do move on to other things and have to come back every once in a while and check up on y'all. ;-)
You can find out more about iBrain from the NeuroVigil website:
http://www.neurovigil.com/press/
As I wrote, Low and his colleagues are about 75 percent of the way into iBrain 2, and iBrain 3 will be built next year. Low is getting clearance from the FDA for the use of the brain-detecting algorithms in research on a project-by-project basis. Eventually he will get FDA approval for marketing the device itself to consumers, but they're not there yet. Of course he's being careful about patent protection, for the device as well as for the algorithms.
Does that help?
Thanks for the link. It seems that this is not the same device I remember. I will keep looking to see if I can find a link to that one.