E-mail updates
Follow on Twitter
Subscribe to RSS
IBM via AFP - Getty Images
IBM's Watson computer is made up of a cluster of 90 servers with a total of 2,880 processor cores.
IBM's Watson supercomputer looks like the clear favorite to win this week's man-vs.-machine match on the "Jeopardy" TV game show in the wake of today's action. Right now, the score totals are $35,734 for Watson, vs. $10,400 and $4,800 for the game's two human champions. But even if by some miracle Watson doesn't take the million-dollar top prize, computer scientists say its performance will be judged a triumph for artificial intelligence.
"Watson is clearly playing at a championship level," inventor/futurist Ray Kurzweil, who predicts that A.I. will match human intelligence by the year 2029, told me today in an e-mail. "Note that it's only going to keep getting better. We cannot say that for unaided human intelligence."
Kurzweil said Watson merits the high praise he bestowed upon the machine after seeing its performance in last month's public practice round. In his essay on KurzweilAI.net, he said computers had "not shown an ability to deal with the subtlety and complexity of language" ... until Watson came onto the scene.
"Watson is a stunning example of the growing ability of computers to successfully invade this supposedly unique attribute of human intelligence," Kurzweil wrote. He said that level of language understanding, combined with a well-programmed aptitude for pattern recognition, would make Watson's descendants "far superior to a human."
Alien intelligence
Boris Katz, a computer scientist at the Massachusetts Institute of Technology who pioneered the development of natural-language question-answering systems, agrees that Watson is a wonder. "IBM did a fantastic job," he told me. But he said Watson's foibles also show that a computer's brand of intelligence is still alien to us.
When Watson is good, it's very, very good. But when it is bad, it's horrid. For example, one of the clues dropped during a practice round was: "This trusted friend is the first non-dairy powdered creamer." The correct answer was "Coffee-mate," but Watson gave a nonsensical non-non-dairy reply: "What is milk?"
Another example: On Monday, "Jeopardy" rival Ken Jennings gave a wrong answer for the decade when Oreo cookies were introduced (the '20s), and Watson followed up with what was basically the same answer. ("What is 1920s?") It was left to the third contestant, Brad Rutter, to come up with the right answer (the 1910s). Expert observers assume that Watson flubbed the answer because it didn't catch the fact that the '20s and the 1920s were just two different ways to refer to the same decade.
"When you look at the blunders, you realize that they did not build a machine that thinks like us," Katz said. "The success of Watson does not bring us closer to the understanding of human intelligence. When we observe it making these mistakes, that should remind all of us that this problem is still with us, and it's waiting to be solved."
Overconfident computer?
Watson draws upon 15 trillion bytes of information in its memory banks, the equivalent of 200 million pages of text, and ranks the potential answers for a given clue using 2,880 parallel processor cores in its 90 computer servers. If the highest-scoring answer exceeds its built-in "confidence threshold," it'll buzz in. If no answer scores high enough to reach the threshold, Watson will keep mum. At least theoretically.
"We're seeing already that there are times when Watson really doesn't have enough information to have a good answer, but has the 'confidence' to give an answer anyway," said Eric Nyberg, a computer scientist at Carnegie Mellon University who helped program the supercomputer.
Despite Watson's occasional missteps, Nyberg is proud of the computer's overall prowess, as well as the speed with which it's answering the "Jeopardy" questions. "I was pleasantly surprised that Watson was able to buzz in against Ken [Jennings], because in all of 'Jeopardy,' he's the guy with the fastest trigger finger," he told me.
Today, during an interview on MSNBC, Jennings acknowledged that Watson has "an edge on that buzzer that human reflexes have a hard time keeping up with." He also acknowledged that the pressure was on, big time, going into the final round. (Jennings actually knows who won, since the three shows were taped last month under tight security.)
"The computer can't get stage fright, it can't get discouraged or frustrated. It's like 'Terminator,' it's just going to keep coming," Jennings said. "And so the human race is going to have to play probably aggressively here — big bets where necessary, play recklessly to win."
On Wednesday, TV viewers will find out how this particular man-vs.-machine match ends. But the computer scientists emphasized that this is just the beginning for Watson and its successors. "The fact that it's this fast, and this accurate, and its abilities allow it to do this well at 'Jeopardy' means that question-answering technology is really ready for prime time," Nyberg said.
Watson was built to serve up quiz-show knowledge, but those question-answering capabilities would probably be most valuable in specialized fields such as medicine and law. Watson's kin could help us puny humans sift through millions of possibilities and come up with the five or six best medical diagnoses, or legal precedents, or chemical configurations, or ... well, you name it.
"We're not thinking about applications where there isn't a human in the loop," Nyberg said. "We're definitely talking about an intelligent information agent that's working with a human."
What do you think? Will Watson win this week's showdown? Will question-answering machines become our most reliable advisers? Or will this turn into a replay of "Terminator 3: Rise of the Machines"? Feel free to weigh in with your comments below.
Correction for 12:20 a.m. ET Feb. 16: Error! Error! I've fixed the humans' totals at the end of the first game, and have corrected The Associated Press' figures in the referenced story as well.
More human-vs.-machine matches:
- Chess computer beats world's best player
- Checkers computer becomes invincible
- Poker-playing robot beats human pros
- New Scientist: Computer beats human at Japanese chess
Join the Cosmic Log community by hitting the "like" button on the blog's Facebook page or following b0yle on Twitter.
Leave a Comment:
when they get it to wear cleats and bat a 1000....THEN they'll have something
and the chgo cubs will be first in line;-)
My College girl friend graduated HS Valedictorian of a class of 300, I merely in the top 10%, yet I would be willing to go head to toe with her in deductive reasoning and advanced economic THEORY! I think Watson has a long way to go before he wins a Nobel Prize or matches the deductive reasoning of a 3 year old child!
Your new competition (between the computer and the two best of your contestants) doesn't measure the knowledge of any of them, but rather it mostly measures the speed between hearing and being given information directly into the computer's 'brain.' the info and hitting the buzzer (big slow distance to the finger for the human) to be able to express their answer. I am certain if you asked either of the human contestant their answer, each would have the right answer most of the time and the sum of the two would actually get all of the answers; but they don't make the buzzer connection. If you watch the two candidates, it can be seen that they are physically trying to quickly activate their buzzer but without much success. Putting them each in their own soundproof area (2 humans and I computer) and recording their response to buzzer time and their answer would show that the computer might be smart, but so would the humans be very smart and the computer just had the fastest time [give the computer some handicap – the humans is the time to realize they know the answer to pushing the buzzer].
Your new competition (between the computer and the two best of your contestants) doesn't measure the knowledge of any of them, but rather it mostly measures the speed between hearing and being given information directly into the computer's 'brain.' the info and hitting the buzzer (big slow distance to the finger for the human) to be able to express their answer. I am certain if you asked either of the human contestant their answer, each would have the right answer most of the time and the sum of the two would actually get all of the answers; but they don't make the buzzer connection. If you watch the two candidates, it can be seen that they are physically trying to quickly activate their buzzer but without much success. Putting them each in their own soundproof area (2 humans and I computer) and recording their response to buzzer time and their answer would show that the computer might be smart, but so would the humans be very smart and the computer just had the fastest time [give the computer some handicap – the humans is the time to realize they know the answer to pushing the buzzer].
What method does the computer use to choose catagory and also how does it choose whether to pick a $100 or a $300 question ?
Yes, that is a good question that I was wondering about also.
On Monday, Watson seemed to be choosing all the $200 clues first, then the $400 clues, etc. Like most Jeopardy contenstants do.
On Tuesday, it seemed to choose a strategy of jumping all over th eboard? Was this stategy chosen to confuse the humans?
Did the handlers make this choice or does the computer have the option of what stategy it wants to run?
Posters who state they are not impressed with Watson are missing the point. It's the team of programmers and what they've achieved that's impressive. However it is erroneous to say that Watson understands natural language. As indicated by most of its second choices being completely irrelevant. Watson can analyze natural language. Basically it still has no idea what a truck is. Still the performance is impressive.
As for matching human intelligence, one poster has it right. Sorry folks, it's been done. Remember Watson is going against the top humans in this field. Even when Kasparov won against deep blue the average desktop computer with a simple chess program could beat 99% of humans, 99% of the time. As for other human endeavors, programming a computer to compose original music would be relatively simple, since music, unlike language, follows mathematical principals. In the long run, the only thing that computers will never match is human stupidity.
Yes, but Jeopardy contestants aren't necessarily very "intelligent". Contestants have been studied over the years and what has been found is that they tend to have a special kind of memory that is quick to retrieve facts. And many of the questions on Jeopardy are redundant and predictable.
In the real world, I think there will be a real good match between Watson's algorithms and asking medical related questions.
The problem I see with this contest is that the reaction time of the hu-mans is too slow!
If Watson is parsing the words as the text clue is fed into its memory, then it has a significant time advantage over Ken & Brad. You may not think that the 2-5 seconds it takes to read the clue is significant, but for a computer that is capable of billions of instructions per second, that is a huge time advantage.
Watch out, here comes the Luddites again. They smashed the first CNC devices that were steam powered in England, where all this started in the first place. The fear that textiles made by proto-robot machines would take the jobs away from the manual labour of the time. And yet just look where we are now; our life times have more than doubled, we have visited the moon we have imagined fabulous things that are now a reality. Yet some think of only themselves, and the fear mongering of the media . We live in wonderful glorous times. I can't wait for the next WOW moment. Still why Linux, and not AIX 7.1.. not that there's anything wrong with that..
I truly enjoyed watching Watson on Jeopardy the past few days quite entertaining. So whats next IBM Vs Cray computers on Jeopardy? Would love to see that show-down!
i think this was the most facinating thing i'v seen in a long time. and to think what an impact this could have on our future and the world of medicine and our everyday lives is amazing. go IBM keep up the great work for the future of my grandchildren.
i am amazed with watson. i started on computers in the 1960's and can't believe how far they have come. i wish IBM well and keep up with new developments our grandchildren have all mastered what they have now available to them they need challenges that we have enjoyed over the past 50 years. GOOD LUCK WITH WATSON ! CAN'T WAIT TO SEE WHAT THE NEXT GREAT CHALLENGE WILL BE!!!!!!!
you couldn't be more wrong first jeopardy is a syndicated program so ABC has nothing to do with its production , go to other city's its on other stations.
and two this is so much more than a commercial, its showing where computers have advanced in the area of AI, and while that may not interest you....... well they invented a little machine called a remote control.
I am quite impressed with the capabilities of Watson, so far. I don't think most people quite understand the level of programming involved to get a computer to interact in a gameshow environment, or any interactive environment for that matter. As far as addressing whether a computer can be "moral" or not, of course it can. Morals are nothing more than a set of rules learned over time, governing what sort of behavior is acceptable in society. Computer programming is nothing but a a set of rules, defined by lines of code. The trick, then is whether the programmers can implement that code in such a fashion that the computer's reaction to any given situation does not fall outside of the bounds of what would be an appropriate reaction. I believe that may be possible in the future, but the computer will have to have the ability to append it's own coding, such as humans do, learning from it's own mistakes, and then having the ability to adjust itself as required.
This is the harbinger of a computer that will go beyond the brainpower of every human brain on earth combined. It will also be very controversial as the future computer will be able to stop aging. Who then will be chosen to become immortal? The possibilities are awesome. It could scan our conscieousness on to a C drive. Yikes do we then become disks? Would if the future Watson will find humans boring and kill us as we now do misquitoes. Maybe we can become half human and half Watson and become cyborgs. Fantastic stuff to think about. I won't be around because I am 67 years old now and the prediction is that this will not happen until 2045. Can you imagine the craziness in Congress on this one? I think because of religion this Singularity has a hard bumpy road in front of them. Other than that if you are young now you may be able to live forever. Outrageous stuff and so interesting that they introduced Watson on a game show. Is that irony or what?
LOL - so many haters on Watson. As a programming enthusiast with no talent, I can say with a bit of authority that what Watson is doing on that show is quite extraordinary.
I think the people who are hating on Watson and IBM must take at face value that terminators are already possible, and since they hate machines like that, Watson was nothing special.
But believe me - Watson is quite special. We all love Google for it's supposed "intelligent" search algorythm, but what if you could have search queries as tricky as a jeopardy question that the system gets mostly right? And what more, what if you can just talk it into the search engine?
IBM Search anyone?
Morality involves the use of reason, which assumes logic. If it involves logic, a computer can learn it.
Given that biological and non-biolgical intelligence are likely to converge in humans, it will be us improving our own morality.
What I find is interesting is, in the article, there were two times that the writer noticed the computer being wrong. The Oreo - 1920's question and the Creamer - Milk question. Watson answered a question that called for a qualifying U.S. city. Watson said Toronto. Could we be wrong that Toronto isn't an American city? I mean, maybe Watson doesn't believe that the Toronto Purchase of 1787 was valid. And, considered the arc and details of history, maybe it wasn't.
I find it interesting that scientists and programmers are consciously attempting to reproduce intelligence artificially. They are attempting to write code and construct components in a fashion so that a computer can think. Human consciousness is attempting to create digital consciousness.
Someone explain to me then why the proposition of intelligent design has caused so much friction in the scientific community. We are generally expected to belief that human consciousness and the laws that define the cosmos are accidental. Many experts are ready to contend that a greater consciousness (a.k.a. God) is non-existent and that either the universe or multiverse is simply a base to allow for a small probability to allow for an organic consciousness to evolve out the basic RNA components that are adenine, cytosine, guanine and uracil.
This is the equivalent of saying that the experts that created "Watson" are mindlessly working at their jobs and that it's intelligence simply occurred by the chance that enough programmers and scientists worked on the project for a computer to succeed at Jeopardy. Are we to believe that Watson was created without the conscious effort of its creators? That notion would be foolish.
Therefore if the above is ridiculous, then why are we to believe that the chance and probabilities that exist in the universe are not designed by a larger awareness? Fractals. If smaller patterns are proof that larger patterns exist in the universe then we can assume, through the presence of fractals, that human consciousness is proof of a greater cosmic consciousness.