• MSN
  • Hotmail
  • More
    • Autos
    • My MSN
    • Video
    • Careers & Jobs
    • Personals
    • Weather
    • Delish
    • Quotes
    • White Pages
    • Games
    • Real Estate
    • Wonderwall
    • Horoscopes
    • Shopping
    • Yellow Pages
    • Local Edition
    • Traffic
    • Feedback
    • Maps & Directions
    • Travel
    • Full MSN Index
  • Bing
  • NBCNews.com
  • TODAY
  • Nightly News
  • Rock Center
  • Meet the Press
  • Dateline
  • msnbc
  • Breaking News
  • Newsvine
  • Home
  • US
  • World
  • Politics
  • Business
  • Sports
  • Entertainment
  • Health
  • Tech
  • Science
  • Travel
  • Local
  • Weather
Advertise | AdChoices
  • Recommended: Scientists identify the mystery killer behind Ireland's potato famine
  • Recommended: Cicada bugfest closes in on the East Coast's cities: How loud will it get?
  • Recommended: Pizza printouts? NASA funds project to make space meals with 3-D printer
  • Recommended: Months after death, Sally Ride wins honors from White House and NASA

Quantum fluctuations in science, space and society, from quarks to Hubble and Mars. Served up by Alan Boyle, NBC News Digital science editor. E-mail Alan, or connect via Facebook, Twitter or Google+.

  • ↓ About this blog
  • ↓ Archives
    • Icons Email E-mail updates
    • Icons Twitter Follow on Twitter
    • Icons Feed Subscribe to RSS
  • 6
    Dec
    2011
    2:29pm, EST

    Computer mimics human ability to match images

    Thibault Camus / AP

    In this file photo, tourists pose in front of the Eiffel Tower in Paris. A computer can match up photos and paintings based on the uniqueness of features such as the Eiffel Tower.

    By John Roach, Contributing Writer, NBC News

    Every year, thousands of tourists stand in front of Paris' Eiffel Tower to have their picture taken, painted, or sketched. Though every image is different, each contains the sky piercing tower. Now, a computer can match up all those images based on that one identifying feature.

    This could be useful, for example, to someone who is wondering how the Eiffel Tower and its surroundings have changed since their grandparents had their picture painted in front of it on their honeymoon. In this case, the computer could find a match to the painting by searching online for a modern match.

    The technique differs from photo-matching methods that focus on similarities in shapes, colors, or composition, which work well when searching for exact or very close matches but fail when applied across domains, such as a picture taken in different seasons or a painting and a picture.


    "The language of a painting is different than the language of a photograph," Alexei Efros, an associate professor of computer science and robotics at Carnegie Mellon University in Pittsburgh, Penn., said in a news release. "Most computers latch onto the language, not what's being said."

    In the video below that explains how this all works, for example, a standard computer algorithm tasked to find images similar to a painting of a temple returns images of clouds and the ground that most closely match the image, not the temple that's of most interest to humans.

    The goal of this work is to find visually similar images even if they appear quite different at the raw pixel level. This task is particularly important for matching images across visual domains, such as photos taken over different seasons or lighting conditions, paintings, hand-drawn sketches,

    Watch on YouTube

    Efros and his colleagues programmed a computer to find the unique element that sets an image apart from others in a sample and then uses that uniqueness to match it with similar images. 

    The uniqueness is computed based on a dataset of randomly selected images. So, to use the Eiffel Tower example, the person standing in front of the tower is likely similar to other people in other photos and thus given little weight, but the tower itself is unlike anything else in the other photos.

    Efros said the approach is the "best approximation" yet achieved to how humans compare images.

    The technique can be used for automated image searches, for example, or combined with a GPS-tagged photo collection to determine where a particular painting of a landmark such as the Eiffel Tower was painted.

    In the following video, the team shows off how the program can also be used to assemble what they call a "visual memex" — a dataset that explores the visual similarities and contexts of a set of photos. It shows a graph of 200 images of Medici Fountain, another Paris landmark, from various distances. 

    This video demonstrates Visual-Memex graph traversal. Graph is built using our similarity metric.

    Watch on YouTube

    More on photo matching technology:

    • Police use facial recognition technology to match rioters
    • How to turn off Facebook's creepy face recognition
    • Google image search can't tell Obama from Bush
    • U.S. spies seek geotagging software

    Efros and colleagues will present their findings Dec. 14 at SIGGRAPH Asia, a computer graphics and interactive techniques conference in Hong Kong.

    John Roach is a contributing writer for msnbc.com. To learn more about him, check out his website. For more of our Future of Technology series, watch the featured video below.

    Kids' play has moved to tablets and PCs. In this new age, toy makers and researchers alike are sorting out the benefits — and detriments — of playful educational interaction in virtual space.

     

    5 comments

    A human child can draw a crude picture of a pig out of magic marker. Any other human child (who knows what a pig is) can recognize it. A computer can't look at that crude drawing, compare it to a database of actual photographs of animals, and determine it is a pig. It might try to match the colors f …

    Show more
    Explore related topics: search, science, image, video, photo, innovation, featured, featur

Browse

  • featured,
  • science,
  • space,
  • images,
  • nasa,
  • innovation,
  • cosmic-log,
  • video,
  • john-roach,
  • tech-science,
  • mars,
  • new-space,
  • daily-dose,
  • technology,
  • energy,
  • participation,
  • environment,
  • whimsy,
  • holiday-calendar,
  • planets,
  • on-the-fringe,
  • archaeology,
  • physics,
  • spacex,
  • curiosity,
  • moon,
  • books,
  • msl,
  • politics,
  • aurora,
  • hubble,
  • sun,
  • robot,
  • religion,
  • japan,
  • 3-d,
  • genetics,
  • iss,
  • movies,
  • astrobiology,
  • saturn,
  • automotive,
  • updated,
  • evolution,
  • shuttle
Also
Advertise | AdChoices

John Roach, Contributing Writer, NBC News

John Roach is a contributing writer for NBC News. From climate change and mass extinctions to human evolution and deep space, his writing explores life on Earth and its place in the universe. He was a staff writer at the Environmental News Network for several years and has contributed to National Geographic News for more than a decade.

Archives

  • 2013
    • May (36)
    • April (55)
    • March (53)
    • February (44)
    • January (45)
  • 2012
    • December (67)
    • November (12)
    • October (39)
    • September (43)
    • August (62)
    • July (45)
    • June (51)
    • May (46)
    • April (40)
    • March (56)
    • February (63)
    • January (66)
  • 2011
    • December (89)
    • November (73)
    • October (62)
    • September (67)
    • August (61)
    • July (70)
    • June (82)
    • May (86)
    • April (69)
    • March (94)
    • February (67)
    • January (82)
  • 2010
    • December (118)
    • November (62)
    • October (82)
    • September (63)
    • August (62)
    • July (54)
    • June (83)
    • May (51)
    • April (31)
    • March (35)
    • February (36)
    • January (35)
  • 2009
    • December (42)
    • November (34)
    • October (35)
    • September (40)
    • August (32)
    • July (38)
    • June (45)
    • May (37)
    • April (42)
    • March (38)
    • February (37)
    • January (35)
  • 2008
    • December (33)
    • November (31)
    • October (42)
    • September (48)
    • August (35)
    • July (37)
    • June (42)
    • May (43)
    • April (40)
    • March (39)
    • February (42)
    • January (42)
  • 2007
    • December (29)
    • November (40)
    • October (57)
    • September (35)
    • August (47)
    • July (38)
    • June (44)
    • May (44)
    • April (43)
    • March (40)
    • February (41)
    • January (47)
  • 2006
    • December (45)
    • November (49)
    • October (39)
    • September (50)
    • August (58)
    • July (45)
    • June (56)
    • May (8)

Most Commented

  • Why sign up for a one-way Mars trip? Three applicants explain the appeal (327)
  • Virgin birth or hanky-panky? Anteater mom sparks a scientific debate (92)
  • Curse or coincidence? Scientists study Tornado Alley's past and future (123)
  • Buggy hordes of cicadas sighted in Virginia ... but New York? Not yet (77)
  • Dolphins persuade Navy trainers to dredge up 130-year-old torpedo (45)
  • Months after death, Sally Ride wins honors from White House and NASA (63)
  • Pizza printouts? NASA funds project to make space meals with 3-D printer (38)

Other blogs

  • The Body Odd
  • Red Tape Chronicles
  • PhotoBlog
  • US News
  • Open Channel

NBCNews.com top stories

3147,10
© 2013 NBCNews.com
  • Science on NBCNews.com
  • About us
  • Contact
  • Help
  • Site map
  • Careers
  • Closed captioning
  • Terms & Conditions
  • Privacy policy
  • Advertise