Many of today’s smart phones and digital cameras geo-tag images with GPS data that can be used to automatically categorize your images based on location. This information can, of course, be extremely useful in a criminal investigation such as a child abduction. Unfortunately, the GPS information is stored in the image EXIF data, which can be easily altered or removed. I will describe a technique that can, using only the visual image content, narrow the likely locations where an image was taken.
Geo-localization based only on visual content is an incredibly difficult problem. The most significant advances in geo-location leverage the collective information found in large-scale photo collections (flickr, photobucket, etc.). In a particularly nice paper by Hays and Efros, IM2GPS: estimating geographic information from a single image, the visual content of an image is compared against a large collection of geo-tagged images (in this case, over 6 million flickr images). Images with similar appearance are clustered together, and their GPS locations averaged. Each cluster yields a single location on a map which are rank-ordered based on how similar each cluster is to the original image.
The efficacy of this approach is critically dependent on the accuracy of the image matching, which over the past few years has gotten very good. Shown below, for example, is an image taken in Barcelona Spain (upper left). Shown along the bottom are the top matches for this image (ordered from left to right), and shown on the right are their physical locations. Although the range of possible regions is relatively large, geo-location was able to correctly narrow the likely location of the image to Western Europe, with the most likely location being in nearby Madrid Spain.
Being able to determine the location of an image in the absence of any identifying EXIF data can be an incredibly powerful forensic tool. By leveraging the collective information in large-scale photo collections, purely visual-based geo-location is becoming a reality. As these photo collections expand, there is little doubt that this type of forensic technique will become more effective.