No Location Metadata? No Problem.

Many of today’s smart phones and digital cameras geo-tag images with GPS data that can be used to automatically categorize your images based on location. This information can, of course, be extremely useful in a criminal investigation such as a child abduction. Unfortunately, the GPS information is stored in the image EXIF data, which can be easily altered or removed. I will describe a technique that can, using only the visual image content, narrow the likely locations where an image was taken.

Geo-localization based only on visual content is an incredibly difficult problem. The most significant advances in geo-location leverage the collective information found in large-scale photo collections (flickr, photobucket, etc.). In a particularly nice paper by Hays and Efros, IM2GPS: estimating geographic information from a single image, the visual content of an image is compared against a large collection of geo-tagged images (in this case, over 6 million flickr images). Images with similar appearance are clustered together, and their GPS locations averaged. Each cluster yields a single location on a map which are rank-ordered based on how similar each cluster is to the original image.

The efficacy of this approach is critically dependent on the accuracy of the image matching, which over the past few years has gotten very good. Shown below, for example, is an image taken in Barcelona Spain (upper left). Shown along the bottom are the top matches for this image (ordered from left to right), and shown on the right are their physical locations. Although the range of possible regions is relatively large, geo-location was able to correctly narrow the likely location of the image to Western Europe, with the most likely location being in nearby Madrid Spain.

[Photo credit: James Hays and Alexei A. Efros]

Being able to determine the location of an image in the absence of any identifying EXIF data can be an incredibly powerful forensic tool. By leveraging the collective information in large-scale photo collections, purely visual-based geo-location is becoming a reality. As these photo collections expand, there is little doubt that this type of forensic technique will become more effective.

Comments

[Axel]

Now turn 90 degrees left or right and take a picture of the brick wall. Then try your algorithm again and tell us how well it worked.

[There are, of course, limits to this approach to geo-localization -- the image must contain features that are distinct and that are present in the geo-tagged image database. -Hany]

[Petter]

Almost all cameras at least attaches a time stamp to the image. If the sun is visible, its direction combined with the time stamp could perhaps be useful when trying to infer the location of the photo.

[In addition, general weather patterns can be useful in geo-localization. See, for, example this nice work by Jacobs, et al. -Hany]