When can a photo be trusted?

Fourandsix on TwitterFollow @Fourand6
« Lighting in Video Games | Main | Photo Forensics from Vanishing Points »

Image Search for Image Forensics

Last week I consulted on a case for a federal law enforcement agency. At issue was whether a pair of images were real or not. The photos seemed to depict a gruesome crime. If real, then law enforcement agents needed to investigate, but if they were fake then the agents obviously didn’t want to waste their valuable and scarce resources. This case reminded me of the usefulness of internet-based image searches and reverse image searches.

I initially spent some time looking for traces of tampering in the photos, but found no obvious inconsistencies. I was, however, still suspicous that the photos were real because they depicted such improbably gruesome contents. A quick image search with the appropriate key words found one of the two images posted to a chatroom, but the source of the image remained unclear. Working with several agents, we then performed a reverese image search in which an image is specified (as opposed to keywords), and instances of that image are found (see, for example, Tineye and Google). This reverse image search found more instances, one of which revealed the true source of the image: a movie prop. A reverse image search on the other image found that it also was a movie prop — what a relief.

Reverse image search is able to not only find the same instances of an image, but also variations on that image. Shown on the right, for example, are the TinEye results of searching for the image in the top left. Even images with significant additions, deletions and modifications to the original are found.

In an image forensic setting it can often be helpful to determine the provenance of an image. In addition, doctored photos might be composed of parts of other images that themselves can be found on-line. As reverse image search technology improves, these tools hold the potential to be highly useful to forensic analysts.

One limitation of this technology is that sensitive evidence cannot be uploaded to a commercial website. (I only uploaded an image once a keyword search made it clear that the image was already in the wild.) Instead of uploading an image to a reverse image search engine, the image features that are used for searching could in theory be extracted locally and then uploaded to the service. Uploading these features would, presumably, be less sensitive than uploading the entire image. It remains to be seen if any on-line services will be able to offer such a technology.


PrintView Printer Friendly Version

Reader Comments

There are no comments for this journal entry. To create a new comment, use the form below.

PostPost a New Comment

Enter your information below to add a new comment.
Author Email (optional):
Author URL (optional):
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>