Not the Null Hypothesis

In my years at Adobe, it wasn’t uncommon to be asked if there were ways to determine when a photo had been modified. Often, this would happen when there was some notable scandal involving a doctored photo in the media. When I was asked that question, my response always centered on looking for mistakes in photo editing. The fact is that it’s hard to make a truly convincing photo composite, and while there are many people who can master some of the key tools in Photoshop, there are far fewer who know how to use them effectively, and who have a sufficient understanding of perspective, lighting, and other fundamentals. Thus, I would say, the key is usually to look for sloppy mistakes—many of which we’ve covered previously in this blog. Is the lighting consistent? Is the texture or grain of the image the same throughout? Are there any slight halos to indicate where someone might have been cut from one background and pasted into another?

Of course, this approach helps you to spot many false images, but it can never give you a definitive answer as to whether something is a fake. And it does take a trained eye to distinguish real signs of tampering from normal image inconsistencies, such as JPEG artifacts. If someone is really an expert forger, they can create an image that escapes detection—at least by the unaided eye. Nevertheless, most of the images that have caused controversy do have such telltale indicators, and in fact it’s usually an eagle-eyed member of the public who spots the inconsistency and reveals the forgery.

Probably close to ten years ago, however, I first got asked a related question that caught me off-guard: How can you prove that an image is authentic? At first, that might sound like the same question, but it’s really not. Based on the methodology I had been giving people, saying that an image is authentic was a bit like a null hypothesis: it was the statement you would try to disprove by finding evidence of tampering. But, by definition, a null hypothesis is not something that can ever be proven, only disproven. The absence of evidence that something is false is not the same as proof that it is true.

Why was it so important to prove that an image was true, as opposed to just being able to prove whether it was false? The key event that made this such an important issue was a U.S. Supreme Court ruling in 2002 (Ashcroft v. Free Speach Coalition) which essentially said that simulated child pornography that did not involve the participation of real children is protected under free speech. Suddenly, recovering pornographic images that appeared to involve children was not necessarily enough to convict a child pornographer. Instead, it became the responsibility of the prosecution to show beyond a reasonable doubt that the images truly involved minors, and were not simply synthetic computer graphics or heavily modified photos featuring only consenting adults.

Unfortunately, at the time, I simply didn’t have an answer to the question. I knew lots of ways to look for evidence of tampering, but I didn’t know of any effective ways to certify that any random photo was likely true. Sure, companies like Canon and Nikon both offered image authentication systems, but these solutions only worked with their professional cameras, and, more importantly, only in a controlled capture and processing environment. These authentication solutions simply weren’t designed to handle this problem. (Plus, as was covered in the news last year, they may also be subject to hacking.)

While I was fielding such questions at Adobe, however, my Fourandsix co-founder, Hany, was busy at Dartmouth exploring the issues around detecting photo tampering from every angle. Rather than looking for a single magic bullet that would automatically identify any image as true or false, he was appropriately studying the issue like a detective, identifying all of the various forms of evidence that could be analyzed to reveal the past history of an image. Importantly, he was identifying not only the evidence that can indicate that an image may have been modified, but also the evidence that can indicate that it was not. Indeed, there are characteristics of images first captured by a camera that tend to change once they’ve been touched by software in any way. 

Our task at Fourandsix is to implement these various detection techniques in software so that they can be put in the hands of the people who need them. This is not about a single technique or even a single product, so what you should expect to see from us is an expanding collection of tools over time. But we need to start somewhere, and I find it a little ironic that the place we’ve chosen to start is with what I once thought was an unanswerable question. More to come…