FourMatch: A 15-year journey

From criminal, civil, divorce, and malpractice, to patent infringement cases, our courts rely on the integrity of digital media as evidence. Readers of this blog don’t have to be reminded that digital evidence is vulnerable to manipulation, which raises the question: How do we evaluate the integrity of digital evidence in legal proceedings?

More than 15 years ago, fresh out of graduate school and beginning my postdoctoral fellowship, I stumbled upon the Federal Rules of Evidence. I was particularly interested in Article X, Rule 1001 (c): 

An “original” of a writing or recording means the writing or recording itself or any counterpart intended to have the same effect by the person who executed or issued it. For electronically stored information, “original” means any printout — or other output readable by sight — if it accurately reflects the information. An “original” of a photograph includes the negative or a print from it.

I was startled at the lack of more rigorous standards for the introduction of photographic evidence. Of course, these are only the US Federal guidelines; different jurisdictions will have different guidelines that may be more or less rigorous. Nevertheless, even 15 years ago, it struck me that the ever increasing sophistication and power of digital photography and photo editing were going to lead to complex issues of authenticity in our courts.

And so, as I was completing my postdoctoral fellowship and during my first year as a Professor at Dartmouth, I started to think about this problem of photo authentication. One of the first forensic techniques that I developed was based on the observation that the JPEG standard affords a remarkable flexibility in precisely how an image is compressed. In particular, JPEG is a lossy compression scheme meaning that engineers must decide how they want to balance image quality and file size. The JPEG standard provides hundreds of parameters that can be tuned to meet a device’s specific needs. In theory there are more than 10250 possible configurations of these parameters — far more than the estimated number of atoms in the observable universe. It struck me, therefore, that these parameters could provide a highly distinct signature for each camera.

In practice, however, I wasn’t sure if different devices would in fact have much variation in their choice of compression parameters. Perhaps there was some relatively well accepted settings that had become an industry standard, in which case the hope of a distinct camera-specific signature would be lost. In the earliest evaluation of this idea, I strolled down the halls of the Computer Science Department at Dartmouth asking faculty and students to send me a few pictures from their camera. I extracted the compression parameters from a few dozen different cameras. I was hoping for some variation, but was surprised to find that each camera produced a unique set of parameters — even different models from the same manufacturer.

It was clear to me that this forensic technique had some real potential. So I started to collect images, lots and lots of images. Over the following years I collected (and continue to collect) tens of millions of images from thousands of different cameras, mobile devices, tablets, software products, and on-line services. From these images I created a database of JPEG signatures that could be used to determine if a JPEG file was consistent with its purported recording device. The resulting database, I believed, would prove useful in determining if an image is an original, having undergone no alteration from the time of recording—precisely the type of analysis that I originally thought might be needed in the courtroom.

After many years of research and about a year and a half of development, we have released FourMatch, a commercial version of this forensic analysis. 

After all this, it may seem that there is no need for FourMatch. Given the US Federal evidentiary guidelines, nearly any representation of a photograph is admissible. In my experience, however, these rules are at odds with the general concern that judges and juries have over the ease with which digital images can be altered. I have testified in and consulted on many cases in which either defense, plaintiff, or prosecution has raised the specter of photo tampering to cast doubt on a key piece of evidence. In such cases I have used earlier incarnations of FourMatch to lay to rest or confirm, as the case may be, these suspicions.

As an academic, research usually stops after the publication of a scientific paper. In the field of forensics, however, I have long believed that in order to truly have an impact, we have to make the results of our research available in the form of reliable and easy to use software. Fifteen years after I started to think about this problem, it is my hope that FourMatch can now help our courts (and others) contend with the critical need of assessing photo authenticity.