As we discussed last week, you can compute the precision of a forensic analysis by considering the true positive and false positive rates of your forensic analysis. The precision tells you the likelihood that the image is a fake given the results of the forensic analysis. What if, after computing the precision, I tell you that the image you are analyzing was downloaded from worth1000 (a popular photo-editing competition website). Surely you are going to want to factor this into your conclusion.
Consider a forensic analysis with a true positive rate of 80% and a false positive rate of 10%. If this forensic analysis detects evidence of tampering, then the precision is 0.8/(0.8 + 0.10) = 88.9%. A third quantity, the prior likelihood, should also be factored into this calculation. The prior is the likelihood that the image is fake or real regardless of the results of any forensic examination (e.g., an image downloaded from worth1000 has a high prior likelihood of being fake, while an image provided by the Associated Press has a relatively low prior likelihood of being fake).
When computing the precision, the true positive and false positive rates need to be multiplied by the prior likelihood that the image is fake or real. Consider again the forensic analysis with a true positive rate of 80% and a false positive rate of 10% applied to an image downloaded from worth1000. Your prior that the image is fake is, let’s say 95%, and of course your prior that it is real is 5%. Now, the true positive multiplied by 0.95 yields a true positive of 76% and the false positive rate multiplied by 0.05 yields a false positive of 0.5%. With these new values, the precision is now 99.3%, considerably higher than the previous 88.9%.
Notice that if the prior is 50%, then it has no effect on the computation — in fact, when we ignore the prior what we are really saying is that the likelihood of an image being fake or real is equally likely. Sometimes this is appropriate, sometimes it is not.
The true and false positive rates are relatively straight-forward to measure. The prior, however, can be more tricky to determine precisely. In general, the reliability of a source (worth1000, flickr, AP/Reuters) can provide a rough measure of confidence in the integrity of the image. Another factor that will help to establish a prior likelihood is the complexity of the scene, the complexity of the purported manipulation and the number of images being analyzed.
Even if you can’t precisely quantify the true positive, false positive, and prior likelihood, some effort should be made on placing reasonable bounds on these values in order to provide a general sense of precision.
I am often asked to analyze images. In almost all cases, I first try to determine the prior likelihood that the image was altered. This establishes an important baseline that must always be factored into an overall forensic analysis. This prior shouldn’t, of course, color your forensic examination—but it should be factored into the overall confidence of your conclusions.