Ever since the assassination of President Kennedy, numerous theories have circulated claiming that Lee Harvey Oswald, the accused assassin, acted as part of a larger criminal conspiracy. It has been suggested, for example, that incriminating photographs of Oswald were manipulated and hence evidence of a broader plot. I have never been particularly interested in these conspiracy theories. I do, however, like that at least one aspect of the theory was testable – were the shadows in the Oswald backyard photo physically plausible or not?
A portion of the argument for photo tampering goes something like this. Consider the shadow cast from Oswald’s body onto the ground. The orientation and length of the shadow suggest that the sun is to Oswald’s left and relatively low on the horizon. The long straight shadow under Oswald’s nose, however, suggests that the sun is directly above him. These seemingly incongruous shadows have led to speculation that Oswald’s head was pasted into the scene. In fact, Oswald himself claimed that the photo was a fake and had been altered to falsely implicate him.
I thought that there was a chance that the photo was fake because it does seem at first glance that the shadows in this photo are inconsistent with one another. I also know, however, that our visual system can be spectacularly bad at judging such things as lighting and shadows in a photo.
In order to reason about the shadows in this scene we need a three dimensional model of the scene (Oswald’s head/body and the ground plane) and the three dimensional location of the sun. In general, determining three dimensional information from a single two dimensional image is an under-determined and difficult problem. Estimating three-dimensional models of a person’s head, however, is relatively easy because of the somewhat constrained and well understood geometry of human heads.
A frontal and profile view are required to build a 3-D model of a person’s head – Oswald’s mugshots were perfect for this. Once constructued, I was able to estimate the 3-D location of the sun. A generic articulated 3-D body model was used to model Oswald’s body. With a model of Oswald’s head and body and the location of the sun, I was able to infer the location of the ground plane. Once all of the pieces were put in place, it became clear that all of the shadows were perfectly consistent. Shown below, from left to right, is the backyard photo, my 3-D rendering, an overlay of the shadows, and a magnified view of Oswald’s head showing the well matched shadow under the nose.
The ability to construct three-dimensional models from a single image is incredibly powerful because it allows us to, among other things, reason about the lighting and shadows in a scene. I predict that the next generation of forensic tools will be increasingly more sophisticated in their ability to construct and reason about the full 3-D scene thus greatly expanding our forensic analyses.