Photo forensics from reflections

As a postdoctoral fellow in Boston I would usually take the T to work. Each morning I would stare out the window and watch the tunnel walls whiz by superimposed atop reflections in the window of my fellow passengers waiting patiently for their subway stop. On occasion someone’s reflection would catch my eye, and I would look back into the subway car to get a better view. But I would invariably not see them where I thought they would be given the position of their reflection on the window. As a budding image and vision scientist this really bothered me - how could it be that it was so hard to reason about something so simple as a reflection in a window?

Shown below, for example, are two images of the same basic scene. The reflection of the table and garbage can are slightly different (everything else is the same) Can you tell which is correct?

The geometry of reflections is fairly straightforward. Consider standing in front of a mirror and looking at your reflection.

As shown above, an imaginary straight line connects each point on your body with its reflection. These lines are perpendicular to the mirror’s surface and are parallel to one another. When photographed at an oblique angle, however, these imaginary lines will not remain parallel but will converge to a single point. This is the same reason why railroad tracks which are parallel in the world appear to converge in a photograph (and, in the photo below, notice that several pairs of railroad tracks and even the contours of the train - all parallel to one another - converge to a single point).

This geometry of reflections suggest a simple forensic technique for verifying the integrity of reflections. Locate any point on an object and its corresponding point on the reflection, and draw a line through them. Repeat for as many clearly defined object and reflection points as possible. As you do this, you will find that all of the lines should intersect at one point. Shown below are the results of this simple geometric analysis, which clearly reveals the first reflection to be the fake.

Notice that this geometric analysis is exactly the same as that used to analyze shadows - the reason is that in both cases we are exploiting basic principles of perspective projection that dictate the projection of straight lines. 

In practice, there are some limitations to a manual application of this geometric analysis.  Care must be taken to select appropriately matched points on the object and reflection. In addition, if the lines in the image are nearly parallel, then the computation of their intersection is vulnerable to slight errors in selecting matched points. And, it is necessary to remove any lens distortion in the image which causes straight lines to be imaged as curved lines which will then no longer intersect at a single point.

We are developing a suite of forensic tools that will automate and simplify the detection of fakes, one of which will almost certainly rely on the analysis of reflections.

[CGI model credit: Jeremy Birn, Lighting and Rendering in Maya; Train photograph (C) istockphoto.com/Miguel Malo]

Comments

[John SMythe]

Actually, the one element that struck me as incorrect was the light above the garbage can in the first picture. No such light is visible on the wall above the garbage can, hence the only correct image can be #2