Photo Forensics under Complex Lighting

In previous posts (Photo Forensics from Shadows, and The Case of Geico’s Gecko) I described an image forensic technique for determining if shadows in a scene are consistent with a single light source. This forensic analysis is particularly well suited for outdoor scenes where it can usually be assumed that there is a single light source (the sun). Indoor scenes with typically more complex lighting can, however, be harder to analyze using this technique. More sophisticated models of lighting allow us to analyze a broader range of scenes and to reason not just about the shadows, but also about the overall appearance of objects.

[Photo credit: David Yu]Consider, for example, the interior of Grace Cathedral in San Francisco shown on the right. The stained glass windows create an incredibly complex pattern of illumination and shadows. Unlike the simple task of modeling the sun, a single distant light source, modeling such a complex lighting environment seems overwhelming. In 2001, Ramamoorthi and Hanrahan proposed an elegant framework for modeling complex lighting (see “An efficient representation for irradiance environment maps” for more details). This work was designed to allow for the efficient computer graphics rendering of scenes illuminated by more realistic and complex lighting than that afforded by the then simpler lighting models.

Imagine standing in Grace Cathedral while holding a spherically-shaped umbrella above you. Imagine now looking at each point on the umbrella and recording the brightness and color of light that is striking that one point on the umbrella. Do this for the entire surface of your umbrella. The resulting pattern of light is an “environment map” that captures the complexity of the surrounding lighting. The very nice observation made by Ramamoorthi and Hanrahan is that a broad range of complex lighting patterns can be modeled with a relatively simple and manageable mathematical model. Such a model allows for the efficient transfer of complex lighting from a real-world scene to a computer graphics rendering, and in turn for the creation of more realistic rendered images.

To be useful in a forensic setting we need to estimate the parameters of the lighting model from a single image. Although more complex then estimating lighting and shadows from only the sun, we have shown that lighting environments can be estimated from a single image (see “Exposing Digital Forgeries from 3-D Lighting Environments”). As a result, we can model fairly complex lighting to determine if it is consistent or inconsistent within an image. This analysis allows us to analyze the appearance of objects even when they are illuminated by complex lighting. The other benefit of this approach is that while a reasonably skilled forger may be able to create a visually compelling fake shadow in an outdoor scene, they are less likely to be able to realistically create the correct pattern of lighting and shadows in, for example, Grace Cathedral.