Celebrities Crash a Party

I recently read a few stories about this set of photo composites created by Everett Hiller, who inserts a variety of celebrities into his holiday party photos each year. What impressed me about these photos was that the creator did an unusually good job of making them look — at least at first glance — plausible. I thought these photos would be a good test of some of our image forensic techniques.

In an earlier post (“Photo forensics from shadows”) I described a forensic technique for determining if cast shadows in a scene are consistent with a single light source. When a scene is illuminated by a distant light source (the sun, for example), then lines that connect points on a shadow to their corresponding points on the object should all converge to a single point in the image. Although the sun is obviously not illuminating the scene below, the camera flash did fire (as can be seen by the cast shadow near each person’s body). Shadows generated by a flash or the sun obey the same basic constraint — lines connecting shadow and object must converge to a single point.

[Photo illustration: Everett Hiller (http://imgur.com/a/s6dgU#52)]

Shown below is a shadow analysis applied to the above photo. The green lines connect points on the cast shadow with the corresponding part of the body. As expected, these lines intersect at a single point. Note, however, that the cast shadow under Tom Cruise’s right arm is inconsistent with this location. Even though the location of this shadow is quite wrong, it is not immediately obvious at first glance.

Shown below is another of the celebrity photos, but this time there are no cast shadows on Laurance Fishburne (far left) which can be analyzed. However, his reflection appears in the mirror behind the door and as I described in an earlier post (“Photo forensics from reflections”) lines connecting points on an object and their reflection must intersect at a single point.

[Photo illustration: Everett Hiller (http://imgur.com/a/s6dgU#52)]

Shown below are lines connecting three points on the women’s face to her reflection (green) and two points on Fishburne’s face and his reflection (red). The failure of all of these lines to intersect at a single point means that the reflections are physically impossible. Notice also that there is a highlight in the middle of Fishburne’s head, but in his reflection in the mirror the highlight is on the side of his face.

The creator of these photos added shadows and reflections in a way that was visually compelling. However, our visual system is not always good at seeing the correct shape and location of shadows and reflections. A quick and relatively easy to execute geometric analysis can detect these inconsistencies. Knowing about this analysis may now make it easier to create consistent shadows and reflections — but we have more tricks up our sleeves… stay tuned.

Comments

[Everett Hiller]

So this is very interesting and all, and I agree with some of your techniques. However, the right arm in the first photo was actually there. That is my right arm. I only put his face on my body. As proof, here is the original photo http://i.imgur.com/BGpFY.jpg

As far as the second photo, I have no doubt your system is correct. I just thought most people would see that Gina Torres has a different smile in the reflection. That one was a real pain in the butt to do.

Good stuff though.

[I mistakenly thought that the two cast shadows were from the same light source -- there are several lights in the room (thanks Everett for confirming). So, the analysis is correct (the shadows are not consistent with one light source), but the conclusion is wrong (the shadows are not the result of manipulation). This is a good reminder that care should be taken when applying this type of analysis to an indoor scene where the lighting can be more complex than outdoors. -Hany]

[bob de bouwer]

umm, when you look at an object through a mirror, you will get highlights at other positions, especially if the light source (aka flashbulb) is near the viewpoint (aka the lens of the camera having said flashbulb).

secondly, in the pic with tom cruise, if you were to move one of the red points even fractionally, the inclination of the red line would change drastically. this is because the points that you use to form that line are so close to each other. I wonder what would happen if you kept the red point on tom cruises shirt, and put the 'red' light source on the green one, would the other red point land on the shadow? I predict a very near (but perhaps ambiguous) hit.

same goes for the 'flash points' in the second picture. put the "Real" flash point inbetween the 'red' and 'green' flashpoint, and for all the lines keep the corresponding anchor point in the mirror image. Does the second anchor point of each line end up on the correct spot of the real faces of laurence and whatshername?

[You are correct that care needs to be taken when performing this type of analysis. In the case of Tom Cruise, even though his shadow and sleeve points are very close to each other, there is no way that they intersect with the other light source -- if you draw a line from the yellow dot through the Cruise's shadow, you will see that it doesn't come anywhere near his sleeve. Same thing goes for the reflection. I should add that we have developed a new technique that makes this analysis much more numerically stable to slight mis-alignment errors... stay tuned for more details. -Hany]

[Anonymous]

You do realize that objects in real life are in THREE dimensions, right?