When can a photo be trusted?

Fourandsix on TwitterFollow @Fourand6
« Technology + Journalism = Chaos | Main | Who Took That Picture? (Or at least, how tall was the photographer?) »

Photo Forensics from Shiny Objects

As I was walking to get lunch the other day, the top of a parking meter caught my attention (I’m weird, I know). It caught my attention because of the way that the highlight on the top of the meter moved as I walked past it. The highlight moved, of course, because the relative position between me, the meter, and the sun was changing. At the same time, the overall shading on the meter stayed the same. It struck me that the highlight and shading can each be used to reason about the location of the light source. It also struck me that digitally editing an object with a highlight could be tricky because it would be hard to make sure that the highlight, shading, and shadows are all physically consistent.

To illustrate how this might work consider this rendered scene in which one of the red spheres was copied from one side to the other. The specular highlight and lighting on the original sphere is consistent with a single point light source (and a secondary overhead area light). By moving the original sphere from one side to the other, I’ve made it so its appearance is physically implausible. Can you tell which is the original and which is the duplicate?

The sphere on the right is the duplicate. The location of the specularity and the shading on this sphere are physically implausible (assuming that the image was not cropped in the horizontal direction — it was cropped vertically). 

To see why, consider the diagram on the right. Shown is a top-view schematic of the rendering of a red sphere placed to the left of the camera. The white circle corresponds to the specular highlight. The yellow arrow points to the light and the black arrow points to the camera. The basic rules of physics tells us that the specularity will appear at a location where the surface normal bisects the angle between the light and camera. The surface normal (red arrow) is a line at any given point on the sphere that is perpendicular to the sphere’s surface.

In this example the specularity is on the front of the sphere, similar to the rendered sphere shown above. Given that this sphere is to the left of the camera, we know that the light must be to the left of the sphere. Notice that for the left-most rendered sphere above the shading is consistent with a light in this location. That is, the left side of the sphere is lighter than the right side (this is most noticeable by looking at the top right and bottom left edges of the sphere).

Now, shown on the right is a schematic illustrating the location of the camera and light that would have given rise to the specularity for the duplicate sphere translated to the right. Notice that in this case, the light is to the right of the sphere. If this were the case, then the right side of the sphere should be lighter than the left side. It is not and therefore can be seen to be the fake.

Shown below is a physically correct rendering of the right-most sphere consistent with the schematic shown to the right. As compared to the above version, notice that the highlight is in approximately the same position, but that the shading is measurably different. In particular, the shading on the right side of the sphere is lighter than on the left, consistent with a light source positioned to the sphere’s right.

In practice reasoning about specular highlights can be tricky because it requires knowledge of 3-D surface normals, which are not always easy to estimate from a single image. Given familiar shapes, however, we can often reason qualitatively about the surface normal at a specular highlight. From this, we can then reason about the relative positions of the light and camera to see if they are consistent with the shading and shadows in the image. This approach can be considerably strengthened if you can build 3-D models as in The Case of the Pole in the Middle of the Road or Photo Forensics from Eyes.




PrintView Printer Friendly Version

Reader Comments

There are no comments for this journal entry. To create a new comment, use the form below.

PostPost a New Comment

Enter your information below to add a new comment.
Author Email (optional):
Author URL (optional):
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>