Cloning: Biological or Photoshop?

Over the years I have worked on several scientific misconduct cases and have been surprised at the egregiousness and scope of the misconduct.  From the biological to the astronomical, many scientific disciplines now rely on some form of scientific imaging. As a result, conclusions in scientific studies often rely on interpreting data in digital images. Because scientific images tend to be simpler than photographs of scenes and people, it is often much easier to manipulate them and can be much harder to detect the manipulations.

A common form of image manipulation is to clone (copy and paste) portions of an image to replicate an object or conceal a person in a scene. In every case that I have worked on, some form of cloning was used to alter the published scientific images. Shown in the left panel below is a typical scientific image. The location and/or presence of the dark bands inform the presence or absence of certain proteins or genes in a biological sample. Shown in the middle and right panels are variants of this image in which I have removed a band (first row) and added a band (second row). The removal was done by cloning a part of the gray background and the addition was done by cloning a band from the lower part of the image. These manipulations are incredibly easy to do and very hard to detect visually. These manipulations also hold the potential to radically alter the interpretation of the scientific study.

[Photo credit (original on left): Adam Coster]

It is relatively easy to determine if a suspected area in an image is cloned from another by overlaying the two regions in Photoshop using the “Difference” blending mode. In this mode, pixels that are the same between the two regions will appear as uniformly black, and the pixels that are different will appear progressively brighter (it is helpful to histogram equalize the image before doing this so that subtle differences become more visually apparent). If, however, it is not obvious which regions to compare then the problem becomes much harder. Searching for all possible pairs of regions and region sizes in even a modest-sized image is computationally intractable because there are simply too many combinations to test. In addition, changes in the size, orientation, or color of the cloned region further increases the complexity of the search. Image forensic techniques that detect cloning must, therefore, contend with this difficult computational problem while at the same time distinguishing between actual clones and legitimate regions of an image that are naturally similar (e.g., the gray background in the above figure). There has been some excellent progress on such forensic techniques  — see “Region Duplication Detection Using Image Feature Matching“ for one such example.

There is often a desire by authors to “polish” images and data prior to publication. This may include finding justification to remove an outlier data point, or applying a variety of enhancements to an image. While in some cases, this type of manipulation may not fundamentally alter the scientific conclusions, in other cases, these same easy to perform “enhancements” are used in an outright fraudulent manner.

The impact of fraudulent scientific images can in many cases be more severe than the impact of fraudulent images in the general media, particularly in medical studies that have direct implications for people’s health. Scientific journals could take a greater role in trying to reduce instances of scientific misconduct and fraud. When it comes to images, journals could insist that authors provide an original image along with the final “camera-ready” version. Editors and reviewers can then determine for themselves the appropriateness of any post-processing enhancements. Combined with forensic techniques for validating original images, this simple step would be helpful in reducing instances of scientific fraud.

Comments

[Jim Hoerricks]

Interestingly, the opposite can be true as well and is happening more frequently.

Let's say that you and I are both medical researchers. Your are far more brilliant, of course, and are just steps away from publishing. I am months away - but moving along the same track. We both receive our funding from the federal government. With me?

I contact the government and allege that you are forging your results. No proof - just random allegations about work you've published, are about to publish, etc. The government watchdogs come and shut you down whilst they investigate. Meanwhile, I finish what I am doing and publish - before you.

You are now left to clear your name by proving a negative - that you aren't a fraudster. In a case of who watches the watchers, who the government employs in their investigation, how they conduct their enquiry, and etc. are important. http://ori.hhs.gov/ is but one organization of the government watching research misconduct. http://ori.hhs.gov/droplets - this contains their tools to assist in their enquiries.

In law enforcement, we must pass the gatekeeper (judge) in order to be qualified as an expert and present expert testimony in trial. In government misconduct investigations - who conducts the technical aspect? What's their qualifications? Anyone can download the droplets. I tried using them - they require the use of Free Transform to align ... ? Seriously?! Free Transform?

Nevertheless, I just wanted to point out that fraud can happen in several directions - and sometimes innocent people get caught in the middle trying to prove a negative - trying to prove the Null as it were.

Keep up the good work. I can't wait to send some money your way when your plug-ins are ready.

Best,

Jim

[Jim, you raise a good point -- false claims of scientific misconduct can be incredibly damaging to a lab and to reputations. I would add that in the cases that I have worked, there were several levels of checks and balances throughout the University and investigating government agency. In addition, each investigation including a panel of scientists and at least one forensic scientist such as myself. I have been impressed with the investigating bodies' care, thoroughness and sense of responsibility. These investigations take an enormous amount of time, so I do hope that some of our forensic tools will make it easier to carry out such investigations. -Hany]