A volunteer and collaborative effort to bring information about shared microscopy facilities to the University of Arizona and the community.

Digital Image Ethics

Douglas Cromey, MS

Longer articles by Mr. Cromey on this topic include a review entitled Avoiding Twisted Pixels: Ethical Guidelines for the Appropriate Use and Manipulation of Scientific Digital Images (2010) and a book chapter entitled Digital Images Are Data: And Should Be Treated as Such (2012). See also the ORI's Online Learning Tool for Research Integrity and Image Processing (2008, Mr. Cromey was a contributing author). This page is also available as a PDF handout.
Introduction to Image Editing Ethics

This topic is increasingly on people's minds given that image manipulation "tricks" that used to take considerable skill in a darkroom now can be done quite easily by anyone using one of the powerful image editing programs that are available. A user does not even have to be intentionally malicious to alter an image in an unethical manner. Unfortunately, many users are unaware of the issues or the effects of their actions.

Journalists have grappled with the credibility problems created by altered images since the early days of photography (see: Faking Images in Photojournalism). In western society a photograph is typically assumed to be an accurate representation of reality, unless it is patently obvious that it has been altered (e.g., SPY Magazine's cover photo of a "pregnant" Bruce Willis in September 1991). Most readers seem to understand and expect that widely respected sources of information will adhere to a higher standard of photojournalistic ethics than sources such as "tabloid newspapers".

Scientists are usually considered to be respected sources of information and there is the understanding within the scientific community that data must not be inappropriately manipulated or falsified. When this essay was first composed in 2001, there were very few written guidelines for scientists. Now some of the major professional societies have issued policy statements regarding digital imaging, and many scientific journals have revamped their instructions to authors to provide clearer guidance of how they require images to be handled. Publications like the Journal of Cell Biology have been testing images in accepted articles since the early 2000s to ensure compliance with their guidelines and the Office of Research Integrity (HHS) continues to watch this issue closely.

In this author’s experience the inappropriate manipulation of scientific digital images typically does not arise from an intent to deceive or to obscure information. More often the inappropriate manipulations are simply due to ignorance of basic principles. It seemed to this author that often what is needed is an explanation of why manipulations are right or wrong. These twelve guidelines are an attempt to address this issue. It should be noted that the author has extensive experience in the microscopic imaging of biological specimens and these guidelines reflect his personal experience in this field.

Guidelines for the proper acquisition and manipulation of scientific digital images:

Images are data arranged spatially in an XY matrix (or grid) and each individual element (pixel) has a numerical value that represents a grayscale or RGB intensity value. These data are a numerical sampling of the specimen as presented by the data acquisition system (e.g., microscope) to the sensor (e.g., CCD camera). The data acquisition system and sensor are subject to all the limitations and aberrations that physics and instrument design may impose on the two devices. To the observer's eye the image data may appear to accurately represent what can be seen, however, it is the user's responsibility to understand the limitations of the particular instrument.

The basic message is that humans are not very good observers, that our vision system ignores a lot of information, that having names and labels for recognized features is very important, and that we often think we see what we expect to see. - Dr. John Russ (1)

 

The original raw data file is the standard to which the final image can and should be compared. Maintaining a copy of the unaltered original image is the user’s only protection against accusations of misconduct. This is also the only way that users can recover from a mistake in image processing. Data should be archived to media that are not easily altered (e.g., CD-R or DVD-R) (2). Maintaining the image in the original file format is highly recommended.

Individual’s and corporations whose research falls under the United States FDA’s “Final Rule on Electronic Records and Electronic Signatures” (21 CFR part 11) have mandatory requirements for maintaining the integrity of the original image. This would include labs using “Good Lab Practices”. Other industries where maintaining the original image is required would include; forensics (rules of evidence) and health care (liability, HIPAA).

 

This would include techniques that are similar to standard darkroom techniques (e.g., different contrast grades of paper, changes in development time). With digital images this would include performing "reasonable" adjustments of the levels and gamma settings. Because changes in gamma are non-linear, many journals are requiring that these types of adjustments be described in the figure legend or the methods section.

Small adjustments to the brightness and contrast are usually acceptable, however, large adjustments are not recommended. This is because it is very easy to truncate intensity information in the image using brightness and contrast.

 

Avoid acquisition bias. Capturing images that only confirm the lab’s “preferred hypothesis” is a form of unethical cropping. Consider the following observation by microscopy core facility director Dr. George McNamara.

I suspect that most published micrographs are "exemplary", "best of", or, "the only one we took", or "the only one that fit our hypothesis" (I call the latter two categories, "N=1 experiments").

If you are putting together figures, and you select for publication a micrograph based on any of these categories, at least be honest to the reviewers and editor and say so (hopefully they'll tell you to go back and collect data correctly ... even better, your coauthors should tell you ... best of all, your inner super-ego should tell you).

What you should be publishing are representative micrographs. That means you need to acquire sufficient images to document/quantify the experiment. Your specimen and images should be good enough that any of the micrographs can be used. In fact, if you can only publish one micrograph per treatment group, use a random number generator to pick which one...
Dr. George McNamara (3)

After you have selected a specific image to use in a figure, what is your motivation for cropping that image? Is it to improve the “composition” of the image, or is it to hide something that disagrees with the hypothesis?

Remember to leave yourself enough pixels so that the image will reproduce well in a scientific journal. If you have to crop too much out, it’s time to re-image your specimen. Do not let Photoshop replace good science.

 

Any processing of images that are to be compared should be identical, especially if they will be published as a group of images in a single figure. If there is a compelling reason that the images in a figure were processed differently, this must be explained in the publication or figure legend. Honesty is the best policy.

If background subtraction or white-level balancing (to compensate for uneven illumination, etc) was performed, this should be acknowledged in the methods section.

 

This would include techniques analogous to "dodging" and "burning" in a photographic darkroom. This is a disputed issue. Purists would state that selective enhancement should never be performed; however, there are very rare occasions when it is legitimate to enhance a specific area in an image.

Honesty is the best policy. If portions of an image for publication were selectively enhanced, the author should state it clearly in the figure legend. 

 

Commercial software designed for desktop publishing cannot be counted on to appropriately and scientifically manipulate the data in a digital image. Digital image filters are typically mathematical functions (convolution kernels) that change the numerical data in the pixels in the image. If the filters are not used carefully, they may create artifacts in an image that can lead to misinterpretation of the data. If filters must be used, they should be noted in the figure legend of published images. The note should include software version, specific filters and any special settings that were used. 

Software filters/Convolution kernel mask tutorial – Choose the sharpening kernel, then press AUTO to start the tutorial. Watch how the filter changes pixel values at every single pixel and compare the before and after values in the small images.

Software filters and to some extent “cloning” (#8) are sometimes used to clean up the background of an image. Scientists must always remember the possibility that someone will look at their data in a way they hadn’t considered. Perhaps the reader will find that the collagen matrix, support media, interface between two structures, or other “unimportant” features in the image contains information that will spark an idea for their research. If the author changes the “unimportant” things to enhance the “important” things, they have lied to the reader.

 

Users often consider using the technique of cloning sections of an image to "clean up" a dirty preparation. If the image requires this much processing, the best solution is to go back and take another image from the sample or a new sample prepared under the same conditions. The use of cloning techniques to create objects in an image that did not exist there originally (e.g., "creating" a new gel band) is completely unethical.

Use of cloning and/or copying is asking for trouble. Most of the falsified image cases that the Office of Research Integrity sees use these techniques. Professional journals that closely examine images (e.g., Journal of Cell Biology) can detect these sorts of things pretty routinely.

Combining images (e.g., two similar gels combined into one figure) is acceptable at most journals only if it is clear to the editors & reviewers that the two images are from separate sources. Often this means a small gap between the two images or a black line that delineates the two images. Scientifically, it is better to re-run the experiment, rather than paste images together.

 

Be aware that some instruments (e.g., fluorescence microscopes of many types) are subject to a number of known fluctuations over time as well as having other physics/electronics limitations. If you are unaware of, or can’t account for, the limitations of the acquisition instrument, you should not be performing intensity measurements.

Users of fluorescence microscopes should read: The 39 Steps: A Cautionary Tale of Quantitative Fluorescence (4), Seeing is believing? A beginners’ guide to practical pitfalls in image acquisition (5), and Multicolor imaging: the important question of co-localization (6).

 

There are very few good reasons to use the JPEG file format on scientific digital images (other than displaying an image on a web page). JPEG compression uses the discrete cosine function to reduce the file size, however, it also changes the XY resolution of the image and the intensity value of any given pixel.

If you must use JPEG, perform the compression as the last thing that is done to an image. With most image manipulation programs, opening and saving a JPEG image multiple times runs the compression algorithm on the image multiple times, further degrading the image each time. 

Important - Users of the Adobe Acrobat writer software should be aware that the default setting in this program is to apply JPEG compression to any images embedded in the document. These settings can be changed by the user.

…many aspects of scientific and industrial usage involve subsequent processing of a digital image, for example to enhance features or count items. Using any form of lossy compression for images in this context may create problems - after all the information thrown away during lossy compression is generally that information that is imperceptible to a human eye - not necessarily showing the same characteristics as computer image processing software.
Joint Photographic Experts Group (JPEG) (7)

The reason for recording images in scientific studies is not to keep remembrances of familiar objects and scenes, but to record the unfamiliar. If it is not possible to know beforehand what details may turn out to be important, it is not wise to discard them. And if measurement of features is contemplated (to measure size, shape, position or color information), then lossy compression, which alters all of those values, must be avoided.
Dr. John Russ (1)

It is tempting to acquire your image files in JPEG format to save disk space, but doing so compromises your data. Always use TIF format.
Journal of Cell Biology (8)

Even with large scientific image formats the cost of storage is vanishingly small. It, therefore, makes no sense not to save an original unprocessed and uncompressed image file. The MSA (Microscopy Society of America) format for this storage is the TIFF file format.
J.M. MacKenzie, M.G. Burke, T. Carvalho & A. Eades (2)

JPEG image compression artifacts tutorial – Select a sample image from the list. The two images should look virtually the same. Now select the “difference image”, which is the mathematical subtraction of the pixel intensities of the JPEG image from the original. If the images were truly identical, there would be no difference. The difference image demonstrates that JPEG compression causes intensity information to be spread out from its origin.

 

Digital images of real world objects sample an object in a way such that each pixel in the image has a scale. This scale may be in meters per pixel for satellite images or in tenths of microns per pixel for microscope images. Ideally the scale is the same in both the X and Y dimensions; however, this is not always the case. The magnification of the image is determined by the difference between the original scale of the pixel and the scale of the pixel in its final form (e.g., paper printout, projected on the wall of a large lecture hall). Since it is often impossible to know in advance what the final magnification will be, a scale bar of known size is the best way to express the magnification. Journals may resize your image for publication, so providing a numerical magnification number in a figure legend may result in errors.

The ability of a microscope to resolve (separate two small, adjacent objects) is limited by the wavelength of light used and the numerical aperture of the objective lens (Rayleigh criterion).

In most cases, to ensure adequate sampling for high-resolution imaging, an interval of 2.5 to 3 samples for the smallest resolvable feature is desirable.
Spring, K.R., Russ, J.C., Parry-Hill, M.J., Fellers, T.J., Zuckerman, L.D. & Davidson (9)

Note that this statement means 2.5-3 samples (pixels) should be used to capture the smallest resolvable features in each of the three spatial dimensions (XYZ). Other dimensions, such as time and/or wavelengths, should also be correctly sampled to avoid artifacts. Undersampling (using too few pixels to describe a spatial feature in a sample) can lead to artifacts masquerading as real structures. Oversampling is not as problematic, however, it should be noted that oversampling does not yield any additional spatial resolution information from the specimen. In many types of fluorescence microscopy oversampling may result in a loss of contrast (due to limited amounts of light) and without contrast it is difficult to resolve closely adjacent objects.

Nyquist sampling is an important and complex technical point. For more information on this topic, see:
http://olympus.magnet.fsu.edu/primer/digitalimaging/digitalimagebasics.html
http://micro.magnet.fsu.edu/primer/java/digitalimaging/processing/spatialresolution/

 

Changing the size of an image (the number of pixels in X and Y) can introduce resampling artifacts. Decreasing the image size (downsampling) can cause the XY resolution in an image to be greatly reduced. If the size reduction is not by a power of two, the software program has to be "creative" in determining the intensity values of each pixel (guessing). Using a power of two is slightly better, since this is a form of averaging, and while the resolution is still decreased, it is decreased in a more reproducible manner.

Increasing the image size (upsampling) causes the software to interpolate (guessing) to "create" pixels in between the existing pixels. Upsampling an image does not increase the resolution, in fact it may make it more difficult to resolve features because of aliasing artifacts. In either case, users should insert a magnification scale bar prior to resampling (magnification may be nearly impossible to calculate afterwards).

Users should only change the total number of pixels in an image one time to avoid compounding any artifacts that might be created.

Adobe Photoshop tip: If you are only changing the dpi of the image for different output devices (e.g., printers), uncheck the resample image box found at the bottom of the window that appears when invoking the IMAGE|IMAGE SIZE menu item. By doing this you change the scale of the image (72 dpi, 300 dpi, etc) without changing the number of pixels in the width or height boxes. For more information, see Potentially the most dangerous dialog box in Adobe Photoshop™.

 

At the 2003 summer council meeting of the Microscopy Society of America the following resolution was adopted as representing the position of the MSA on ethical digital image processing (as published in Microscopy Today Nov/Dec 2003, p61):

Ethical digital imaging requires that the original uncompressed image file be stored on archival media (e.g., CD-R) without any image manipulation or processing operation. All parameters of the production and acquisition of this file, as well as any subsequent processing steps, must be documented and reported to ensure reproducibility.

Generally, acceptable (non-reportable) imaging operations include gamma correction, histogram stretching, and brightness and contrast adjustments. All other operations (such as Unsharp-masking, Gaussian blur, etc.) must be directly identified by the author as part of the experimental methodology. However, for diffraction data or any other image data that is used for subsequent quantification, all imaging operations must be reported.

(Current policy: http://jcb.rupress.org/editorial-policies#data-integrity) 

No specific feature within an image may be enhanced, obscured, moved, removed, or introduced. The grouping of images from different parts of the same gel, or from different gels, fields, or exposures must be made explicit by the arrangement of the figure (i.e., using dividing lines) and in the text of the figure legend. If dividing lines are not included, they will be added by our production department, and this may result in production delays. Adjustments of brightness, contrast, or color balance are acceptable if they are applied to the whole image and as long as they do not obscure, eliminate, or misrepresent any information present in the original, including backgrounds. Without any background information, it is not possible to see exactly how much of the original gel is actually shown. Non-linear adjustments (e.g., changes to gamma settings) must be disclosed in the figure legend. All digital images in manuscripts accepted for publication will be scrutinized by our production department for any indication of improper manipulation. Questions raised by the production department will be referred to the Editors, who will request the original data from the authors for comparison to the prepared figures. If the original data cannot be produced, the acceptance of the manuscript may be revoked. Cases of deliberate misrepresentation of data will result in revocation of acceptance, and will be reported to the corresponding author's home institution or funding agency.

See also NATURE - Image Integrity and Standards

Note: this document (Digital Imaging: Ethics) is an original work of the author (Mr. Cromey). Endorsement by the Microscopy Society of America, The Journal of Cell Biology, or any other persons or institutions cited here should not be implied.

*Note: access to these articles may require a subscription

 

*Note: access to these articles may require a subscription

  1. Seeing the Scientific Image (parts 1,2,3), John Russ, Proceedings Royal Microscopy Society 39(2); 39(3); 39(4) (2004).
  2. Ethics and Digital Imaging, J.M. MacKenzie, M.G. Burke, T. Carvalho & A. Eades. Microscopy Today 14(1):40-41. (2006)
  3. Crusade for Publishing Better Light Micrographs – Light Microscopy publication guidelines, George McNamara
  4. The 39 Steps: A Cautionary Tale of Quantitative 3-D Fluorescence Microscopy, James Pawley, BioTechniques 28(5): 884-887 (2000).
  5. Seeing is believing? A beginners’ guide to practical pitfalls in image acquisition, Allison J. North, JCB 172(1): 9-18. (2006)
  6. Multicolor imaging: the important question of co-localization, Anna Smallcombe, Biotechniques 30, 1240-1242 (2001). free registration required
  7. Scientific and Industrial, Joint Photographic Experts Group
  8. The JCB will let your data shine in RGB, Mike Rossner and Rob O’Donnell, Journal of Cell Biology 164:11-13. (2004)
  9. Digital Image Sampling Frequency, Spring, K.R., Russ, J.C., Parry-Hill, M.J., Fellers, T.J., Zuckerman, L.D. & Davidson, M.W. (2006)