Peanut Contamination: How Not to Interpret Test Results

The scrutiny of Peanut Corporation of America’s food safety practices is uncovering more problems and forcing closures of more plants. This week a second plant in Texas was forced to shut down.

An infuriating revelation from this fiasco is that Peanut Corporation failed to tell inspectors that samples from its Georgia plant had tested positive for salmonella in 2007 and 2008. The company continued to sell products after having a second set of tests performed by another lab that came back negative.

It makes sense to repeat tests that reveal potentially expensive problems. For example, patients are counseled to seek second opinions before agreeing to expensive and invasive medical procedures. But interpreting secondary tests, particularly ones that conflict with earlier tests, requires care.

Managers of the peanut processing plant are right to conclude that a single positive test doesn’t provide conclusive proof that salmonella is a problem. But, that same reasoning also means that a single negative test doesn’t provide conclusive proof that the product is safe. Tests need to be repeated and more importantly, conflicting results need to be understood. Choosing to believe the test that provides the most convenient result is a recipe for disaster.

Test shopping has led to other expensive high-profile disasters. When the main mirror for the Hubble telescope was manufactured, preliminary tests showed that the mirror did not meet specifications and suffered from an optical flaw known as “spherical aberration.” But much more elaborate and much more expensive tests showed that the mirror had no abnormalities. Managers of the project reasoned that precision and expense must mean results that are more accurate and reliable. Unfortunately this is faulty reasoning. When tests give conflicting results there must be underlying reasons. A test that is designed to be more precise can still be performed improperly.

Rather than seek to understand why the test results conflicted, the mirror was approved for launch. When the first images came back so blurred as to be unusable, astronomers immediately knew the problem—spherical aberration. An expensive optical corrective system had to be designed and a space shuttle launched to install it, before the Hubble could provide usable images.

The lesson is that test results do not make something true. If peanut plant conditions are conducive to the growth of salmonella, the microbes are probably present whether the tests come back positive or negative. Testing cannot be a substitute for actual sanitation.

It’s of course easy for managers to fool themselves with wishful thinking. It’s easy to fool health inspectors and the public with “certified” test results conducted according to “standard” procedures that meet all the legal requirements. But the corporations, government officials, and public, should all be mindful of Richard Feynman’s famous last sentence in a report on an earlier deadly debacle—the space shuttle Challenger disaster. That failure stemmed from the same underlying cause—managers choosing what to believe rather than understand the reasons for the conflicts.

Feynman wrote: “For a successful technology, reality must take precedence over public relations, for nature cannot be fooled.”


Joseph Ganem is a physicist and author of the award-winning The Two Headed Quarter: How to See Through Deceptive Numbers and Save Money on Everything You Buy