Statistical significance in court

In March 2011, the U.S. Supreme Court took up the question of how much emphasis courts should place on statistical significance in a legal context. The case, Matrixx Initiatives, Inc. v. Siracusano, presented the question in the somewhat unlikely context of a securities fraud class action lawsuit. The plaintiffs alleged that Matrixx, a pharmaceutical company, had failed to disclose reports of a possible link between Zicam Cold Remedy, its leading product, and anosmia, or loss of smell. Matrixx argued that because the number of reported instances of anosmia “was not statistically significant,” it was not obligated to disclose these reports to investors.

According to the plaintiffs’ complaint, Matrixx first became aware of potential problems with Zicam in 1999, when a physician reported to Matrixx that a “cluster” of patients had developed anosmia after using Zicam. In 2002, another researcher discussed similar complaints with Matrixx’s vice president for research and development. In 2003, Bruce Jafek, MD, a researcher at the University of Colorado, presented a poster at the American Rhinologic Society describing 11 patients who had lost their senses of smell after using Zicam.

Shortly after Jafek’s presentation, two lawsuits were filed against Matrixx by Zicam users who claimed that they had suffered anosmia after using the product. Matrixx did not disclose these suits to investors. That same month, however, Matrixx told investors that Zicam was “poised for growth in the upcoming cough and cold season.” Over the next several months, an additional seven plaintiffs filed suit. Notwithstanding these lawsuits, Matrixx increased its public profit estimates for Zicam.

On Jan. 30, 2004, Dow Jones Newswire reported that the Food and Drug Administration was looking into claims that Zicam caused some users to develop anosmia. Following this report, Matrixx’s stock price fell approximately 11 percent. Matrixx issued a press release calling the claims against Zicam “completely unfounded and misleading” — and its share price rebounded. However, on Feb. 6, 2004, “Good Morning America” presented a story featuring Jafek’s findings, and the stock sank again.

The plaintiffs — Matrixx investors — claimed that Matrixx had failed to disclose “material information” to investors in an attempt to keep the company’s stock price high and that its statements about predicted revenues and product safety were misleading. Matrixx filed a motion to dismiss, arguing that even if all of the plaintiff ’s allegations were true, the lack of “a statistically significant number of adverse events” among Zicam users meant that the patients’ complaints would not have been “material” to a “reasonable investor.”

The court rejected Matrixx’s argument that statistical significance should serve as a bright line rule. Although the court did not carefully distinguish between statistical significance and causal inference (Kaye, 2011), the court rightly acknowledged that the question before it was what would have been material to a “reasonable investor.” Justice Sonia Sotomayor, writing for a unanimous court, found that the reports about a possible link between Zicam and anosmia presented an important threat to the “commercial viability of Matrixx’s leading product” and held that this information was likely to have changed the “total mix” of data available in the eyes of a reasonable person considering investing in the company.

It is important for social scientists to recognize the role that statistical significance plays in the law. The court noted that whether a statistically significant association had been detected was relevant in assessing the degree to which the adverse event information would be important to investors, but found that statistical significance could not be legally determinative in this context. Other considerations are inevitably at play. Indeed, it might be that even a statistically significant association does not have practical significance for a particular legal question. In this case, whether or not a statistically significant association between Zicam and anosmia could have been ascertained and whether or not Zicam and anosmia could be causally linked, the adverse event reports and pending lawsuits would likely have been material to investment decisions.

Judicial Notebook is a project of APA's Div. 9 (Society for the Psychological Study of Social Issues).