'Voodoo' fMRI?

Functional magnetic resonance imaging has become a seminal technique for researchers looking to map behaviors onto the brain. But a study in press in Perspectives on Psychological Science questions the validity of many fMRI studies and suggests that social neuroscience researchers who use the technique need to design their studies more carefully.

Edward Vul, a cognitive science graduate student at the Massachusetts Institute of Technology, was amazed by the remarkably high correlation scores between behavioral traits and regional brain activation in many studies. For example, a 2005 paper in NeuroImage found that a person's proneness to anxiety attacks correlated very strongly—r = 0.96—to his or her brain activation in the right cuneus when listening to angry speech (Vol. 28, No. 4). By comparison, Vul says, most personality correlations found using the Big Five and MMPI usually have r values between 0.7 and 0.8.

It seemed too good to be true, so he investigated the methodology of 54 studies published between 1998 and 2008. In those studies, researchers used fMRI to measure blood oxygenation—a marker of neuronal activity—in specific brain regions during behavioral tasks. As is typical in fMRI studies, researchers divided up the brain into tiny cube-shaped regions called voxels and looked for activation within regions they believed were key to the behavior.

The problem, Vul says, is that there are fundamental flaws in the way most researchers determine which voxels to include in their analyses. Many only include voxels that reach a certain threshold of activation; if they hit that threshold, it's a correlation. Since they average these data across many individuals, even random "noise" in the data gets amplified into a false correlation—something Vul refers to as "voodoo correlation."

Vul says that a "disturbingly large and quite prominent segment" of fMRI investigations into behavioral neuroscience are guilty of this. Mark Blumberg, PhD, a cognitive psychologist at the University of Iowa in Iowa City and editor of Behavioral Neuroscience, agrees that researchers who rely heavily on fMRI need to be more thorough.

"Most of these papers [mentioned in Vul's studies] could have been criticized before they even got to the statistics," he says.

Even so, he says, the fact that this sloppiness is coming to light is proof that the scientific process eventually weeds out its own errors.

The good news? There are other techniques that allow researchers to identify more accurate, representative voxel regions. Those techniques can even be applied to existing data sets.

—M. Price