Reading People’s Minds: From fMRI to Multi-voxel Pattern Analysis

Artwork by Thao Do

Mind-reading would be an incredibly useful skill to have in many social situations. Given the size and complexity of brain-imaging tools, they may not be of immediate use in person-to-person interactions. However, if it is possible to predict the content of thought based on brain activity we move closer to understanding the underlying mechanisms of human thought and behaviour.

A technique that in the early ’90s directed us in the right direction was functional magnetic resonance imaging (fMRI). fMRI measures the relative level of oxygenated blood in the brain. Because active neurons require oxygen, fMRI can obtain an image of the brain activity in a region that correlates with certain functions, whether sensory, cognitive or behavioural. From an aesthetic point of view, many have also admired fMRI results demonstrating how brain areas “light up” in beautiful colours in response to different tasks.

The widespread fascination with brain imaging is also pronounced. British singer-songwriter Sivu filmed an entire music video in a brain scanner with a remarkable result. Brain imaging is not only fascinating, however – it also makes for convincing evidence. McCabe and Castel’s study demonstrated the way in which a research publication including brain images will be rated as more coherent than an identical article including bar graphs instead. Undoubtedly, fMRI has advanced our body of knowledge immensely, both within the scientific community and beyond in the general public.

Now, shortly after its twentieth birthday last fall, it may be time to further emphasize the important advancements that have emerged within the field. The level of activity, averaged across a brain region, reflects a simplified theory of the brain. Individual neurons within one region may be activated by different types of information as well. fMRI cancels out such activation patterns when averaging the activity. A region important to the execution of a certain function in that way may be neglected when investigating fMRI data. Even if the overall region does not exhibit selective activity to particular information, activity patterns within and between regions may nevertheless play a crucial role in representing the information in question.

Maintaining sensitivity to such activity patterns is the purpose of multivoxel pattern analysis (MVPA). MVPA uses pattern-classification technique of fMRI data to determine whether particular patterns of activity is associated with particular information. While MVPA is not a new technique, it hasn’t yet received the amount of public attention as some of its predecessors. A shame, given the intriguing results – the patterns, or “neural fingerprints” as they have been nicknamed, can more or less accurately predict the content of a person’s mind.

This phenomenon was recently demonstrated in a study investigating how visual dreams are represented in the brain. At the University of Electro-Communications in Tokyo, participants were put to sleep inside the brain scanner. The researchers recorded their brain waves by electroencephalography (EEG) and woke them up whenever they detected brain waves of early sleep, a stage associated with dreaming. When awake, the researchers asked participants to describe what they had dreamt about, then let them go back to sleep. This procedure was repeated to obtain at least 200 dream reports for each individual. The neural activity patterns measured immediately before waking up was then linked to reports of the visual experience during the dream. This allowed a computer to learn “dream content – activity pattern” associations. The specific content of the dreams was afterwards predicted based on the pattern of brain activity during sleep with 60 % accuracy. Particularly activity in late visual areas such as the fusiform face area, the lateral occipital complex and the parahippocampal place area was predictive of dream content. Given the numerous possible visual experiences of people’s dreams, an accuracy of 60 % is definitely significant, and significantly better than chance.

Beyond dream visualization, confirmation of truth and honesty is also a social situation in which fMRI mind reading would be priceless. Lie detectors have been around for almost a century. What is new, though, is the ability to reliably distinguish lies from truth based on brain imaging. In 2005, Christos Davatzikos et al. demonstrated how pattern classification of neuroimaging data can reveal whether a person is lying or being truthful. Participants were presented with a sequence of playing cards. For each card they had indicate which card out of two they were seeing. In one condition, participants lied about the identity of the card. In another, they had to respond truthfully. A computer was trained with the brain activity patterns under the lying and truth conditions. On subsequent trials, the researchers found that the activity patterns in right prefrontal regions and in the posterior cortex could predict with 87.9 % accuracy whether a participant was lying or telling the truth. What is interesting is what the study reveals about the underlying mechanisms of lying and the real life applications associated with such a finding. Furthermore, it shows the way in which neural activity patterns reveal not only mind content but also distinguish between different cognitive processes.

Decoding people’s dreams, thoughts, and mental processes from patterns of neural activations is undoubtedly cool. But how is mind reading of any scientific relevance? Since science (fortunately) does not really care about what you or I are thinking about at any given moment, the relevance lies elsewhere. Namely in the extent to which the accurate decodings can tell us something about the underlying neural mechanisms of thought and mental processes. fMRI showed us how different regions of the brain are selective to different types of information. This was how the “colour area” (V4) and the “face area” (FFA) were discovered. The MVPA studies described above further widen our horizon by suggesting that information is not only represented by the level of activity in brain regions. Information is also represented in the pattern of maximal or minimal activations across several neurons within or across brain regions.

MVPA confirms a theory of the brain that information is represented (at least partly) in a distributed fashion across neurons, a theory that fMRI, because of its limitations, could never have confirmed nor rejected. That is not saying that the results of fMRI are no longer of significance. Rather, the combination of simple fMRI data and pattern-classification techniques manage to integrate evidence that used to seem contrasting. Each brain region may be of greater importance to the processing of some types of information than other, as evident from fMRI, while the information is represented in a distributed fashion across neurons within or between brain regions, as evident from MVPA.

Different techniques are sensitive to different aspects of the complex activity going on in the brain. Consequently, the fascination with one technique – regardless of which – will lead to a simplified theory of the brain. The combination of techniques is therefore crucial to gain a more complete knowledge of neural functions. It is the job of the scientists to embrace the great public interest in the field to make sure that curiosity brings a further and deeper understanding of the brain.