‘Most Studies On Neuroscience Are Unreliable’

Chocolates can boost brain power. Exercise makes you feel happy. Pomegranate juice will keep your brain healthy. Recent years have seen a flood of such studies. It is being called the Golden Age of neuroscience – the study of how the human brain works. Riding on a combination of imaging technology, computing power and genetics, neuroscientists are dizzy with success. And the money is flowing in. President Obama has announced a $100 million BRAIN Initiative to map every neuron, the European Commission has given a billion euros to build a computer model of all 86 billion neurons in the human brain. 

    
But a study published this week in Nature Reviews Neuroscience has thrown a bucket of cold water on the euphoria. It found that most brain related studies are not reliable and may be exaggerating things. Scientists from the University of Bristol, UK, teamed up with those from Stanford University, the University of Virginia and the University of Oxford to analyze published neuroscience studies and came to a startling conclusion: the average “statistical power” of these studies was just 20%. This means only one in five times will the studies’ claim be valid. Most scientists regard an 80% power as sufficient. 
    
Kate Button, one of the authors from Bristol University told INN that the statistical power of a study is its ability to detect the effect it is looking for. “Power is dependent on both sample size (number of participants) and the size of effect being investigated, with increases in both leading to increased power,” she said. 
    
The other problem that Button found is that of exaggeration of effect. The smaller the sample, the more probable it is that a small individual variation will get highlighted as a major effect. “Imagine that antidepressants actually improve mood by 10% on average, but we select a group of people and do a study and find that, in our select sample, it improves mood by 20% on average. This would be an overestimation of the true effect.” 
    
Putting these two effects together and you will find the euphoria about neuroscience studies flagging. Button and her colleagues used 49 metastudies published in 2011 that had collated the results of 730 studies on neuroscience themes. The scientists analysed 461 brain imaging studies and found that their statistical power was just 8%. They analyzed 41 rat-in-the-maze type of studies which study memory functions and found that their average power was between 18 and 31%.