Open Science explained

27 Aug 2018

Open ScienceThe UQ School of Psychology is leading the way in the Open Science movement; but what exactly is Open Science? Dr Eric Vanman explains.

What is Open Science?

Open Science is a fairly new movement in which scientists, particularly in psychology, are trying to be more transparent about everything they do, from the design of the study to conducting the research, through to analysing it and reporting it.

Why has the Open Science movement come about?

It is largely a response to the so-called replicability crisis that occurred in 2015. An important paper was published that year in Science reporting a project in which a large team of researchers attempted to replicate 100 recent psychology studies from top journals. Fewer than half of these yielded the same conclusions as the original study. What’s more, these studies had been conducted just a few years before, so the most obvious explanation was that the original studies had not been of such high quality as everybody assumed that they were. So, one of the movement’s main goals is to improve the quality of all research.

How is research conducted differently in Open Science?

One of the big differences is that open science scientists register their research on a public database, and one of the big ones is at the Open Science Framework. It is funded by a very large philanthropy grant that’s enabling thousands of scientific projects to be registered at the site. Researchers publicly register everything they do: their hypotheses, the plan of how they’re going to run the study, how big the sample is going to be, etc. This last one, sample size, is very important because many times researchers in the past have conducted their studies with only just enough people to make sure they had a significant result, and then they stopped. Or they stopped, and they didn’t have a significant result, so they added a few more people to the sample and kept doing that until they had a significant result. There are all sorts of statistical problems with that approach –basically, it is capitalising on chance. In Open Science, we’re trying to get people to commit beforehand what that sample size might be and what other statistical techniques they’re going to use, so that they can stick to it and not be fishing around with their data afterwards.

Is Open Science making a difference to how journals publish research findings?

Some journals now are requiring pre-registration to be part of the article. Many top journals are making this a requirement, or you have to say why you can’t make it registered. Other journals are just stating things like ‘make sure all your data are open and sharable so that anyone else can come along and check you’. We’ve had this problem, in psychology in particular, that when you go and ask somebody about a dataset from four or five years ago, they can’t find it or they’re just not willing to respond to your email or the person has dropped out of academia, so by putting it on a public database and making it sharable then people can check on your work if they want, and they can do further analyses.

What does Open Science mean for the teaching of psychology?

I teach statistics and methodology in the first year of the psychology degree, so I’ve been teaching my first-year students about all of this. Some of my colleagues in 2nd, 3rd and 4th year are also starting to do it as well. It’s clear that all students who are getting their degree today will have to know about Open Science methods sometime in the next five years. We’re trying to get ahead of it by teaching the new students now, before they’re at the end of their training, and they’re the last ones to find out. We want to be a leader in this.

There’s a group of us, including six or seven academics in the School, who have really been on top of the Open Science movement since the very beginning. We’ve been encouraging our PhD students and undergraduate Honours students to use pre-registration for all their projects. Collectively, as a group, we started encouraging others in the School to get more and more pre-registrations to happen. We now have an active Open Science group in the UQ School of Psychology that promotes this even more. We talk about it at academic committee meetings or workshops and try and get all of our fellow researchers to pre-register. Because of that push, I think we are just ahead of the game now. That’s why so many of our projects are pre-registered and other universities are just starting to catch up.

Can you tell us about the upcoming Australasian Open Science Conference?

I have attended the annual meeting of the Society for the Improvement of Psychological Science (SIPS) meeting for the last two years. This is the largest open science group in psychology. The meetings have been held in the United States, and next year it’s going to be in Europe. Australasian students and academics have faced challenges attending the conference because of these overseas locations. So, the UQ Open Science group, including Associate Professor Jason Tangen, Associate Professor Blake McKimmie, Professor Barbara Masser, and I, decided to host our own meeting for people in this region. We want people in Australia and the Southern Hemisphere to share the same sort of excitement and be part of the process that others who get to travel to Europe or North America do.

The response to our announcement of the meeting has been fantastic. We already have over 100 people planning to attend, who come from five different countries and 14 universities in Australia and New Zealand. For two days they will be going to workshops, hackathons, and presentations about Open Science methods and developments in psychology and related fields, such as pre-registration, using R for analysis, open source experimental programming, promoting diversity, and more.

The Australasian Open Science Conference will take place at UQ’s St Lucia campus, 24-25 September 2018. Visit the website for more details and registration.

Any closing comments about the Open Science movement?

This is something that the younger generation of scientists is embracing so we’re hearing from a lot of postgrads and undergraduates who are really, really excited about it. What’s been a little disappointing is that some senior academics have not been as excited about this process. At the U.S. meetings, the open science group is largely made up of younger people who are quite excited about the future of our field based on these new approaches. In fact, when I go to those meetings I’m usually one of the oldest people there because I think younger people are more willing to embrace change and not be stuck in the status quo. This generational gap may also reflect the fact that a lot of these new methods require some technological savvy. Part of the Open Science movement is people using open tools, where they don’t pay some big corporation exorbitant fees for their glitzy software, and instead use open source software. For instance, instead of using SPSS for statistics, which is proprietary software with a very expensive licence, many Open Science advocates use R, which is an open source language but does require more technical expertise to use it. Again, younger people seem more willing to adopt this new set of tools than people like me—someone who has been using SPSS for 30 years!

Eric VanmanAUTHOR: Dr Eric Vanman is a Senior Lecturer in the UQ School of Psychology. After receiving his Ph.D. in social psychology from the University of Southern California in 1994, he was a post-doctoral fellow in cognitive and behavioral neuroscience at USC and then spent a year as a research scientist at Texas A&M University. He was then a lecturer at Emory University until his appointment as an Assistant Professor at Georgia State University in 2000. He left Georgia State in 2007 as an Associate Professor to take up his current position. His research interests include the social neuroscience of emotion and intergroup prejudice, and his studies have incorporated several kinds of psychophysiological and neuroimaging methods.