Unlocking Research Credibility: A Guide to Preregistration
Preregistration is more than just a trend — it's a crucial practice that separates hypothesis generation from testing, strengthens research integrity, and curbs cognitive biases.
Table Of Contents
Preregistration is transforming the way researchers approach their studies by offering a structured method to enhance transparency and credibility in scientific work. When committed to a research plan before data collection or analysis, scientists can effectively separate hypothesis generation from testing, and thus reduce the risk of cognitive bias.
In this post, we’ll explore the key aspects of preregistration: what it is, why it matters, and how you can integrate it into your research process.
What is Preregistration?
Preregistration is a practice that involves outlining your research plan before you begin your study.
According to the Center for Open Science (COS), “when you preregister your research, you’re simply specifying your research plan in advance of your study and submitting it to a registry”. In other words, it’s like making a promise to yourself — and the world — about what you intend to do before you actually do it.
Why Should You Consider Preregistering Your Research?
Preregistration might sound like an extra step in an already complex research process, but it’s a step that can make a world of difference in the credibility and transparency of your work. Here’s why:
The Benefits of Preregistration
Enhances credibility: According to the COS, preregistration helps separate hypothesis-generating (exploratory) research from hypothesis-testing (confirmatory) research. This distinction is crucial because it prevents the same data from being used to both generate and test a hypothesis.Without this separation, there’s a risk of unintentionally biasing your results, which can undermine the credibility of your study. By laying out your research plan upfront, preregistration keeps your research on track and reduces the temptation to “cherry-pick” data or analyses that support a particular narrative.
Stakes your claim early: Preregistration allows you to formally document your ideas before anyone else. This means you can establish priority for your hypotheses and research design, which gives you a clear timestamp that can be important in fast-moving fields.
Facilitates better research planning: By forcing you to think through your research design, methods, and analysis before collecting data, preregistration encourages more thoughtful and robust study planning. It’s a way of ironing out potential issues ahead of time that leads to more rigorous and reliable research outcomes.
Combats publication bias: An open letter in the Guardian signed by scientists with positions on more than 100 journal editorial boards highlights how the current publishing culture often favors positive, novel, and eye-catching results. This bias can lead to a “file drawer problem,” where negative or inconclusive results are never published. Preregistration counters this by encouraging the publication of all results, regardless of outcome.
Journals like Cortex have introduced “registered reports,” where research is peer-reviewed before data collection and accepted for publication based on the soundness of the research question and methods, not the results. This approach helps ensure that the scientific record is more complete and less biased toward positive findings.
The Drawbacks of Preregistration
However, preregistration isn’t without its critics. Some researchers argue that it might stifle scientific exploration and serendipitous discoveries. The concern, as highlighted in a critique of preregistration, is that it could limit the flexibility needed to explore unexpected findings that arise during research.
Preregistration often assumes that science is purely confirmatory — where you make a prediction and then test it. But in reality, many important scientific discoveries happen by chance or through exploration.
Moreover, there’s a worry that preregistration might force research to conform to standard statistical techniques (like null-hypothesis testing) instead of allowing researchers to choose the best methods for their data. This could potentially lead to a narrowing of scientific inquiry, where only certain types of research questions are asked because they fit better with preregistration requirements.
To mitigate this, the open letter points out that initiatives like registered reports do allow for exploratory analysis. Researchers can still report on unexpected findings, but these are clearly labeled as exploratory to distinguish them from the preregistered, confirmatory analyses. This way, the benefits of preregistration — such as reducing bias — can be realized without completely sacrificing the potential for unexpected discoveries.
Potential Impact on Journals
One of the deeper concerns about preregistration, particularly from the perspective of academic journals, is that it might disrupt the current “prestige” hierarchy in scientific publishing. As mentioned in the open letter, there’s a fear that preregistration could lower a journal’s impact factor because it encourages the publication of all results, not just those that are positive or groundbreaking. However, this shift might ultimately benefit science as a whole by promoting a more honest and comprehensive scientific record.
How Do You Preregister Your Research?
Preregistering your research is a straightforward process that can be done entirely online. While the specific steps can vary depending on the platform or provider you use, the overall process is generally quite similar. Here’s a generic outline of how you can preregister your study:
Develop a comprehensive blueprint: Begin by creating a detailed protocol that outlines your study design, methodologies, and analysis plans. This document will serve as the foundation of your preregistration.
Be thorough in specifying your hypotheses, the methods you’ll use to collect and analyze data, and any other relevant details. The more precise your blueprint, the more robust your preregistration will be.
Submit your protocol: Once your protocol is ready, submit it to a preregistration platform. After submission, you’ll usually go through a review process where you might need to approve or make minor adjustments to your protocol before it is officially registered.
Collect data and conduct analysis: With your protocol registered, you can proceed to collect your data and carry out your study as planned. Stick to the methods and analysis you’ve outlined in your preregistration to ensure that your research remains transparent and credible.
Update your protocol if necessary: During the research process, unexpected issues might arise that require you to adjust your protocol. Most platforms allow you to make these updates, but it’s important to clearly document any changes and explain why they were necessary. This helps maintain the integrity of your preregistration while allowing some flexibility in your research.
Finalize your analysis and report findings: After completing your data collection and analysis, you can finalize your study and report your findings. At this stage, it’s crucial to distinguish between the preregistered, confirmatory analyses and any exploratory analyses you conducted. This transparency ensures that your research is accurately represented and helps others understand the full context of your findings.
To help you navigate the submission process, the Open Science Framework (OSF) Registries offers a wealth of resources and tutorials. Whether you’re in the preparation phase or ready to submit, you can find detailed guides and support here.
Additionally, the OSF Registries provides example preregistrations categorized by discipline and study type to give you a clearer idea of what a strong preregistration looks like within your specific research area. This can be an invaluable resource as you work through your own preregistration process.
When Can You Preregister?
Preregistration isn’t just for new studies. According to the COS, whether you’re at the start of a research project or revisiting existing data, preregistration can be a valuable tool to enhance the rigor and transparency of your work.
It’s about committing to a plan before the outcomes are known, which can be done at various stages of the research process depending on your needs and circumstances:
Right before your next round of data collection: If you’re about to begin collecting new data, this is an ideal time to preregister. By doing so, you can clearly outline your research questions, hypotheses, and methods before any data is gathered.This ensures that your study remains focused and that your analysis will be aligned with your initial intentions, which boosts the credibility of your findings.
After you are asked to collect more data in peer review: Sometimes, during the peer review process, reviewers may suggest that you collect additional data to strengthen your study.In this case, you can preregister your plans for this new round of data collection and analysis. Preregistering at this stage allows you to transparently address reviewers’ concerns while maintaining the integrity of your research.
Before you begin analysis of an existing data set: If you have an existing data set that you haven’t yet analyzed, you can still take advantage of preregistration. Before conducting the analysis, preregister your hypotheses and analysis plan. This prevents potential biases that can arise from knowing the data beforehand, which ensures that your analysis remains objective and your results are more trustworthy.
Where Can You Preregister Your Research?
Whether you opt for a generic repository, a thematic repository, or a journal-based approach, the key is to select the option that best aligns with your research needs and goals.
Generic Repositories
If you’re looking for a flexible and widely recognized platform, generic repositories are a great place to start. These repositories cater to a broad range of disciplines and offer user-friendly tools to help you through the preregistration process.
OSF Registries: The OSF is one of the most popular platforms for preregistration. It guides you step by step through the process by offering plenty of resources to help you understand and complete your preregistration. OSF Registries also allow you to choose different types of registrations depending on your needs.
AsPredicted: AsPredicted is known for its simplicity and ease of use. It asks you nine straightforward questions about your research design and analyses. Once completed, the platform generates a time-stamped, single-page document with a unique URL that can be used for verification purposes. This straightforward approach makes it an excellent choice for researchers who want a quick and efficient way to preregister their studies.
Zenodo: Zenodo is another major repository that serves forall disciplines. It’s an open-access platform where you can archive and register your scientific materials, making them citable, shareable, and discoverable for the long term. Zenodo is particularly useful if you don’t have access to a specific institutional or thematic repository, as it offers a versatile option for all types of research outputs.
Thematic and Institutional Repositories
For researchers working within specific fields or institutions, thematic and institutional repositories offer tailored options for preregistration. These platforms often align more closely with the norms and expectations of particular disciplines.
Preregistration in Psychology: Hosted by the Leibniz-Institut für Psychologie, PreReg is specifically designed for psychology research. It offers a structured process for preregistering studies in the field.
Preregistration in Language Studies: IRIS is a free, searchable repository of datasets, research instruments, materials, and stimuli used for studies in first, second, and foreign languages. Hosted and maintained by the University of York’s digital library, IRIS ensures quality by requiring all materials to have been peer-reviewed or part of a doctoral dissertation or thesis project.
In Journals
Some academic journals offer preregistration options as part of their publication process. This integration can streamline your workflow by combining preregistration with peer review and eventual publication. To illustrate a few:
PLOS Biology: PLOS Biology accepts Preregistered Research Articles, which are peer-reviewed before any data is collected or analyzed. This type of article ensures that the study design and proposed analyses are rigorously evaluated before the research begins, which enhances the transparency and credibility of your work.
PLOS ONE: PLOS ONE offers two options for preregistered research: Study Protocols and Registered Reports. Study Protocols outline your research plan in detail, including the background, objectives, and methodology. Registered Reports, on the other hand, undergo peer review at the study design stage, with publication virtually guaranteed regardless of the study’s outcome. This approach not only supports transparency but also helps combat publication bias.
Further Readings
Preregistration is just one piece of the puzzle when it comes to enhancing the rigor and reliability of scientific research. It’s closely linked to broader efforts in promoting reproducibility and reducing cognitive biases in science. If you’re curious to learn more about these important topics, here are some valuable resources to explore: