By Amanda Kay Montoya

Back in high school chemistry, I remember waiting with my bench partner for crystals to form on our stick in the cup of blue solution. Other groups around us jumped with joy when their crystals formed, but my group just waited. When the bell rang, everyone left but me. My teacher came over, picked up an unopened bag on the counter and told me, “Crystals can’t grow if the salt is not in the solution.”

To me, this was how science worked: What you expect to happen is clear and concrete. And if it doesn’t happen, you’ve done something wrong.

If only it were that simple.

It took me many years to realize that science is not just some series of activities where you know what will happen at the end. Instead, science is about discovering and generating new knowledge.

Now, I’m a psychologist studying how scientists do science. How do new methods and tools get adopted? How do changes happen in scientific fields, and what hinders changes in the way we do science?

One practice that has fascinated me for many years is replication research, where a research group tries to redo a previous study. Like with the crystals, getting the same result from different teams doesn’t always happen, and when you’re on the team whose crystals don’t grow, you don’t know if the study didn’t work because the theory is wrong, or whether you forgot to put the salt in the solution.

A May 2025 executive order by President Donald Trump emphasized the “reproducibility crisis” in science. While replicability and reproducibility may sound similar, they’re distinct.

Reproducibility is the ability to use the same data and methods from a study and reproduce the result. In my editorial role at the journal Psychological Science, I conduct computational reproducibility checks where we take the reported data and check that all the results in the paper can be reproduced independently.

But we’re not running the study over again, or collecting new data. While reproducibility is important, research that is incorrect, fallible and sometimes harmful can still be reproducible.

By contrast, replication is when an independent team repeats the same process, including collecting new data, to see if they get the same results. When research replicates, the team can be more confident that the results are not a fluke or an error.

The “replication crisis,” a term coined in psychology in the early 2010s, has spread to many fields, including biology, economics, medicine and computer science. Failures to replicate high-profile studies concern many scientists in these fields.

Why replicate?

Replicability is a core scientific value: Researchers want to be able to find the same result again and again. Many important findings are not published until they are independently replicated.

In research, chance findings can occur. Imagine if one person flipped a coin 10 times and got two heads, then told the world that “coins have a 20% chance of coming up heads.” Even though this is an unlikely outcome – about 4% – it’s possible.

Replications can correct these chance outcomes, as well as scientific errors, to ensure science is self-correcting.

For example, in the search for the Higgs boson, two research centers at CERN, the European Council for Nuclear Research, ATLAS and CMS, independently replicated the detection of a particle with a large unique mass, leading to the 2013 Nobel Prize in physics.

The initial measurements from the two centers actually estimated the mass of the particle as slightly different. So while the two centers didn’t find identical results, the teams evaluated them and determined they were close enough. This variability is a natural part of the scientific process. Just because results are not identical does not mean they are not reliable.

Research centers like CERN have replication built into their process, but this is not feasible for all research. For projects that are relatively low cost, the original team will often replicate their work prior to publication – but doing so does not guarantee that an independent team could get the same results.

When projects are costly, urgent or time-specific, independently replicating them prior to disseminating results is often not feasible. Remember when people across the country were waiting for a COVID-19 vaccine?

The initial Pfizer-BioNTech COVID-19 vaccine took 13 months from the start of the trial to authorization from the Food and Drug Administration. The results of the initial study were so clear and convincing that a replication would have unnecessarily delayed getting the vaccine out to the public and slowing the spread of disease.

Since not every study can be replicated prior to publication, it’s important to conduct replications after studies are published. Replications help scientists understand how well research processes are working, identify errors and self-correct. So what’s the process of conducting a replication?

The replication process

Researchers could independently replicate the work of other teams, like at CERN. And that does happen. But when there are only two studies – the original and the replication – it’s hard to know what to do when they disagree. For that reason, large multigroup teams often conduct replications where they are all replicating the same study.

Alternatively, if the purpose is to estimate the replicability of a body of research – for example, cancer biology – each team might replicate a different study, and the focus is on the percentage of studies that replicate across many studies.

These large-scale replication projects have arisen around the world and include ManyLabs, ManyBabies, Psychological Accelerator and others.

Replicators start by learning as much as possible about how the original study was conducted. They can collect details about the study from reading the published paper, discussing the work with its original authors and consulting online materials.

The replicators want to know how the participants were recruited, how the data was collected and using what tools, and how the data was analyzed.

But sometimes, studies may leave out important details, like the questions participants were asked or the brand of equipment used. Replicators have to make these difficult decisions themselves, which can affect the outcome.

Replicators also often explicitly change details of the study. For example, many replication studies are conducted with larger samples – more participants – than the original study, to ensure the results are reliable.

Registration and publication

Sadly, replication research is hard to publish: Only 3% of papers in psychology, less than 1% in education and 1.2% in marketing are replications.

If the original study replicates, journals may reject the paper because there is no “new insight.” If it doesn’t replicate, journals may reject the paper because they assume the replicators made a mistake – remember the salt crystals.

Because of these issues, replicators often use registration to strengthen their claims. A preregistration is a public document describing the plan for the study. It is time-stamped to before the study is conducted.

This type of document improves transparency by making changes in the plan detectable to reviewers. Registered reports take this a step further, where the research plan is subject to peer review before conducting the study.

If the journal approves the registration, they commit to publishing the results of the study regardless of the results. Registered reports are ideal for replication research because the reviewers don’t know the results when the journal commits to publishing the paper, and whether the study replicates or not won’t affect whether it gets published.

About 58% of registered reports in psychology are replication studies.

Replication research often uses the highest standards of research practice: large samples and registration. While not all replication research is required to use these practices, those that do contribute greatly to our confidence in scientific results.

Replication research is a useful thermometer to understand if scientific processes are working as intended. Active discussion of the replicability crisis, in both scientific and political spaces, suggests to many researchers that there is room for growth. While no field would expect a replication rate of 100%, new processes among scientists aim to improve the rates from those in the past.

Amanda Kay Montoya is an Associate Professor of Psychology, University of California, Los Angeles.

The Conversation