For over a hundred years, the standard method for testing hypotheses has been the p-value. Scientists use this to determine whether an outcome is statistically significant. Researchers are not allowed to add test subjects if an experiment appears to give promising results. However, they often do this anyway. This can lead to false positives: it seems as if you have found something, for example that a drug works better than a placebo, but that is not the case.
New theory
Part of this problem comes from the math behind the p-value. VU scientist De Heide is one of the pioneers of a new theory for hypothesis testing, the e-value. This new theory is much more flexible: as a researcher, for example, you can look at your results and decide whether to add more subjects to your experiment, but the chance of false positives remains very small. Not only did they come up with the theory for this research, but De Heide also wrote software to test the methods on real data and compare them with existing methods.
Science is going to change
In 2019, the researchers put their first paper about this theory online. Together with other scientists, they initiated a joint movement for better science. Various research groups around the world study and develop this theory. This has already led to the first software packages that everyone can use.
Replace the p-value
Now it's time for the next step: replacing the p-value with this theory, thus making a huge improvement in the way in which essentially all applied science is conducted. According to the researchers, the theory has now been developed far enough to completely change the way science is practiced in the coming decades.
Royal Statistical Society
On January 24, De Heide, Grünwald and Koolen have the honor of presenting their paper at the prestigious Royal Statistical Society (RSS). They will also discuss this with experts who have been specially invited for this occasion. The draft version of the paper can already be viewed on the RSS website.