What existing drugs work against corona? In the pandemic we saw science was in a hurry. To be able to combine global research, PhD student Judith ter Schure of CWI's Machine Learning group developed a new kind of statistics: ALL-IN meta-analysis. This makes it possible to determine more quickly whether, for example, a vaccine works. Judith ter Schure will defend her PhD thesis on 7 April.
At the beginning of the pandemic, fifteen research groups worldwide investigated whether a tuberculosis vaccine could also be used against corona. The vaccine is known not only to help against tuberculosis, but also ensures that people become much less ill from other infectious diseases.
For their studies, the scientists gave some of their patients the tuberculosis vaccine and some a placebo. Then they had to wait for the subjects to get infected with corona. If each separate research group would have to wait until it had enough data to draw a conclusion, it could take a long time. Ter Schure's statistics makes it possible to combine the data from all the research available so far, allowing to draw conclusions about the effectiveness of the vaccine more quickly. In this way, results from different research groups can immediately be placed in context with other studies. In classical statistics this is often done afterwards.
Gambling
Science is always a gamble. And just like in the casino, a series of chance results can fool you, causing you to make the wrong decision. Ter Schure's statistics help to prevent an accidental wrong conclusion. There is a crazy statistical phenomenon which means that if you ask the same question often, there is a good chance that it will be answered wrong sometimes. 'Most people who play roulette don't win much. But when a lot of people play roulette, there's always one lucky person who wins a lot of money. This does not mean we should conclude that this person has shown that there is money to be won always.'
To prevent a wrong conclusion being drawn by pure chance too often a certain minimum result is required when using statistical methods. Consider, for example, the amount of corona infections in subjects who received a placebo as opposed to the number of corona infections in subjects given the vaccine. The method indicates when a certain threshold value has been reached in order to draw a valid conclusion. After some time, coincidence is really too unlikely. The statistical method is thus similar to how professional poker players work. Ter Schure: 'They do get to earn their salary in the casino. To recognize a professional, you observe their results over many poker tournaments. They have a long-term strategy. That's what science needs, too.'
Software
A software package and tutorials are online allowing others to use ALL-IN meta-analysis also. With these tools, they can aggregate results from studies, and see how far away the finish line is. That can make the research more dynamically and efficiently, and make collaboration central focus. Any research group working on the same question can upload data, for example, and track the impact on the results so far tracked on a dashboard. The method can now be used for clinical trials similar to the tuberculosis vaccine study, and many other simple analyses (t-test, 2x2 tables, logrank test). For studies that are more complex, Ter Schure and her colleagues also want to develop software.
Culture shift
The results of the study into the tuberculosis vaccine against corona will be published soon. Ter Schure likes the fact that the researchers focused on the research line rather than on their own research. She thinks it would be nice if this were to happen more often in the future, but that would require a whole different culture in academia. It would mean quite a cultural shift in the academic world. 'At least we have an example now that it can be done. Who knows ALL-IN will help with science where everyone is less concerned with their own paper and more and more with the big picture.'
More information
- Source: Dagmar Aarts, Universiteit Leiden.
- Article in Dutch
- Judith ter Schure will defend her PhD thesis on 7 April 2022 from 15-16 hrs at Universiteit Leiden. Her supervisors are prof.dr. P.D. Grünwald (CWI & Leiden University) and dr. Daniël Lakens (TU/e). A livestream will be available.
- More information about Judith's PhD defense can be found here
Header image: Shutterstock.
