Probiotic Trial Reader¶
Run the Probiotic Trial Reader MicroSim Fullscreen
About This MicroSim¶
Supplement ads love to quote "a clinical study." But one study can be misleading, especially a small one. This MicroSim imagines a made-up probiotic with a small true effect (an effect size of 0.2, shown by the dashed green line). Each time you run a trial, the sim adds random chance, the way real trials do, and plots the result the trial would report as one dot.
You control three parts of the trial design:
- People per group: small trials scatter widely; big trials land close to the truth.
- Outcomes measured: each trial reports its best-looking outcome, the way an ad might. Measuring more outcomes makes chance "wins" more likely, which is why good trials name one primary outcome in advance.
- Placebo group: without a comparison group, the improvement people show just from taking something (the placebo response) gets counted as if the probiotic caused it.
Orange dots are results that would look "impressive" (statistically significant) in a headline.
How to Use¶
- Look at the 20 example trials (30 people per group, 1 outcome, placebo group on). Notice how widely the dots scatter around the true effect.
- Press Run 20 a few more times. Watch the share of "impressive" results and the average reported effect.
- Press Clear, set People per group to 300, and run 20 trials. Predict first: will the dots spread out or bunch up?
- Press Clear, set people per group back to 30, and set Outcomes measured to 10. Run 20 trials and compare the average reported effect with the true effect.
- Press Clear, uncheck Placebo group, and run 20 trials. What happens to the reported effect?
- Decide which settings look most like a typical supplement study, and which look like a trial you would trust.
Learning Objective¶
Bloom level: Evaluate. The learner will judge how trial size, the number of outcomes measured, and the presence of a placebo group change how much confidence a reported probiotic result deserves.
Iframe Embed Code¶
<iframe src="https://aaryavvatts-lab.github.io/gut-brain-textbook/sims/ch16-trial-reader/main.html" height="562px" width="100%" scrolling="no"></iframe>
Lesson Plan¶
Grade level: 9 to 12 (biology, health science, statistics, or psychology)
Duration: 20 to 30 minutes
Activity:
- Warm-up (3 minutes). Show a real supplement ad that says "clinically proven." Ask: "What would you want to know about that study?"
- Explore size (8 minutes). In pairs, students run 20 trials at 20, 60, and 300 people per group and record the share of "impressive" results and the average reported effect.
- Explore outcomes and placebo (8 minutes). Students repeat at 30 people per group with 1, 5, and 10 outcomes, then with the placebo group off.
- Discuss (6 minutes). Why does reporting the best of many outcomes inflate results? Connect this to the Rahmannia 2024 review in Chapter 16, where one of four depression scales showed a benefit.
- Limits (3 minutes). Read the "What this model leaves out" box in the chapter and name one way real trials differ from the sim.
Assessment questions:
- Why do small trials sometimes report large effects even when the true effect is small?
- How does naming a primary outcome in advance protect against chance findings?
- What does a placebo group add to a trial, and what goes wrong without one?
- A supplement ad cites one trial of 20 people that measured 8 outcomes. How confident should you be in its "best" result, and why?