12. Confidence Intervals and P-Values, Explained for Epidemiology
After this lesson you can
- Separate random error from systematic error, and precision from validity
- Read a confidence interval as a range of values compatible with the data and relate its width to study size
- Define the p-value correctly and name the three things it is not
- Fill in the decision-versus-truth table with alpha, beta and power, and read an underpowered null correctly
- Explain why twenty unplanned subgroup tests are expected to give one false alarm
- Judge public health importance from the absolute excess risk rather than the p-value
Video coming soon
Scheduled for release on November 9, 2026.
The first lesson on alternative explanations, asking whether chance alone could produce an association. It compares two invented Oakhaven cohorts with the same risk ratio of 2.0 but different confidence intervals to teach precision, p-values, power and the difference between significance and importance.
Questions and answers
Questions a learner would ask about this lesson. Think through your own answer before you open each one.
- How is random error different from systematic error?
Show answer
Random error comes from sampling and measurement noise, has no preferred direction and shrinks as a study grows. Systematic error, or bias, comes from a flaw in design or measurement, pushes one way and survives any sample size. Precision is freedom from random error and validity is freedom from systematic error, so a large biased study becomes precisely wrong.
- How should a 95 percent confidence interval be read?
Show answer
Read it as the range of values compatible with the data: for the Oakhaven factory cohort, "the data are compatible with risk ratios from 1.5 to 2.6." Values inside are supported and values outside are poorly supported. The 95 percent describes how often the method captures the true value over repeated sampling, so it is wrong to say there is a 95 percent probability that the truth lies inside this one interval.
- Two Oakhaven studies both found a risk ratio of 2.0. Why was only one trusted?
Show answer
The factory cohort of 1,000 workers had a confidence interval of 1.5 to 2.6, while the smaller workshop cohort of 200 had 1.1 to 3.7. Both had the same risks of 25 and 12.5 percent, but the wider interval is compatible with almost no excess risk. Width is precision made visible, and it narrows as the sample grows: four times the people halves the width.
- What is a p-value, and what is it not?
Show answer
A p-value is the probability, computed assuming the null hypothesis is true, of data at least as extreme as those observed. It is not the probability that the null hypothesis is true, not the probability that the result arose by chance, and not a measure of the size of the effect. The 0.05 threshold is a convention, not a law.
- What does "no significant difference" mean when a study has low power?
Show answer
With 20 percent power, a study would miss a real effect of the planned size four times in five, so a null result is close to what the design predicted whether or not the treatment works. The right reading is that the study could not detect a plausible effect, not that the treatment does not work. Look at the confidence interval rather than the verdict.
Practice quiz
- Question 1A risk ratio is 1.4 with a 95 percent confidence interval of 0.9 to 2.1. What is the correct reading?
- Question 2Two studies report a risk ratio of 2.0, one with a confidence interval of 1.5 to 2.6 and the other with 1.1 to 3.7. Which is more precise?
- Question 3A p-value is the probability of:
- Question 4A trial finds "no significant difference" and had 20 percent power. What is the right conclusion?
- Question 5Twenty unplanned subgroups are tested at alpha 0.05, and none has a true effect. How many significant results are expected?
- Question 6A surveillance system detected 50 of 200 true cases, a sensitivity of 25 percent. Using about two standard errors on each side, what is the approximate 95 percent confidence interval?
- Question 7A study is made four times larger with the same design. What happens to the width of its confidence interval?
- Question 8Concluding that there is an association when there is none is:
- Question 9Study one: risk ratio 1.05, confidence interval 1.02 to 1.08, baseline risk 1 per 1,000. Study two: risk ratio 3.0, confidence interval 0.8 to 11, baseline risk 10 percent. What is the absolute excess risk in study two, if its estimate were true?
- Question 10Which change does NOT raise the power of a study?
Saved in your browser only: no account, no server.
Principles of Epidemiology is education in epidemiologic reasoning, not clinical, screening, or policy guidance. Every example is set in the fictional Oakhaven Health District, and every number is invented for teaching.