Hypothesis Testing, Explained Clearly
Hypothesis testing is the foundation of statistical inference and the concept students most often memorize without truly understanding. “Reject the null if p < .05” is easy to repeat; explaining what a p-value actually is is much harder, and it is exactly what exams, committees and reviewers ask.
Since 2014, I have helped students and researchers build a genuine understanding of hypothesis testing, so you can choose the right test, interpret the result correctly and avoid the most common mistakes in statistics.
The Logic of Hypothesis Testing
- State the hypotheses. The null hypothesis (H₀) is typically a statement of no effect or no difference; the alternative (H₁) is what your research predicts.
- Choose a significance level (α), the risk of a false positive you are willing to accept, conventionally .05.
- Select the appropriate test based on your research question, design and type of data.
- Compute the test statistic and p-value.
- Make a decision and interpret it in context, including effect size and confidence interval.
What a p-Value Is, and Is Not
A p-value is the probability of observing data at least as extreme as yours if the null hypothesis were true. It is not:
- the probability that the null hypothesis is true;
- the probability that your result occurred “by chance”;
- a measure of how large or important an effect is.
This is why modern reporting standards, including APA 7, expect effect sizes and confidence intervals alongside p-values.
Errors, Power and Sample Size
| H₀ is true | H₀ is false | |
|---|---|---|
| Reject H₀ | Type I error (α) | Correct decision (power, 1 − β) |
| Fail to reject H₀ | Correct decision | Type II error (β) |
Statistical power, the probability of detecting a real effect, depends on effect size, sample size, α and design. An underpowered study can miss important effects, which is why committees and ethics boards ask for a power analysis. Try the free Sample Size Calculator to see how these pieces interact.
Common Hypothesis Tests I Teach
- z-tests and one-sample t-tests: comparing a mean with a known value
- Independent and paired t-tests: comparing two group means
- ANOVA: comparing three or more means
- Chi-square tests: associations between categorical variables and goodness of fit
- Correlation tests: Pearson and Spearman
- Tests on regression coefficients
- Non-parametric alternatives: Mann-Whitney U, Wilcoxon signed-rank, Kruskal-Wallis, Friedman
- Tests of proportions, one- and two-sample
Unsure which one fits? Use the free Which Test to Use? guide.
Beyond the Basics
- One-tailed vs. two-tailed tests, and when each is justified
- Multiple comparisons and corrections (Bonferroni, Holm, false discovery rate)
- Equivalence testing (TOST) when you want to show that an effect is negligible
- Practical vs. statistical significance
- An introduction to Bayesian alternatives (Bayes factors)
Hypothesis Testing in Your Software
I teach hypothesis testing in SPSS, R, Stata, Python (SciPy, statsmodels) and Excel, or by hand, if that is what your exam requires.
Get Clear, Confident Answers
Whether you are preparing for a statistics exam or formulating hypotheses for your thesis, book a free 15-minute consultation. Tell me your course or project, the test you are working with and your deadline, and you will receive a clear plan and quote within 24 hours.
Prefer email or chat? Write to [email protected] or message me on WhatsApp at +1 (425) 999-3993.