Power
Recall that Type I error is the probability of falsely rejecting the null hypothesis (H0) based on our sample even though H0 is true in the population. Alternatively, Type II error is the probability of not rejecting H0 based on our sample even though there is a true difference in the population.
So What is Power?
Power is the chance that we will conclude from our sample that there is a difference (reject H0) at the level of significance that we have specified (if a difference truly exists in the population). In other words, power is the chance that we won't make a Type II error (1–β).
Four factors contribute to power:
- Sample size: The larger the sample size, the smaller the standard error around the sample mean, and the more power the study has to detect a given difference (imagine the sampling distributions above becoming “skinnier” as the sample size increases).
- The difference you wish to detect: If the expected difference between groups is small, the study will have less power to detect that difference compared to a larger expected difference between the groups (imagine the two distributions sliding further apart as the expected difference increases).
- Standard deviation: The smaller the standard deviation, the smaller the standard error around the sample mean, and greater the power (similar to a larger sample size, a smaller standard will make the sampling distribution “skinnier”).
- Predetermined alpha (Type I error): As we become more “strict” with our chance of committing Type I error (say, setting our α at 0.01 instead of 0.05) then our power will decrease proportionally (a smaller α will make the yellow area larger and the red area smaller).
* BONUS *
If we are designing a study and we want to know how many subjects we need to enroll in order to detect a predetermined difference between the groups, we can use this equation:
where n = sample size, α = predetermined Type I error,
β = predetermined desired power, and μ1 – μ2 is the difference between the groups that you wish to detect.