PlayInfoBox

Home /Education /Statistical Significance Explained

Statistical Significance Explained

Click through each step to understand the basics.

  1. What Is Statistical Significance?

    A result being statistically significant means it's unlikely that the observed outcome happened purely by chance.

  2. What Is a P-Value?

    The p-value represents the probability of observing a result as extreme as, or more extreme than, the one actually found, assuming there is truly no real difference or effect.

  3. What Is the Significance Level?

    The significance level is the threshold used to decide whether a result should be treated as chance or as a meaningful difference β€” 0.05 (5%) is commonly used.

  4. When the P-Value Is Below the Significance Level

    If the p-value is below the significance level of 0.05, the result is considered unlikely to be due to chance alone and is judged statistically significant.

  5. Statistical Significance Isn't the Same as Practical Importance

    With a very large sample size, even a tiny difference can turn out statistically significant, so a significant result doesn't necessarily mean the difference matters much in real life.

  6. The Drug Trial Example

    When testing whether a new drug is more effective than an existing one, researchers check whether the difference between the two groups is statistically significant to confirm it isn't just due to chance.

  7. Avoiding Misuse of Significance Results

    Rather than drawing a conclusion from the p-value alone, correctly interpreting results also requires looking at the actual size of the difference, the study design, and how representative the sample was.

The Standard for Judging Whether a Result Is Chance or Not

From drug trials to psychology experiments, statistical significance is the concept used to judge whether a research result reflects a genuinely meaningful difference or is simply due to chance. Knowing the concepts of p-value and significance level lets you read research results in the news more critically.

Sample Size Also Affects Judgments of Significance

Statistical significance is closely tied to how large a sample is and how it was drawn, which is worth understanding alongside how sampling itself works.

Frequently Asked Questions

Does a lower p-value mean a more important result?

No β€” a low p-value only means the result is unlikely to be due to chance; it doesn't indicate the size of the difference or its practical importance.

If a result isn't statistically significant, does that mean there's no difference at all?

Not necessarily β€” with a small sample size or insufficient data, a real difference may exist but fail to show up as statistically significant.