Posted by Level of Significance
The level of significance in hypothesis testing refers to the probability of making an error in rejecting the null hypothesis when it is actually true. This is known as Type I error. The significance level is the maximum risk we are willing to take of making this error.
For example, if we choose a significance level of 5% in a hypothesis testing procedure, it means that we are willing to accept a 5% chance of incorrectly rejecting the null hypothesis when it should be accepted. In other words, we are 95% confident that our decision is correct. Therefore, we say that the hypothesis is rejected at a significance level of 0.05, indicating a 0.05 probability of making a wrong decision.
Why Use a Significance Level of 0.05?
The significance level of 0.05 was introduced by the prominent British statistician R. A. Fisher in 1925 in his book “Statistical Methods for Research Workers”(SMRW). Fisher stated, “The value which the standard normal distribution gives for 5% in the tails is 1.96. Thus, setting the significance level at 0.05 makes it convenient to judge deviations exceeding approximately twice the standard error as significant.”
In 1926, Fisher further explained that it is convenient to select a criterion at the level where the probability of random occurrence is 0.05, meaning that such an event would not be expected to occur more than once in 20 trials. This became the foundation for the 0.05 significance level. However, Fisher did not intend for this to be a strict decision-making rule.
Fisher expected analysts to combine background knowledge with data analysis to draw conclusions. However, in the late 1920s, Polish mathematician Jerzy Neyman and British statistician Egon Pearson introduced a new framework for analysis, suggesting that a p-value less than 0.05 is statistically significant. This led to the widespread adoption of the 0.05 significance level in many analyses, becoming a standard convention over time.
Conclusion
The 0.05 significance level has become a standard in hypothesis testing due to its historical introduction by Fisher and the subsequent endorsement by Neyman and Pearson. While it is a widely accepted convention, it is important for analysts to consider the context of their data and combine statistical analysis with domain knowledge to make well-informed decisions.