Notice

Rejecting or Failing to Reject the Null Hypothesis

If the p-value is less than the significance level, reject the null hypothesis. For example, if α = 0.05 and the p-value is 0.03, reject the null hypothesis because we expect to see the observed outcome only 3% of the time if the null hypothesis is true. So the observed outcome isn’t very likely. More specifically the probability of the observed outcome happening was less than 5% if the null hypothesis is true. So reject the null hypothesis in favor of the alternative hypothesis and say, “there is sufficient evidence to reject the null hypothesis”. To summarize with non technical language, if something is not very likely, reject it.
If the p-value is greater than the significance level, fail to reject the null hypothesis. For example, if α = 0.05 and the p-value is 0.15, fail to reject the null hypothesis. The observed outcome is expected 15% of the time if the null hypothesis is true. This may not seem very likely, but it is more likely than 5% so the conclusion is to fail to reject the null hypothesis, and we say, “there is not sufficient evidence to reject the null hypothesis.”
Why shouldn’t the conclusion be, “there is sufficient evidence to accept the null hypothesis”? It is a convention based on the fact that in mathematics, statements are not proved with examples. A claim can be disproven with one example, but even one million examples in favor of the claim can’t prove it. To borrow a common phrase, “Absence of evidence is not evidence of absence”. However, hypothesis tests don’t actually prove anything anyways. They are just a method of judging the evidence for or against a hypothesis. Yet, the tradition is strong enough, that a conclusion should never be, “there is sufficient evidence to accept the null hypothesis”.

Analogy

Until the 17th century Europeans thought every swan was white because for centuries, every swan they saw was white. Then, a black swan was discovered in Australia, instantly disproving the hypothesis that all swans are white.

Analogy

Suppose a person thinks that there might have been a skunk in his yard the previous night. A null hypothesis is that there was no skunk in the yard (status quo). The alternative hypothesis would then be that there was a skunk in the yard.
H0: There was no skunk in the yard.
HA: There was a skunk in the yard.
He could go outside the next day and look for evidence that there was a skunk. If he finds skunk fur or smells a skunk, then he would have evidence to reject the null hypothesis in favor of the alternative hypothesis (that there was a skunk).
On the other hand, if he doesn’t find evidence that a skunk was there, that does not mean that the null hypothesis is true. A skunk could have been there without leaving evidence. That is why he shouldn’t say he accepts the null hypothesis. He doesn’t know for sure that there wasn’t a skunk. He just doesn’t have evidence to support the claim that there was a skunk. So he says there is not sufficient evidence to reject the null hypothesis or that he fails to reject the null hypothesis.
If he rejects the null hypothesis, he could technically say that he accepts the alternative hypothesis. However, tradition dictates that conclusions are always stated as rejecting or failing to reject hypothesis rather than accepting hypothesis.
If he finds skunk fur in the yard, he would reject the null hypothesis. Yet he still hasn’t proved that there was a skunk. The dog could have brought the fur into the yard. Because he hasn’t proved the alternative hypothesis, (he might have strong evidence that there was a skunk, but he hasn’t proven it) he shouldn’t say that he accepts the alternative hypothesis.