1. Select Power Demonstration from the main menu.
2. Select a sample size between 1 and 20.
Three distributions of sample means are displayed on your screen. Regardless of the sample size that you select, the blue distribution has a mean (mu) of -1, the white distribution has a mean (mu) of 0, and the red distribution has a mean (mu) of +1. The standard deviation of each distribution is 1. The white distribution represents the distribution for the null hypothesis, in this case that mu equals zero; the coloured distributions represent the distributions for two alternative hypotheses, in this case that mu equals +1 or -1. The shaded (coloured) areas of the two distributions for the alternate hypotheses represent the power of the hypothesis test. On the right hand of your screen is a value for standard error, alpha, and the critical value of the sample mean, for a test of one mean ( a z test).
In the Alpha Demonstration, we discussed the possibility of rejecting the null hypothesis when it is true (Type I error). A second error (Type II error) can occur when the null hypothesis is not rejected and it is, in fact, false. The probability of making a Type II error is called beta, and is closely related to the power of a statistical test. The power of a statistical test refers to the probability that we will reject the null hypothesis when it is false. In this program, the effect of sample size on the power of a test of one mean is demonstrated.
Power increases with Sample Size
There are several factors that effect the power of a hypothesis test: the standard deviation of the null and the alternate (which we assume to be equal), the distance between the means of the two distributions (commonly called the effect size), whether we are conducting a one- or two- tailed test, alpha, and sample size. In this demonstration, standard deviation is held constant at 1, alpha is held constant at .05 (one-tailed), and the effect size is held constant at an absolute value of 1. In this program, you can manipulate and observe the effect of sample size.
In this demonstration, our null hypothesis is that mu = 0 (white distribution). Our alternative hypothesis for a one-tailed test is EITHER that mu =1 (red distribution), OR that mu = -1 (blue distribution). (Remember that if we tested both of these hypotheses simultaneously, we would be conducting a two-tailed test. Furthermore, in practice we would not know whether there was an alternate distribution - that is the purpose for conducting the hypothesis test - nor would we know the value of its mean .)
Try Power demonstration a number of times, and increase the sample size each time. You will see that as you increase sample size, standard error of the distribution of sample means decreases (pay close attention to the values along the x axis); the distributions become more narrow (if this point is unclear, consult the Central Limit Demonstration). This necessarily has an effect on the critical value of the sample mean - it becomes smaller as sample size increases. For example, for our null distribution with a mu of 0 and standard deviation of 1: if you choose a sample size of 9, the critical value of the sample mean is 0 + ( 1.64 * 0.333 ) = 0.55. If you choose a sample size of 20, the critical value of the sample mean is 0 + ( 1.64 * .224 ) = 0.37. As standard error and the critical value decrease with increasing sample size, the more likely the mean is to fall within the alternate distribution, i.e., there is an increase in the statistical power of the hypothesis test.