T-Test

Analysts are often interested in comparing estimates of population parameter to one another, or to known constants. For example, one might test whether the observed differences in average test scores between two samples of students reflects real differences between the populations, or simple reflect chance sampling error. Evaluating questions like this typically requires an estimate of the sampling distributions of the average test scores. With the sampling distribution in hand, we can ask question of the form: "If these two populations are really the same, in what proportion of samples like this one would we observe a difference at least as large as we observed?"

Many parameter estimates are approximately normally distributed in large samples. We can use this information to construct a confidence interval around the hypothetical value of a population parameter (say, zero for an hypothesis of no difference). The probability of observing the observed difference (or a larger one) if that hypothetical value were true is given by the area under the normal curve.

As an example, suppose we observe that the average difference in test scores for two samples is 1.5, and that our sampling error (standard error) is 1. We want to know whether the true difference is greater than zero, so we ask how likely our sample would be under that condition. We can draw the normal distribution with a mean of zero (our hypothetical value) and a standard deviation of 1 (our estimated standard error). We can then mark the area above 1.5 in red, like so:

The red area marks the probability of observing a between-sample difference of the observed size or greater, if the population means were actually equal. That is 6.68% (.0668) of the are under the curve, implying that we would have about a 6.68 percent chance of observing a difference of at least this size if none really existed.

Often, analysts will not have an expectation of whether the differences will be positive or negative, so they will ask about the probability of observing an absolute difference (positive or negative) of at least that magnitude. This is called a two-tailed test, and the probability is simply twice the probability under a one-tailed test.

In smaller samples the normal approximation can prove less acceptable, and a related distribution, the Student's T distribution is used.

Click on the Overview tab to read an overview of this statistical analysis. Click on the Details tab to learn more about the details of this analysis. Select the References tab to retrieve the references for this analysis. Select the How To tab to learn how to conduct this analysis.