Fundamentals of Statistics contains material of various lectures and courses of H. Lohninger on statistics, data analysis and here for more.

Paired Experiments

When we try to compare methods, treatments, etc. by applying each to the same population, the resulting values are no longer independent. Some examples may contribute to a clarification: (1) comparison of the temperature measurement of a device to the calibrated temperature (both measurements origin from the same "real" temperature; (2) comparison of the yield of an orchard in two consecutive years (normally the trees stay the same); (3) the influence of alcohol on the reaction time; each client has to perform a rection test both in sober state and after drinking a 1/4 l of wine (the persons stay the same).

Think e.g. of comparing analytical methods applied to environmental samples. The variation among the different samples will probably be larger than the difference between the individual methods. Due to the large pooled variance sp2, we cannot use the independent sample t-test to distinguish between the two methods. We can calculate the pairwise differences, di, and consider the di as a new variable that follows a t-distribution. The mean and standard deviation of the di are , and sd, respectively. nD is the number of pairs. Depending on the sample size nD we use the one-sample tests based on t scores.

Assumption: the distribution of the differences is normal(1) and the values are measured at least at the interval level.

The paired difference experiment is often more powerful, since it can eliminate differences in the samples that increase the total variance, σ2. When the comparison is made between groups (of similar experimental units), it is called blocking. The paired difference experiment is a simple example of a randomized block experiment.

When the normality assumption is not fulfilled, one can use the non-parametric Wilcoxon sign rank test for paired difference designs.

(1) A common error, which happens to be made especially by beginners in the field, is that the two samples are tested for normality, and not the differences of the samples.