Bug: Kolmogorov-Smirnov normality test, fixed in GraphPad Prism 4.01 and 4.0b and InStat 3.06 and 3.0b

Prism can test for normality as part of the Column Statistics analysis. InStat shows the results of a normality test as part of the summary stats page.

Both programs use the Kolmogorov-Smirnov (KS) test to determine whether the distribution differs significantly from a Gaussian distribution. The KS test itself assumes that you know the mean and SD of the entire population, not just the sample you measured. So both the Prism and InStat manuals say that they use the Dallal and Wilkinson approximation to Lilliefors' method for performing the normality test. In fact, version of InStat up to 3.05 (Windows) and 3.0a (Mac) and versions of Prism up to 4.00 (Windows) and 4.0a (Mac) did not perform this correction. They performed the KS test without the correction.

We fixed this bug in Prism release 4.01 (Windows), 4.0b (Mac), and InStat 3.06 (Windows) and 3.0b (Mac).

How much difference does it make? The reported value of the KS distance were correct. But with small samples, the bug caused InStat and Prism to report P values for the normality test that were too high. Thus, these versions of InStat and Prism had too little power to detect violations of a normal distribution, at least with small samples.