Discarding Data
From CheLabWiki
…from CheLabWiki, an online resource for chemical-engineering laboratories located at www.chelabwiki.org; Site Revision #420; 6 January 2009.
Community Portal → Data Analysis → Discarding Data
When we repeat measurements of the same quantity, we sometimes obtain one value that differs noticeably from the others. We have an outlier and the question is, what should we do about such a value? Under what conditions might we discard that measurement?
On the one hand, an outlier might result from some blunder in performing the measurement: we misread a meter, or an instrument drifted out of calibration, or a thunderstorm caused a spurious voltage fluctuation. On the other hand, an outlier might signal the presence of some unexpected phenomena. A famous example is Lord Rayleigh’s dogged preoccupation with small discrepancies in his measured masses of volumes of air, which led him to discover the element argon.[1]
Chauvenet’s Criterion
| N | t | N | t |
|---|---|---|---|
| 3 | 1.38 | 16 | 2.16 |
| 4 | 1.54 | 18 | 2.20 |
| 5 | 1.65 | 20 | 2.24 |
| 6 | 1.74 | 25 | 2.33 |
| 8 | 1.86 | 30 | 2.39 |
| 10 | 1.96 | 40 | 2.50 |
| 12 | 2.04 | 50 | 2.58 |
| 14 | 2.10 | 100 | 2.81 |
| Run # | Flow Rate | Run # | Flow Rate |
|---|---|---|---|
| 1 | 5.5 | 6 | 5.6 |
| 2 | 5.85 | 7 | 5.75 |
| 3 | 5.55 | 8 | 5.65 |
| 4 | 5.8 | 9 | 5.4 |
| 5 | 5.9 | 10 | 5.7 |
The issue is what to do when you cannot decide whether an outlier is either a blunder or a real phenomenon. One way to distinguish between these two possibilities is to apply Chauvenet’s criterion:[2] if the outlier lies more than t standard deviations from the mean of N measurements, then it is probably not a natural fluctuation, so you may consider discarding that value. The value of t depends on N; minimum values of t for various N are given in Table 1.
Note that we have only suggested that you consider discarding the measurement; it would be better to keep the value, but repeat the measurements in an attempt to explain it or to reduce its impact on the mean and standard deviation. In practice, however, constraints of time and resources often prevent additional experiments.
To illustrate, we consider the ten measured flow rates from Table 2. The mean of those ten values is 5.67 gpm, with a possible outlier at 5.4 gpm. To apply Chauvenet’s test, we compute t,
where y is the value of the possible outlier, ym is the mean of the N measurements, and s is the standard deviation of any one value from the mean. For our example, we find
Chauvenet’s criterion is a purely statistical test. More often decisions about discarding or keeping data are made within the theoretical context that guides the experiment. The theoretical context may identify some systematic error that explains an outlier and allows us to remove it from further analysis. We emphasize that considered judgments to discard data do not necessarily signal poor experimental procedures; rather, such judgments are a legitimate part of any experimental process.[3]
Reporting Discarded Data
But no matter what we decide, we must report all measured data and then discuss what steps were taken to explain possible outliers.[4] If any data are discarded, we must still report those values and explain the criteria used to justify eliminating particular points from subsequent analyses. In this way we maintain a complete document of what was done; so, if more information becomes available at some later date, we might be able to explain or reconcile outliers.
References
- ↑ Lord Rayleigh, Proceedings of the Royal Society, 55, 340 (1894); reprinted in Scientific Papers by Lord Rayleigh, Dover, New York, vol. IV, 104 (1964).
- ↑ J. R. Taylor, An Introduction to Error Analysis, University Science Books, Mill Valley, CA, 1982.
- ↑ P. J. Galison, How Experiments End, University of Chicago Press, Chicago, 1987.
- ↑ J. M. Haile, Analysis of Data, Macatea Productions, Central, SC, 2003. ISBN 0-9728602-0-7.
