Uncertainty Analysis

by Matt Young

Based on a talk given to the Laser Measurements Short Course, Boulder, Colorado, August, 2000

No measurement is exact

• Measuring stick is not calibrated precisely correctly
• Repeated measurements of the same quantity yield slightly different values
• Meter has only a finite number of digits
• Constants themselves are known only approximately
• Change of some external variable changes the outcome of the measurement

•
A measured value is therefore meaningless without some statement of its accuracy

Heresy

• Define true value

•
• Measurements deviate from true value
• either randomly
• or systematically
• Deviations from the true value are called errors

•
• Calculate uncertainty by analyzing errors

•
Postmodernism: No fact can be known with absolute certainty

GUM: True value does not exist; only measurements exist

• No errors; only deviations from mean

•
I take the approach deprecated in Annex E.5 of GUM (Guide to the Expression of Uncertainty in Measurement; see References)

The result is the same as the GUM's but, uh, intelligible and more nearly consistent with common parlance

Errors

• True value of measurand = X
• N measurements xi, i = 1, 2, ..., N

•
Mean value

µ = (1/N) S1N (xi)

is an estimator of X

Measurements may or may not cluster about the true value

(1) The values xi cluster about the true value X.

(a) The spread is narrow, so µ is a good approximation to X.

(b) The spread is wide, so µ is only a fair approximation to X.

(2) The values xi do not cluster about the mean, so µ is not a very good approximation to X

Statistical errors

Error ei is the difference

ei = xi - X

We do not (and cannot) know X

Approximate X by the mean µ of the experimental values:

ei* = xi - µ

• ei* approximates ei and is called an estimator of ei.

•
• Errors that cluster about their mean (not necessarily the true value!) are called random errors
• Sample standard deviation is an estimator of the average error
s = sqrt[(1/(N-1)) S1N (ei*2)]
• µ and s will vary from sample to sample
Gaussian or normal distribution of errors

• Not all errors follow a Gaussian distribution (cosine error, for example)
Standard deviation of the mean

Peak of the Gaussian curve estimates the value for which ei = 0 (not X = 0)

• More data --> easier to locate peak of curve

•
• Uncertainty of set of measurements not the width s of curve but rather

•
s = s/sqrt(N)

is the standard deviation of the mean
as opposed to the sample standard deviation s

Type A uncertainty

Uncertainty that is measured by statistical means

• s is an estimator of the uncertainty of the mean of a set of measurements
• Decrease s to small value by making large number N of measurements
• Square root in the denominator diminishing returns

•
Mean µ of N measurements has a 68 % probability of being between µ  - s and µ + s , or
• 68 % probability of being in error by less than
• We define s, not s, as the standard uncertainty
ur = s

due to random errors

Standard uncertainty is always positive

• Experimental results are expressed as
mean ± k times standard uncertainty
• Usually k = 2 (see below)
Type B uncertainty

Uncertainty that is not measured statistically

• Measuring instruments not precisely calibrated
• Drift in a meter
• Constant offset, as due to background intensity
• Uncertainty imported from manufacturer
• Etc., etc., and so forth

•
I think of Type B errors as belonging to several subsets
• None defined in the GUM
Estimated errors (my terminology)

Example

Measure long distance with steel tape

• Estimate the error due to changes in the length due to thermal expansion
• Estimate the range of temperatures
• Coefficient of thermal expansion
• Uncertainty of coefficient

•
• Expected temperature range is ±DT
• Maximum error of length = em (calculated)
Rectangular distribution
• Assume any value between µ - em and µ + em is equally likely
• Standard deviation of a rectangular distribution = em/sqrt(3)
• Therefore, define standard uncertainty
u = em/sqrt(3)

Example

Digital voltmeter, no electronic noise

• Least significant digit = 1 mV
• em = 0.5 mV
• Standard uncertainty = em/sqrt(3),
• or 0.5 mV/sqrt(3) = 0.3 mV

•
Note no division by sqrt(N) since repeated measurements will be identical

Imported errors (my terminology)

Manufacturer specifies an uncertainty

Example

• Uncertainty specified as 1 % of full scale
• On 1 V scale, DV = 10 mV
• 1, 2, or 3 times s?
• Assume that the manufacturer means 2s
• Take em = 3s = 1.5 DV = 15 mV (1.5 times the manufacturer's quoted uncertainty)
More generally,

u = 1.5 uf/sqrt(3)

(f for manufacturer)

Systematic errors (deprecated term)

Errors that generally have only one sign

• either positive or negative
• result in an offset or a bias

•
Example: thermal expansion in a steel tape
• Could be + or - but at any one time is constant

•
• Distance between two parallel but rough walls

•
• Cosine error: measured value high by 1/cosq
• No matter the sign of q
• Estimate a mean value of q and correct for error
• Estimate the uncertainty of correction
Example continued

• Roughness of walls
• Measurement = distance between the high points
• Peak-to-valley distance = 1 mm
• Assume roughness completely random
• Mean position of each wall = 0.5 mm behind the peaks
• Measuring rod contacts the peaks
• Measurements too low by 0.5 mm, each wall
• Correct bias by adding b = 0.5 mm, each wall
Uncertainty of correction

Assume maximum error emof the correction = one-half the correction itself

• em = b/2 = 0.25 mm, each wall
• Standard uncertainty ug = 0.25 mm/sqrt(3) = 0.14 mm, each wall
• Due to both walls, sqrt(2) ug
• More conservative? Assume em = b, not b/2

•
This approach allows us to specify a result ± a single uncertainty
• rather than mean + u1/-u2
To sum it up:

 Table 1. Standard uncertainties. Type of uncertainty Distribution of errors Standard uncertainty Value Random or statistical (Type A) Gaussian Standard deviation of mean Estimated (including uncertainty of systematic error or bias) (Type B) Rectangular Standard deviation of rectangular distribution em/sqrt(3) Imported (Type B) Rectangular Manufacturer's specification 1.5 um/sqrt(3) Systematic (Type B) Bias -- b

Combined standard uncertainty

uc = sqrt(ur2 + u12 + u22 + u32 + ...)

Express experimental results in form

µ ± 2 uc

• U = 2uc = expanded uncertainty
• Factor 2 is called coverage factor

•
• Coverage factor of 2 means
• confidence interval is between µ - 2s and µ + 2s, or
• 95 % confidence interval

¡Uncertainty analysis is approximate and subjective!

• Subjective estimates of many parameters
• Arbitrary assumption of rectangular distribution
• Assumption that uncertainties are uncorrelated
• Ignoring of high-order terms

•
Dirty secret:

A statement of uncertainty tells what we think about our measurement more than it tells about the measurement itself (thanks to Ron Wittmann)

Appendix *

Older sources add systematic errors arithmetically. In our notation,

uc = sqrt(ur2) + u1 + u2 + u3 + ...

where ui here represents a systematic error

• All the systematic errors may well have the same sign
• Add them for a conservative estimate

•
• No longer accepted
• No reason to believe that systematic errors will have same sign if they are not correlated

•
* A vestigial part of a book for which no one has yet discovered a use

References

Anonymous, Guide to the Expression of Uncertainty in Measurement, International Organization for Standardization, Geneva, 1993.

Barry N. Taylor and Chris E. Kuyatt, Guidelines for Evaluating and Expressing the Uncertainty of NIST Measurement Results, Natl. Inst. Stand. Technol. Tech. Note 1297, Washington, 1994. Available on the Web at http://physics.nist.gov/Pubs/guidelines/outline.html.  This is sort of a guide to the GUM.

John Taylor An Introduction to Error Analysis: The Study of Uncertainties in Physical Measurements, University Science Books, Mill Valley, California, 1997. (This excellent book does not teach or conform to the methodology of the GUM.)