Uncertainty Analysis
by Matt Young
Based on a talk given to
the Laser Measurements Short Course, Boulder, Colorado, August, 2000
No measurement is exact

Measuring stick is not calibrated
precisely correctly

Repeated measurements of the
same quantity yield slightly different values

Meter has only a finite number
of digits

Constants themselves are known
only approximately

Change of some external variable
changes the outcome of the measurement
A measured value is therefore
meaningless without some statement of its accuracy
Heresy

Define true value

Measurements deviate from true
value

either randomly

or systematically

Deviations from the true value
are called errors

Calculate uncertainty
by analyzing errors
Postmodernism: No fact
can be known with absolute certainty
GUM: True value does
not exist; only measurements exist

No errors; only deviations from
mean
I take the approach deprecated
in Annex E.5 of GUM (Guide to the Expression of Uncertainty in
Measurement; see References)
The result is the same as
the GUM's but, uh, intelligible and more nearly consistent with
common parlance
Errors

True value of measurand
= X

N measurements x_{i},
i
= 1, 2, ..., N
Mean value
µ
= (1/N) S_{1}^{N}
(x_{i})
is an estimator of
X
Measurements may or may
not cluster about the true value
(1) The values x_{i}
cluster about the true value X.
(a) The
spread is narrow, so µ is a good approximation to X.
(b) The
spread is wide, so µ is only a fair approximation to X.
(2) The values x_{i}
do not cluster about the mean, so µ is not a very good approximation
to X
Statistical errors
Error e_{i} is the difference
e_{i}
= x_{i}  X
We do not (and cannot) know X
Approximate X by the mean µ
of the experimental values:
e_{i}*
= x_{i}  µ

e_{i}* approximates
e_{i}
and is called an estimator of e_{i}.

Errors that cluster about their mean (not
necessarily the true value!) are called random errors

Sample standard deviation is an
estimator of the average error
s = sqrt[(1/(N1)) S_{1}^{N}
(e_{i}*^{2})]

µ and s will vary from sample
to sample
Gaussian or normal distribution
of errors

Not all errors follow a Gaussian distribution
(cosine error, for example)
Standard deviation of the mean
Peak of the Gaussian curve estimates
the value for which
e_{i} = 0 (not X = 0)

More data > easier to locate peak of
curve

Uncertainty of set of measurements not
the width s of curve but rather
s
= s/sqrt(N)
is the standard deviation of the
mean
as opposed to the sample standard
deviation s
Type A uncertainty
Uncertainty that is measured by statistical
means

s is an estimator
of the uncertainty of the mean of a set of measurements

Decrease s
to
small value by making large number N of measurements

Square root in the denominator diminishing
returns
Mean µ of N measurements
has a 68 % probability of being between µ  s
and µ + s
, or

68 % probability of being in error by
less than

We define s,
not s, as the standard uncertainty
u_{r} = s
due to random errors
Standard uncertainty is always positive

Experimental results are expressed as
mean ± k times standard uncertainty

Usually k = 2 (see below)
Type B uncertainty
Uncertainty that is not measured statistically

Measuring instruments not precisely calibrated

Drift in a meter

Constant offset, as due to background
intensity

Uncertainty imported from manufacturer

Etc., etc., and so forth
I think of Type B errors as belonging
to several subsets
Estimated errors (my terminology)
Example
Measure long distance with steel tape

Estimate the error due to changes in the
length due to thermal expansion

Estimate the range of temperatures

Coefficient of thermal expansion

Uncertainty of coefficient

Expected temperature range is ±DT

Maximum error of length = e_{m}
(calculated)
Rectangular distribution

Assume any value between µ  e_{m}
and µ + e_{m} is equally likely

Standard deviation of a rectangular distribution
=
e_{m}/sqrt(3)

Therefore, define standard uncertainty
u = e_{m}/sqrt(3)
Example
Digital voltmeter, no electronic noise

Least significant digit = 1 mV

e_{m} = 0.5 mV

Standard uncertainty = e_{m}/sqrt(3),

or 0.5 mV/sqrt(3) = 0.3 mV
Note no division by sqrt(N) since repeated
measurements will be identical
Imported errors (my terminology)
Manufacturer specifies an uncertainty
Example

Uncertainty specified as 1 % of full scale

On 1 V scale, DV
= 10 mV

1, 2, or 3 times s?

Assume that the manufacturer means 2s

Take e_{m} = 3s
= 1.5 DV = 15
mV (1.5 times the manufacturer's quoted uncertainty)
More generally,
u = 1.5 u_{f}/sqrt(3)
(f for manufacturer)
Systematic errors (deprecated
term)
Errors that generally have only one
sign

either positive or negative

result in an offset or a bias
Example: thermal expansion in a steel
tape

Could be + or  but at any one time is
constant
Example adapted from micrometry:

Distance between two parallel but rough
walls

Cosine error: measured value high by 1/cosq

No matter the sign of q

Estimate a mean value of q
and
correct for error

Estimate the uncertainty of correction
Example continued

Roughness of walls

Measurement = distance between the high
points

Peaktovalley distance = 1 mm

Assume roughness completely random

Mean position of each wall = 0.5 mm behind
the peaks

Measuring rod contacts the peaks

Measurements too low by 0.5 mm, each wall

Correct bias by adding b
= 0.5 mm, each wall
Uncertainty of correction
Assume maximum error e_{m}of
the correction = onehalf the correction itself

e_{m} = b/2 = 0.25 mm,
each wall

Standard uncertainty u_{g}
= 0.25 mm/sqrt(3) = 0.14 mm, each wall

Due to both walls, sqrt(2) u_{g}

More conservative? Assume e_{m}
= b, not b/2
This approach allows us to specify a result
± a single uncertainty

rather than mean + u_{1}/u_{2}
To sum it up:
Table 1. Standard uncertainties. 
Type of uncertainty 
Distribution of errors 
Standard uncertainty 
Value 
Random or statistical (Type A) 
Gaussian 
Standard deviation of mean 

Estimated (including uncertainty of
systematic error or bias) (Type B) 
Rectangular 
Standard deviation of rectangular
distribution 
e_{m}/sqrt(3) 
Imported (Type B) 
Rectangular 
Manufacturer's specification 
1.5 u_{m}/sqrt(3) 
Systematic (Type B) 
Bias 
 
b 
Combined standard uncertainty
u_{c}
= sqrt(u_{r}^{2} + u_{1}^{2}
+ u_{2}^{2} + u_{3}^{2} +
...)
Express experimental results in form
µ ± 2 u_{c}

U = 2u_{c} =
expanded uncertainty

Factor 2 is called coverage factor

Coverage factor of 2 means

confidence interval is between
µ
 2s and µ +
2s, or

95 % confidence interval
¡Uncertainty analysis is
approximate and subjective!

Subjective estimates of many parameters

Arbitrary assumption of rectangular distribution

Assumption that uncertainties are uncorrelated

Ignoring of highorder terms
Dirty secret:
A statement of uncertainty tells what
we think about our measurement more than it tells about the measurement
itself (thanks to Ron Wittmann)
Appendix *
Older sources add systematic errors
arithmetically. In our notation,
u_{c}
= sqrt(u_{r}^{2}) + u_{1} + u_{2}
+ u_{3} + ...
where u_{i} here represents
a systematic error

The logic was this:

All the systematic errors may well have
the same sign

Add them for a conservative estimate

No longer accepted

No reason to believe that systematic errors
will have same sign if they are not correlated
* A vestigial part of a book for which
no one has yet discovered a use
Copyright 2000 by M. Young. All rights reserved.
References
Anonymous, Guide to the Expression of Uncertainty in Measurement,
International
Organization for Standardization, Geneva, 1993.
Barry N. Taylor and Chris E. Kuyatt, Guidelines for Evaluating and
Expressing the Uncertainty of NIST Measurement Results, Natl. Inst.
Stand. Technol. Tech. Note 1297, Washington, 1994. Available on the Web
at http://physics.nist.gov/Pubs/guidelines/outline.html. This
is sort of a guide to the GUM.
John Taylor An Introduction to Error Analysis: The Study of Uncertainties
in Physical Measurements, University Science Books, Mill Valley, California,
1997. (This excellent book does not teach or conform to the methodology
of the GUM.)