Uncertainty, evidence and inference

What can a set of measurements tell you, and what can it not?

Some questions the data cannot answer, however carefully you measure. If all you know of a rectangle is that its area is 12 square centimetres, its sides could be 3 and 4, 2 and 6, or 1 and 12. The area fixes their product, not each side; to get one side you must measure the other.

The Brownian paper meets the same limit. In its §3 the diffusion coefficient is D = RT/(N · 6πkP), with k the liquid's viscosity and P the particle's radius. So watching particles spread gives D, and with the temperature and viscosity known, D fixes the product of N and P, not each of them: a smaller particle and a larger N give the same spreading. To get N, the radius has to be measured some other way.

An estimate comes from a finite sample, and a different sample gives a different number. Two questions follow: whether the method lands on the right value on average, which is its bias, and how widely its answers scatter from sample to sample. Fitting a drift from the same displacements uses up part of the information and leaves less to estimate the scatter.

Uncertainty comes in four kinds, and each needs its own statement. Sampling uncertainty is the scatter from one sample to the next. Measurement uncertainty is the error in each reading, such as where the microscope places a particle. Model uncertainty is the chance that the formula itself does not fit these particles. Numerical uncertainty is the error of the computation. A confidence interval is not an interval-arithmetic enclosure, which is guaranteed to contain the exact result of a calculation, and a numerical error bar, which bounds the arithmetic, is not a measurement uncertainty.

A 95 per cent confidence interval comes from a method that, used on many samples, catches the true value in 95 of every 100 of them, provided its assumptions hold. It is not a 95 per cent chance that this one interval contains the value. A plot of many repeated intervals, with the misses left in, shows what the promise means.

For a diffusion coefficient the promise has an exact form. With M independent displacements in d coordinates there are q = dM squared displacements, and their sum divided by 2Dt has a known spread from sample to sample: the chi-square distribution with q degrees of freedom. Its 2.5 and 97.5 per cent points give the 95 per cent interval. In a synthetic example with 50 displacements in two coordinates, so q = 100, and an estimate of 0.43 μm²/s, those points are 74.2219 and 129.5612:

Inverting an estimate adds a bias of its own. The molecular number is found by dividing by the estimated D, and although that estimate is right on average, its reciprocal is not: with q = 100 degrees of freedom the average of the inverted estimate is too large by the factor q/(q − 2) = 1.020408, about two per cent. The interval itself survives inversion, with its ends swapped, as the worked example shows.

Reading positions adds noise of its own. If each position is read with an error of spread σ in each coordinate, each measured step gains 2σ² in its mean square per coordinate, and two neighbouring steps share one reading, so their errors have a covariance of −σ². Neighbouring steps are then no longer independent, and cutting one path into overlapping windows does not make independent samples.

An interval that treats the radius, viscosity and temperature as exact holds only on that condition. Allowing for their uncertainty needs to know how each was measured and how often its own interval misses. Without that, a plus-or-minus has no stated coverage and is not a confidence statement.

A path simulated from a chosen N is useful for checking that a method recovers the N it was given. It is not evidence about N in nature. Since 2019 the SI fixes Avogadro's number by definition, so recovering it from modern data checks consistency. Measuring N, as Perrin did from 1908, needs inputs measured independently of it.

Worked example: Inverting an interval, and watching its endpoints swap

1. Take a made-up relation, N = 12/D, with D measured as 4. The estimate is N = 12/4 = 3.
2. Suppose the method gives D between 3 and 6. A larger D means a smaller N, so N runs from 12/6 = 2 to 12/3 = 4: the ends swap over.
3. Now suppose N = 12/(aD), with a an unknown scale. The same D fits many pairs of a and N, and no interval for N follows until a is measured separately.
4. Declaring a = 2 halves both the estimate and its interval without changing a single measurement. It changes an assumption, not the data.
5. If three inputs each come from a method that misses at most once in 100 uses, and D's method at most twice, the combination misses at most 5 times in 100, even if the misses are related. Without the stated miss rates, this arithmetic guarantees nothing.