Foundation lesson

Entropy and the number of ways

Entropy measures, on a logarithmic scale, how probable a state is: the entropy difference is Boltzmann's constant times the natural logarithm of the probability W. Two independent systems multiply their probabilities and add their entropies, and only a logarithm does both. §5 of the light-quanta paper uses this rule, which it calls Boltzmann's principle.

Written for this edition, not translated from Einstein. Editorial review pending.

Why is entropy the logarithm of a probability?

Four particles move independently in a box. The chance that all four happen to be in the left half at a given moment is ½ × ½ × ½ × ½ = 1/16. For a hundred particles it is (½)¹⁰⁰, about 8 × 10⁻³¹: possible, but never seen. A state is more probable when more of the ways the particles can be arranged produce it.

Entropy measures this probability on a logarithmic scale. Boltzmann's principle, as §5 of the light-quanta paper states it, makes the entropy of a system a function of the probability W of its state, and that function must be a logarithm:

S - S_{0} = k_B\,\ln\left(W\right)

S minus S nought equals k B times the natural logarithm of W.

Why a logarithm: two independent systems have a combined probability W = W₁ × W₂, while their entropies add, S = S₁ + S₂. Only a logarithm turns the product into the sum, and that is the argument §5 gives. The constant $k_B$ is R/N, which is how the paper writes it, and the paper prints the natural logarithm as lg.

Apply it to the box. Squeezing n independent particles from a volume v₀ into a part v of it has probability W = (v/v₀)ⁿ, so the entropy changes by S − S₀ = n $k_B$ ln(v/v₀), a decrease, since v/v₀ is less than 1. §5 derives exactly this, and remarks that it needs no assumption about the law by which the molecules move.

The probability here is a frequency: the fraction of moments at which the state is found. §5 insists on this, and criticizes calculations in which the equally probable cases are simply postulated.

Worked example: Squeezing a gas into half its volume

Take one gram-molecule of gas, n = N = 6.02 × 10²³ particles, and ask for all of them in the left half: v/v₀ = ½.
The probability is (½) to the power 6.02 × 10²³, far too small to write out. Its logarithm is N ln ½, about −4.2 × 10²³.
The entropy change is $k_B$ × N × ln ½ = R ln ½ = 8.314 × (−0.693), about −5.76 joules per kelvin.
Thermodynamics gives the same number for compressing an ideal gas to half its volume at fixed temperature, R ln ½. Counting ways and measuring heat agree.

§6 of the paper finds the same form for dilute radiation, with E/(βν) standing where the gas has Rn/N, and reads it as radiation behaving, in this respect, like independent quanta.

JavaScript is off, so the controls below cannot change anything. The construction shows its first setting, and its closing paragraph, “What it shows, in words”, describes the rest.

Try it: counting arrangements

Cut the box into equal parts and let each particle land in any part with the same chance, whatever the others do. Every arrangement is then as likely as every other, and the probability of a state is the share of arrangements that produce it. Choose how many particles, and how much of the box they must all be found in.

How many particles

All of them in

Of 16 equally likely arrangements, 1 put all 4 in the left half. So W = (1/2)⁴ = 0.0625.

ln W = 4 × ln(1/2) = −2.77, so the entropy changes by S − S₀ = k_B ln W = −3.83 × 10⁻²³ joules per kelvin.

Under each box, the marks say where particle 1, particle 2 and so on are, in that order. The same picture can come up more than once, because the particles are told apart: particle 1 alone outside and particle 2 alone outside are two different arrangements.

How many arrangements give each count in the left half
Particles in the left half	Arrangements
all 4	1
3	4
2	6
1	4
0	1

L L L Lall inside
L L L R
L L R L
L L R R
L R L L
L R L R
L R R L
L R R R
R L L L
R L L R
R L R L
R L R R
R R L L
R R L R
R R R L
R R R R

What it shows, in words

With four particles and the box cut in half there are 16 arrangements, and one of them has all four on the left, so W = 1/16. Each particle adds a factor of ½ to W and adds ln ½, about −0.693, to ln W: the chances multiply and their logarithms add. Two on each side comes up in 6 of the 16 arrangements, more than any other split, which is why particles left to themselves are found spread out. At a hundred particles W is about 8 × 10⁻³¹. For a gram-molecule W cannot be written as a decimal at all, yet the entropy change is an ordinary number, R ln ½, about −5.76 joules per kelvin. Squeezing into nine tenths of the box lowers the entropy less, but for a gram-molecule it is still a state that is possible but never seen.

Where this lesson stops

This lesson stops at entropy as the logarithm of a probability. What the comparison with radiation shows, and what it does not, is §6 of the light-quanta paper.

If you came here from a passage, Back returns you to the exact place you left.

All foundation lessons · Read as Markdown