In Agenda, the “what’s on in Cambridgeshire” magazine for June 2010, there is an article about Improbable Games, an event created by David Spiegelhalter, the Winton Professor of the Public Understanding of Risk in the University of Cambridge. What a pity I missed it by more than a year! (It was David who programmed the “monkeys and typewriters” for the BBC Horizon programme To Infinity and Beyond last year.)
I read once the opinion that, if the human lifespan were a million years, then nobody would ever cross the road; it would be far too dangerous! A nice way of saying that even if probabilities are in some sense absolute, our understanding of them is relative.
Anyway, to the topic. Why is the world unpredictable? What is randomness, and where does it come from?
I think that, in all but rather extreme cases, randomness is simply ignorance. You toss a coin; the only reason I cannot guess reliably whether it will come down heads or tails is the fact that I lack detailed information about its starting position and the momentum and angular momentum imparted to it by the tossing. The work of Persi Diaconis, Susan Holmes and Richard Montgomery on coin tossing is well worth reading. When I was a student, one of my lecturers could toss a coin so that the axis perpendicular to the plane of the coin precessed about the vertical, but without very close inspection it was difficult to see that the coin was not spinning conventionally. A useful trick!
Indeed, teaching probability to first-year students led me to the view that “all probability is conditional”: the probability I assign to an event depends on what I already know (and even events for which what I know is merely probabilities). When I learn more, I update my probabilities according to Bayes’ Theorem. (Well, I probably don’t actually do that – humans are not very good at thinking about probability – but I know that that is what I should be doing.)
But two developments in twentieth-century science suggest that there is some randomness in the universe which is not just our ignorance. Laplace’s view that an omniscient being who knew the positions and momenta of everything in the universe at one moment could predict the state of the universe in the indefinite future has been challenged, first by quantum mechanics, and then by chaos theory.
Quantum mechanics is a completely deterministic theory: the state of something (for example, the universe) is described by a wave function, which evolves in time according to Schrödinger’s equation. So how does randomness come in? If we apply the principles of quantum mechanics to a system which we observe from the outside, then a mysterious “collapse” of the wave function occurs to an eigenfunction of the linear operator associated with the measurement we make (and the value we measure is the corresponding eigenvalue). There is no way to predict in advance which eigenfunction the system will “choose”; we can only assign probabilities to them. Einstein famously thought that this was tantamount to “God playing dice”, and rejected it on these and other grounds.
If we could apply the principles of quantum mechanics to the entire universe, then this source of randomness would disappear, since observation from outside would not be possible. This is seen as a possible way to reconcile quantum theory with relativity. But it has not been carried through successfully yet.
An intermediate stage, where a quantum effect has a macroscopic consequence, was proposed by Schrödinger, in the famous “Schrödinger’s cat” thought experiment. If the quantum system remains in a superposition of states until observed, and if it has macroscopic effects, then we are faced with a difficult philosophical problem: what exactly constitutes an observation? Can the cat be an observer of its own fate?
I don’t know if the experiment has ever been performed. But instances in the real world where quantum effects have macroscopic consequences are very difficult to imagine. (Roger Penrose has proposed that this happens in microtubules in our brains, as a possible way in which free will could be introduced into the universe; but his proposal is not generally accepted.)
In my view, chaos theory poses an even more serious philosophical dilemma. As far as I know, all models of chaos theory which could be observed in the world depend on assuming that the world is modelled by real numbers; and a single real number is an infinite object, in the sense that we need to know infinitely many bits of information to specify it uniquely. Laplace’s omniscient being needs to have an infinite amount of information to know even one coordinate of the position of one object. We assume that the world is governed by deterministic differential equations; if we can solve them analytically, then all is well and good, but if we are forced to resort to numerical calculation, then rounding errors will necessarily occur, and chaos is the principle that such errors (no matter how small) can sometimes grow exponentially and completely swamp the accuracy of the calculation after a finite time.
It is claimed, for example by Ian Stewart in his book Does God play dice, that this makes the universe fundamentally unpredictable in a theoretical, rather than just practical, sense.
However, the principal difficulty I see is not whether, with bigger and bigger computers and more and more detailed observations, we can get more and more accurate predictions of the weather. It is whether the universe, at bottom, is continuous or discrete. If there is a lower limit on distances in the universe (not merely on our ability to measure them), then measurement to that limit can not lead to chaotic results, since there will be no smaller errors to propogate.
Some modern theories of the universe, such as loop quantum gravity and Rafael Sorkin’s causal sets, propose that the universe is discrete, and only appears continuous to us because its natural scale is far too small for us to perceive. If this is so, then we can return to the idea that randomness is ignorance rather than any in-built physical limitation of the universe. Perhaps Einstein was right about this.
Remarkably, there is some evidence. Loop quantum gravity (along with some other theories such as noncommutative geometry) predicts that electromagnetic radiation does not all travel at the same speed: when we observe an extremely distant event, high frequencies will reach us before low frequencies do. This phenomenon has now been observed.