Are all models wrong?
November 6, 2013 35 Comments
George E. P. Box is famous for the quote: “all models are wrong, but some are useful” (Box, 1979). A statement that many modelers swear by, often for the wrong reasons — usually they want preserve their pet models beyond the point of usefulness. It is also a statement that some popular conceptions of science have taken as foundational, an unfortunate choice given that the statement — like most unqualified universal statements — is blatantly false. Even when the statement is properly contextualized, it is often true for trivial reasons. I think a lot of the confusion around Box’s quote comes from the misconception that there is only one type of modeling or that all mathematical modelers aspire to the same ends. However, there are (at least) three different types of mathematical models.
In my experience, most models outside of physics are heuristic models. The models are designed as caricatures of reality, and built to be wrong while emphasizing or communicating some interesting point. Nobody intends these models to be better and better approximations of reality, but a toolbox of ideas. Although sometimes people fall for their favorite heuristic models, and start to talk about them as if they are reflecting reality, I think this is usually just a short lived egomania. As such, pointing out that these models are wrong is an obvious statement: nobody intended them to be not wrong. Usually, when somebody actually calls such a model “wrong” they actually mean “it does not properly highlight the point it intended to” or “the point it is highlighting is not of interest to reality”. As such, if somebody says that your heuristic model is wrong, they usually mean that it’s not useful and Box’s defense is of no help.
On the opposite end of the spectrum are abstractions, these sort of models are rigorous mathematical statements about specific types of structures. These models are right and true of their subjects in any reasonable definition of the words. They are as right or true as the statement that there are infinite number of primes; or that in Euclidean geometry, the tree angles of a triangle sum to two right angles. When somebody says that an abstraction is wrong, they mean one of two things:
- It is mathematically false. For example: if I say that given the first n primes , their product plus one () is prime then that statement is false. It was a statement that was no made carefully enough (N is not divisible by any of , but could be a composition of other primes not on your list).
- Or, the structure you are applying it to does not meet the requirements of the abstraction. For example, in general relativity, space is non-Euclidean, so triangles don’t sum to 180 degrees. More concretely, if you are measuring triangles on the surface of the Earth then at large scales you can have triangles that sum to more than 180 degrees, even if on local scales Euclidean geometry is a good approximation.
In the first case, your model is not actually an abstraction since it is not valid on all structures meeting its prerequisites (but maybe you can convert it to a heuristic). In the second case, your critic is actually saying that your model is not useful, and Box’s defense is again of no help.
This brings us to models in physics and, more generally, to insilications where all of the unknown or system dependent parameters are related to things we can measure, and the model is then used to compute dynamics, and predict the future value of these parameters. Now, if we look at any currently extant model then just based on our history of model improvements, it seems presumptuous to assume that this model is not wrong. As David Deutsch eloquently wrote at the end of the Beginning of Infinity:
[S]cience would be better understood if we called theories “misconceptions” from the outset, instead of only after we have discovered their successors. Thus we could say that Einstein’s Misconception of Gravity was an improvement on Newton’s Misconception, which was an improvement on Kepler’s.
However, the question remains: is it conceivable that there is a perfect infallible model of the universe? Is the world fundamentally mathematical and comprehensible? As long as I can remember, I believed that the world is not inherently mathematical, that mathematics is a reflection of our conscious and there is no reason to expect that it can perfectly capture external reality in any sense of the word. In fact, I have always asserted that assuming the opposite is anthropically arrogant.
I think that many people agree with this sentiment, but some do so for the wrong reason. This wrong reason is usually a reiteration of On Exactitude in Science — Jorge Luis Borges meditation on the map-territory relationship:
In that Empire, the Art of Cartography attained such Perfection that the map of a single Province occupied the entirety of a City, and the map of the Empire, the entirety of a Province. In time, those Unconscionable Maps no longer satisfied, and the Cartographers Guilds struck a Map of the Empire whose size was that of the Empire, and which coincided point for point with it. The following Generations, who were not so fond of the Study of Cartography as their Forebears had been, saw that that vast Map was Useless
Loosely, the argument is that a perfect model of the universe has to incorporate every detail of the universe and thus must necessarily be the universe itself. I don’t think this is a valid argument because the map analogy for insilications is flawed, and there might even be a deep reason for this flaw. In particular, if we concentrate on Tarski’s elementary geometry then our theory is consistent, complete, and decidable and thus not capable of any deep self-reference. If we lived in world that was perfectly captured by such a formal system then the perfect model would in fact be a copy of the universe. But what reason would we have for assuming that if the world was mathematically structured, it would be such a simple structure and not one capable of recursion?
The point of a model (perfect or otherwise) is to provide a set of rules that can then be instantiated, and mathematics definitely allows deeper models than elementary geometry. Most rich structures are capable of self-reference, and encoding their own rules: this is a basic observation on which Godel’s incompleteness theorem, or universality of computation rests upon; it can also serve as a launching point for fun programming concepts like quines. These systems are capable of describing themselves completely without any error or trivial mile-for-mile representation. Hence, we can’t use the map-maker argument to defend our rejection of a perfect model of the universe.
Building on the theme of no unnecessary assumptions about the world, @BlackBrane suggested on Reddit a position I had not considered before (and that prompted this blog post) for entertaining the possibility of a mathematical universe:
[Box’s slogan is] an affirmative statement about Nature that might in fact not be true. Who’s to say that at the end of the day, Nature might not correspond exactly to some mathematical structure? I think the claim is sometimes guilty of exactly what it tries to oppose, namely unjustifiable claims to absolute truth.
Is believing that the universe is inherently not mathematical as presumptuous as believing that it is? I agree that we definitely should not rule it out on ethical grounds before we explore it more completely. Believing that the universe is inherently mathematical does give a lot of hope and courage to theoretical physicists. However, if we assume that it is, what does this really mean for Box’s aphorism?
If the universe is a formal system then we (and all our thoughts and hence theories) are just derivations in that system. To avoid the map-making argument, we have to assume that this system is capable of self-reference. Unfortunately, Godel showed that such systems are also incomplete: there are true statements about such systems that are not derivable within the system. As such, no theory we are capable of building as entities inside this system can resolve certain questions that we can ask within the system about the system. If a model can’t answer certain questions, is it wrong? Is it useful? Alternatively, if we are beings that evolved in this system then is there any reason to believe that this would necessitate the evolution of mental facilities that are capable of comprehending the system? There is no reason to believe that evolution encourages perfect understanding of the environment, or even rational learning or decision making about the environment. In other words, even if the universe was inherently mathematical, Box’s quote could still be true of achievable insilications.
Finally, this segues to a third option on insilications that is embedded in algorithmic philosophy and stems from my belief that mathematics is a reflection of our consciousness. We can suppose that the external world is inherently not mathematical and thus not fully knowable or comprehensible, but that all our theories or thoughts about it must be. As such, these theories can always be formalized and one of them might be the best possible description of the things-in-itself that we are capable of thinking. We would not be able to comprehend any deviation from such a model, even though these deviations would exist (in some sense of the word). Would such a model be wrong? Of course, as with the discussion on completeness before, there is no reason to believe that even if such an ideal model existed that we could arrive at it. There is also no reason, to favor this view over a mathematical universe, except in that it is assuming limitations of us and not properties of the thing-in-itself.