It’s a great struggle to unravel a paradigm of understanding. Paradigms have the feature of quickly organizing data and experience within a referential framework. Paradigms have the bug that when data and experience do not conform to that framework, solutions are hard to come by. Systemic bias is not a denial of reality so much as it is an unwillingness to examine the epistemic framework in which reality is processed and understood. A map or a table, if you prefer to speak philosophically. We (all thinkers, not just scientists) will go to great lengths to avoid a paradigmatic upheaval. So many squares will be pounded into circular holes before unlearning what we previously believed was true.

One example is the classical notion that the universe is compromised of matter and forces. It is such an ingrained popular notion that it takes enormous effort to unlearn this dualism. Coupled with the idea that electrons “orbit” a dense proton/neutron core, one learns anecdotally that atoms are mostly made of empty space. Both ideas are false. If electrons orbit the proton like a planet around a sun they would spiral into the core in less than a split second. If atoms were really mostly empty space then it should be rather easy to squish them together, which it is not.

The old paradigm could not account for this evidence. Subsequently, a new paradigm evolved. But here is the thing. Paradigms are not one-off events. It’s not as if we wipe away the old software and add a bunch of new software. Often, we are struggling to find the contours of such a paradigm with only a dim notion of what it might be but knowing well what it cannot be.

It took awhile before a new paradigm emerged, one in which the universe is field-like in nature. In one sense, the world is more abstract and harder to intuitively grasp. The distinction between force and matter is more of a taste than a hardwired empirical fact. In another sense, the new paradigm is much more satisfying, with several Ah! moments. The way in which the inverse square law of EM and gravity naturally jumps out from fundamental wave-like properties of interacting fields and unconstrained force carriers propagating at the speed of light through space. The way in which particles are excitations within quantum fields and are distinguished by those that can share the same quantum state (bosons) and those that cannot via the Pauli Exclusion Principle (fermions).

What is the truth? Our bias is to affirm a conclusion we thought was true. We are humbled by a conclusion that is shown to be false. Rather, we should be focused on a better result. Not “what is the truth” but rather “why is it false?” This is where our hard work begins, both the beginning and the culmination of our efforts.

A theory does not allow you to pick or choose its predictions you like and refuse the ones you dislike. You can’t embrace general relativity yet reject the probable existence of black holes or the possibility of wormholes simply because it leaves an aesthetic bad taste in your mouth. Accept a theory and all its predictions that follow, then compare predictions to evidence/experiment. Whenever prediction and evidence significantly depart, that’s usually a good sign that the theory needs to be improved or abandoned.

The predictions of QM are straightforward enough. If you accept a probabilistic interpretation of the Schrödinger equation for a wave function evolving through time, then a continuous superposition of quantum (eigen) states will reduce to a single state when observed. The predictions do not allow you to categorically declare the observable state is the one, true, real state while the superposition is somehow unreal, an illusion, or simply a mathematical artifact.

At first blush, it is hard to understand why this result should cause so much consternation. The idea that the (relatively speaking) calm reality of our everyday existence emerges from a sum of fuzzy probabilistic microstates is fascinating, with its own set of questions. Does space itself emerge from this process? Does time? Is there only one way this picture resolves, or are there several copies each with its own separate reality? What happens to all those other worlds? Can we detect them by experiment? What does it mean to observe? What counts as a measurement?

None of this would bother us if the other predictions of QM did not conform with such exact precision to experiment. It’s apparent success leads us to accept all of the predictions. The real question is why does this bother us? Is the prediction anymore bizarre than a black hole or the speed of light being constant? A result can not be judged as nonsense simply because it doesn’t conform to normal everyday observation. Especially since normal everyday observation is inherently a brute cudgel of guesswork.

Science with the aid of powerful mathematical ideas has incessantly led us to shrink the dominion of the kingdom of humankind. From the center of God’s eye, we became a lonely little world of remote significance on the grand stage of the universe. Most scientists accept this fate, but QM for whatever reason seemed a bridge too far for many. You can reduce my domains, but how dare you take away my uniqueness! The violation of a uniform self, the last protected sphere of an infinite conscience, multiplied and stretched out across a (perhaps) infinite Hilbert Space became the straw that broke the camel’s back. Here, we must object! Here we must draw the line and make our final stand against this inerrant assault on humanity’s greatness and fortune.

But this is not science. It is faith. And until we break the arrogance of faith, our science will suffer, has suffered, has stalled on the fear of the power of imagination to build upon the foundation of QM. Is QM the final say? Of course not. But we will go no further so long as we insist on reducing scientific interpretation to mere positivism.