Bell’s and sycophants’ criticism of von Neumann’s hidden-variable no-go theorem is misguided
Luboš Motl, November 26, 2014
John Preskill wrote a blog post, Bell’s inequality 50 years later, which argues that «without Bell, the broader significance of quantum entanglement would have unfolded quite differently and perhaps not until much later.» Preskill concludes: «We really owe Bell a great debt.»
Your humble correspondent is much less convinced that the 1964 Bell’s paper was either new or pushing the physics research in the right direction. I am closer to Feynman who would say that it is not a theorem that anybody considers of any particular importance in quantum mechanics. Everyone knew that it [the difference between the/any classical local and quantum predictions] would happen, he just demonstrated it mathematically, Feynman says.
It is a nice undergraduate textbook example very explicitly showing the differences between the quantum mechanical predictions and predictions of some simple «local realist» theories a beginner could expect to be relevant. Some people could have still believed that the question whether the probabilistic character of the physical predictions has to be intrinsic would become forever undecidable and that was shown to be wrong.
However, I strongly believe that
- the fathers of quantum mechanics could collectively solve the particular thought experiment and see the incompatibility of the quantum vs local realist predictions; even without that, the amount of evidence they had supporting the need for the new, quantum core of physics has been overwhelming since the mid 1920s
- much of the explicit findings and slogans about entanglement had been known for 29 years, since the 1935 works by Einstein, Podolsky, Rosen; and Schrödinger
- Bell’s results didn’t really help in the creation of the quantum computing «engineering industry» which would only start in 1970 and which has little to do with all the quasi-philosophical debates surrounding entanglement
- most frustratingly, Bell’s correct results were served in a mixed package along with lots of wrong memes, unreasonable expectations, and misleading terminology and the negative price of these «side effects» is arguably larger than the positive price of Bell’s realizations
Concerning the last point, Bell really remained a champion of the fundamentally flawed Bohmian ways to think about – or classically model – quantum mechanics. He was the man who made it popular for the laymen and deluded physicists to think that quantum mechanics was new because it was «non-local» even though the truth is that the quantum laws controlling Nature are local, thanks to relativity – but they are «non-classical» or «non-realist», if you wish. He has proved that «locality» and «realism» couldn’t have been simultaneously true – but he kept on making the fundamental error in deciding which of these two principles is actually right and which of them is wrong.
In some sense, while the anti-quantum movement should have been almost completely killed by Bell’s results, it was actually re-energized because one of its members, the quantum mechanics hater called John Bell, was able to write a moderately famous paper. So a cult of personality of a sort was created around this mediocre physicist. The fact that the paper was just another piece of evidence that the main idea underlying the movement is fallacious wasn’t a problem for these folks.
More generally, Bell was arguably the main person who helped to degenerate much of the physics culture into the popular-book-driven and media-tainted contest between media fads of the current, postmodern type whose success is being decided by millions of readers who don’t have a clue and who routinely interpret physical results exactly in the opposite way than what the results actually imply.
But I want to discuss a particular aspect of the claims that Bell’s result – which disproves a sociologically important class of hidden-variable theories – was truly new: John von Neumann’s 1932 anti-hidden-variable no-go theorem. Bell and his sycophants have argued for half a century that this theorem can’t be counted because it was either completely wrong or didn’t eliminate a damn thing.
John von Neumann’s proof against the hidden-variable theories appeared in Chapter 4 of
John von Neumann: Mathematische Grundlagen der Quantummechaniek,
Springer, Berlin, 1932 (deutsch)
John von Neumann: Mathematical foundations of quantum mechanics, Princeton
University Press, Princeton, 1955 (English).
I will discuss it momentarily but let me begin with the criticisms. In the 1960s, Bell would offer some strong (and repetitive) words:
Yet the von Neumann proof, if you actually come to grips with it, falls apart in your hands! There is nothing to it. It’s not just flawed, its silly! … When you translate [his assumptions] into terms of physical disposition, they’re nonsense. You may quote me on that: The proof of von Neumann is not merely false but foolish!
It was not hard to restore the original italicization of the words – the italicized words are «nothing», «silly», and «foolish». Did the mediocre physicist John Bell have some actual reasons to trash the result by one of the true geniuses of the 20th century?
(The Bell cultists typically worship Bell for this criticism as well – but it turned out that Bell just literally repeated a criticism written by an unknown female physicist Grete Hermann in 1935. While the cultists are mostly SJWs and self-described feminist warriors, when it comes to the worshiping of true flagships of their culture, such as John Bell, the fact that he just plagiarized a woman who wrote the same thing 3 decades earlier has to be forgotten.)
The critics generally agree that von Neumann’s proof is mathematically valid – that his assumptions do imply the conclusions and there is no logical mistake in the derivation. But they have a problem with the proof, anyway. What is it?
If you study these folks’ criticism of von Neumann’s theorem (and they copy more than 90 percent from a text by Bell: that’s true for Elemer E Rosinger 2004, too), about 95 percent of the texts are repeated expletives and non-technical mudslinging. The remaining 5 percent may be summarized by the following three claims:
- von Neumann was implicitly assuming that any two observables \(A,B\) have to commute when he was discussing their sum \(A,B\)
- he claimed that the spectrum of \(A+B\) is the set of the sum of eigenvalues of \(A\) and eigenvalues of \(B\)
- von Neumann’s theorem fails to eliminate the «truly interesting» models of hidden variables such as Bohmian mechanics
You may read von Neumann’s original text to see that none of these criticisms is fair. Alternatively, open the 2010 analysis by Jeffrey Bub of University of Maryland:
It is not my goal to copy the content of the paper but I do believe that what I would say about the criticism is largely equivalent to Bub’s comments.
I will discuss why the criticisms are indefensible momentarily. But before I do so, it may be a good idea to sketch what the theorem actually assumes and says.
In 1932, John von Neumann assumed that for every observable we know in quantum mechanics, there is a function of the «beables» – observables in an intrinsically classical theory with hidden variables – and asked how the expectation value of the observable \(O\) may be calculated from the state \(\psi\). Quantum mechanics tells us its answer to this question,
\[ E_\psi(O) =\bra \psi O \ket\psi. \]
The hidden-variable theory should allow you to calculate the expectation value differently, i.e. to have a map
\[ E: \mathcal{H} \times \mathcal{O} \to \mathbb{R} \]
The expectation value \(E\) has to satisfy three axioms, von Neumann’s assumptions. First, for the «unit observable» (one, a constant), we have to have
\[ \forall\psi\in \mathcal{H}: \quad E(\psi,{\bf 1})=1. \]
That’s sort of trivial. Second, for a projection operator \(P\in O\), we have
\[ \forall\psi\in\mathcal{H}:\quad E(\psi,P)\geq 0. \]
That’s because the expectation value of the projection operator quantifies the probability of the corresponding No/Yes question and the probability can’t be negative. Most nontrivially, we have some linearity in the observables:
\[ \forall \psi\in\mathcal{H},\,\,A,B\in O, \,\,\alpha,\beta\in \mathbb{R}: \qquad E(\psi, \alpha A+\beta B ) = \alpha E(\psi,A)+ \beta E(\psi,B).\]
Fine. Right afterwards, von Neumann straightforwardly proves that
\[ E(\psi,O)={\rm Tr}(U_\psi O) \]
where \(U_\psi\) is a positive operator on \(\mathcal{H}\) with a unit trace. When he applies this partial result on the projection operators \(P_\chi\) projecting on one-dimensional spaces generated by the state \(\chi\), he sees that
\[ E(\psi,P_\chi) \in \{0,1\}. \]
You know, in an intrinsically classical theory, the state \(\psi\) is either the same as \(\chi\) or different, so the expectation value must be equal to a sharp eigenvalue that squares to itself. But none of these two values of the function works because they produce the trace of \(U_\psi\) that is either equal to zero or the dimension of the Hilbert space. But it should be one.
The intrinsically probabilistic character of the wave function is needed for quantum mechanics to avoid this paradox. We can no longer say that \(\chi,\psi\) are mutually exclusive. Instead, the probability that a normalized \(\psi\) is the same state as a normalized \(\chi\) is equal to
\[ P_{\chi=\psi} = | \bra \psi \chi\rangle |^2 \]
which depends on the state continuously, despite the rule \(P^2=P\) that implies discreteness of the eigenvalues! You may look for the original sources if you need a more detailed treatment or analysis of what von Neumann actually did say. But I think that the broad strategy is rather clear.
Why the criticism is idiotic
I’ve mentioned that the criticism is a mixture of three points. The first point, the claim that von Neumann forgot about the uncertainty principle, is completely stupid. The nonzero commutator of \(A,B\) is really the key novelty that makes quantum mechanics differ from any classical theory and John von Neumann has appreciated this fact. From the very beginning, he is very careful about the difference between pairs of observables that are simultaneously measurable and pairs that are not. See pages 297-309 of the English translation.
He says that in general, he can’t write down a precise description of the procedure to measure \(f(A,B)\) if he has procedures to measure \(A\) and \(B\) but if \(A,B\) refuse to commute with one another. However, even if \(A,B\) have a nonzero commutator, one may talk about their sum and the expectation values of \(A+B\) in a given state may be constrained by his linearity rule.
Some of the critics of von Neumann – and perhaps John Bell himself – even claim that the linearity in the operators doesn’t hold in quantum mechanics if \(A,B\) fail to commute. But of course that it does!
\[ \bra \psi (A+B) \ket\psi = \bra\psi A \ket \psi + \bra\psi B \ket \psi. \]
Be sure that this holds in quantum mechanics so the critics are making an elementary mistake! What is true is that \(A+B\) cannot be measured by a «multi-staged» measurement in which we would measure \(A\) and \(B\) after each other, in the same repetition of the experiment (i.e. when the initial state is only set once). But von Neumann has never claimed such a thing. Even though \(A+B\) can’t be measured in this «multi-staged» way, it corresponds to a Hermitian operator and according to quantum mechanics, it may be measured.
And quantum mechanics, much like any hypothetical competing theory, has to predict the odds what may happen if we measure \(A\) or \(B\) or \(A+B\). We may end up measuring something else but the theory must have some ready-to-use predictions for either scenario! In particular, a theory must be able to predict the expectation values of \(A\), of \(B\), and of \(A+B\), simply because we may prepare the system in the same state many times and measure \(A\) (or the other two operators) many times to find the expectation value as the average!
So this whole criticism of von Neumann is indefensible, much like the claim that he would think that the spectrum of \(A+B\) is the set of numbers \(a+b\) where \(a,b\) are eigenvalues of \(A,B\), respectively. Such a mistake could be done by a critic of quantum mechanics but John von Neumann really wasn’t an idiot!
Bell himself would notice that \(\sigma_x\) and \(\sigma_y\) have eigenvalues \(\pm 1\) but \((\sigma_x+\sigma_y)/\sqrt{2}\) which corresponds to \(\sigma\) along the axis «in between» \(x\) and \(y\) has eigenvalues \(\pm 1\) as well – and not eigenvalues \((\pm 1\pm 1)/\sqrt{2}\). John von Neumann, like any competent physicist after 1925 or 1926, realized that perfectly. He would never claim that the eigenvalues of \(A+B\) may be obtained in this wrong way and his theorem has never implied or suggested such a claim.
So far, my point was that all the criticisms linked to the observation that \(A,B\) usually refuse to commute and John von Neumann would contradict this fact in one way or another – all these criticisms have been spectacularly idiotic.
The final part of the criticism was that von Neumann’s theorem failed to disprove the «really interesting hidden-variable theories» such as Bohmian mechanics – which emulate the predictions of quantum mechanics by a theory within the «realist» framework that needs some non-local mechanisms, however.
Indeed, von Neumann’s theorem doesn’t eliminate such theories. But von Neumann has never made such a claim and these theories are not interesting or viable, either. The main reason why such theories are not viable candidate theories in physics is that they violate the theory of relativity. But von Neumann’s proof had nothing to do with relativity – so of course it didn’t discuss the problems of similar theories, and they’re primarily problems with relativity.
But even if you ignore relativity, there’s one purely epistemic reason why von Neumann didn’t consider theories such as the pilot wave theory – a reason why it’s totally OK to assume that we’re not dealing with a theory of this kind. What is the reason? What is the assumption that von Neumann made?
The assumption is that the theory includes a particular set of «beables» and that this set may be mapped, in a one-to-one way, to the observables of quantum mechanics. Bohmian mechanics doesn’t satisfy this assumption because there are some extra variables that might be in principle measurable in Bohmian mechanics but they aren’t measurable according to the state-of-the-art quantum physics. So these Bohmian theories contradict the known list of «things that may be measured» that was extracted from the experiments. That’s a serious problem!
Even if you ignore the state of the experiments, there is still a problem with the «list of beables». Bohmian mechanics needs to assume that there are classical observables that are completely physical because they do influence all other observations; but they cannot be measured themselves. And that’s a problem because these observables are «operationally meaningless» yet «physically consequential» at the same moment. These variables resemble God who may affect others but can’t be seen or affected – and natural physical theories simply don’t allow such one-way «interactions».
Whenever you eliminate this problem, a highly contrived character of the candidate theory that requires some physical observables as well as a waterproof mechanism that keeps them unmeasurable, you’re back to the assumptions of von Neumann’s theorem and the proof of the theorem applies. You may conclude that given the known values of all the expectation values, quantum mechanics can’t be replaced by any realist hidden-variable theory using an isomorphic «list of observables».
If one looks at all these aspects of von Neumann’s proof, it is mostly a consistency check on quantum mechanics and a way to show that certain deviations from the precise quantum mechanical rules would ruin the consistency, indeed. All the «competing theories» are required to work with the sane «list of observables». But this «trivial character» of von Neumann’s proof is largely inevitable because quantum mechanics may be shown to be the only «consistent alternative» to classical physics, indeed.
Bell’s theorem is just a homework exercise analyzing a simple thought (or, later, real) experiment. And the actual reason why the Bell cultists are worshipping it is not what Bell has proved – but what he hasn’t proved, namely all the extra philosophical gibberish that he added into his package and that he defended by his newly earned «political capital» rather than evidence. They worship him because they love to celebrate things like Bohmian mechanics – and so did Bell. But Bohmian mechanics is incompatible with relativity and utterly stupid for many other reasons, especially because it tries to completely revise the «list of observables» and introduce lots of observables (from the mathematical viewpoint) that must be kept unobservable by lots of fine-tuning and conspiracies at the same moment (because we know from the experiments that there’s no way to measure them).
In 1932, John von Neumann didn’t consider these candidate theories – like the 1927 Louis de Broglie pilot wave theory, currently known as Bohmian mechanics – because he considered it a low-brow conspiracy theory and a contrived set of demagogic ideas by which some people want to reject even the most obvious changes of the physics paradigm that were forced upon us by quantum revolution. John von Neumann shouldn’t be criticized just because he wasn’t proving theorems about completely deluded ideas and John Bell shouldn’t be celebrated for promoting them.
