Non-orthogonal quantum states are not mutually exclusive

Luboš Motl, July 21, 2014

«Interpreters» of quantum mechanics deny nothing less than Born’s rule

In classical physics, if we know that the system has generalized coordinates \((x_i,p_i)\), i.e. that it sits at the corresponding point of the phase space, then we may say that it certainly doesn’t have generalized coordinates \((x’_i,p’_i)\) if the collections of numbers differ,

\[ (x_i,p_i)\neq (x’_i,p’_i). \] Different points of the phase space are mutually exclusive even if they are very close to each other. This lesson holds in any classical theory, including classical field theory. If two configurations of a classical field differ as functions

\[ \Phi(x,y,z)\neq \Phi'(x,y,z), \] then we may say with certainty that if the system is found in the configuration \(\Phi(x,y,z)\), then it certainly isn’t found in the configuration \(\Phi'(x,y,z)\), not even if the latter is close to the former (but not equal).

The people who are incapable of understanding that the quantum revolution has overthrown the general framework of classical physics almost universally assume that the state vector \(\ket\psi\) in quantum mechanics is a form of a classical variable. They’re wrong and their being wrong has very dramatic consequences.

Assume that you know that a physical system is found in the pure state \(\ket\psi\) («psi»), e.g. because it is the eigenstate of a complete set of observables and you have just measured the values of all these observables that uniquely determine \(\ket\psi\), up to an overall normalization factor.

To put it more simply, imagine that you have just measured the value of the observable

\[ \hat O = \ket\psi \bra\psi \] and you got \(\hat O = 1\). This \(\hat O\) is a Hermitian linear projection operator and \(\ket\psi\) is its only eigenstate with the eigenvalue \(+1\) while all states orthogonal to \(\ket\psi\) have the eigenvalue equal to \(0\).

Now, let’s ask the question: Is this physical system found in the state \(\ket\chi\) («chi») which is a different state,

\[ \ket\chi \neq \alpha \ket\psi, \] which is however not orthogonal to \(\ket\psi\)? Our arguments – and the difference between classical physics and quantum mechanics – become extreme if \(\ket\chi\) is very close to \(\ket\psi\) but it is still different. Let’s assume that both states are normalized,

\[ \langle\psi\ket\psi = \langle\chi\ket\chi = 1. \] Now, the people who haven’t started to think quantum mechanically will imagine that the state vector is a form of a «classical field» in some space, so if we know that this «classical field» has one value, we know for sure that it doesn’t have another value. In other words, the physical system in the state \(\ket\psi\) has the (conditional) probability equal to zero

\[ P(\ket\chi | \ket\psi) = 0 \] to be in another state \(\ket\chi\). That’s how all the anti-quantum zealots, critics, interpreters, Bohmian crackpots, many-world crackpots, GRW crackpots, and members of several other major groups of crackpots will answer the question.

However, the answer provided by quantum mechanics is very different. It is known as Born’s rule. If the physical system is in the state \(\ket\psi\), the probability that it is in the state \(\ket\chi\) is equal to

\[ P(\ket\chi | \ket\psi) = {| \langle \chi\ket\psi |}^2 \] In particular, if the states \(\ket\psi\) and \(\ket\chi\) are «very similar» or «nearly collinear» but still different, the probability that the physical system in the state \(\ket\psi\) is in the state \(\ket\chi\) approaches 100 percent. Recall once again that the answer by the anti-quantum «interpreters» is 0 percent. The difference couldn’t be more striking.

As the title has already told you, two non-orthogonal states in quantum mechanics are simply not mutually exclusive. Only orthogonal states are mutually exclusive; the orthogonality is the same thing as the mutual exclusiveness, it is the translation of the mutual exclusiveness to a more precise language of mathematics.

The anti-quantum cranks are not getting this simple point. Because the point is really nothing else than Born’s rule and because Born’s rule is one of the most important general rules in all of quantum mechanics, you must agree that those people are really not understanding the very basics of quantum mechanics.

It’s important that \(\ket\psi\) may emulate – with some nonzero probability – a (non-orthogonal) \(\ket\chi\) in all respects that are in principle measurable. So an anti-quantum zealot could say that the measurements may react to \(\ket\psi\) in similar ways that they would react to \(\ket\chi\), but in principle, one may still say that it is «strictly untrue» for the state to be in \(\ket\chi\) if it is known to be in \(\ket\psi\). They may say that only \(\ket\psi\) is the right description.

But that’s not the case. If a system is known to be in the state \(\ket\psi\), the different but non-orthogonal state \(\ket\chi\) also has a nonzero probability to be the «perfectly right description» of the physical system.

The thought experiment involving Wigner’s friend makes this point (and other points) clear. In that experiment, Eugene Wigner confines a killing machine containing a hammer, Schrödinger’s cat, and his friend to an isolated box. The lethal device measures whether a radioactive nucleus has decayed, and when it sees the decay, it launches the hammer mechanism and kills the cat. Wigner’s friend is sitting in the box and observes whether the cat is alive or dead.

(The box contains enough oxygen for all laws of the European Union about the protection of animals to be obeyed. Moreover, the friend is ethnic Russian so if he gets killed by the hammer as well, the current fascist establishment in the U.S. and the EU won’t have a moral problem with the experiment.)

After some time, the nucleus evolves in some superposition of «not yet decayed» and «decayed» states,

\[ \ket\psi_{\rm nucleus} = 0.6\ket{\rm notdecayed} + 0.8i \ket{\rm decayed}. \] Because the hammer system detects the nucleus and guarantees a correlation (or entanglement) between the cat and the nucleus, the state of the nucleus+cat system evolves to

\[ {
\ket\psi_{\rm nucleus+cat} = 0.6\ket{\rm notdecayed}\ket{\rm alive} + 0.8i \ket{\rm decayed}\ket{\rm dead}.} \] where the products of the ket vectors are really \(\otimes\) tensor products. Similarly, to be sure that quantum mechanics applies to humans as well, the nucleus+cat+friend combined system evolves to

\[ {
\ket\psi_{\rm nucleus+cat+friend} = 0.6\ket{\rm notdecayed}\ket{\rm alive}\ket{\rm happy} + 0.8i \ket{\rm decayed}\ket{\rm dead}\ket{\rm sad}.} \] where «happy» and «sad» refer to the state of Wigner’s friend if he hasn’t seen or has seen a dead cat, respectively. Note that this superposition is the most accurate description of the nucleus+cat+friend system according to Eugene Wigner himself. He is sitting outside the box and knows that quantum mechanics applies to all elementary particles – including those that his friend is composed of.

So Eugene Wigner himself uses this «macroscopic superposition» vector in the Hilbert space to describe the state of the box at some moment – the box with the nucleus, cat, and Wigner’s friend. We know what the right state \(\ket\psi_{\rm nucleus+cat+friend}\) is. Again, we may ask the following question: Is it right to describe the physical system by a different state, e.g. by the state below?

\[ \ket\psi_{\rm nucleus+cat+friend} = \ket{\rm notdecayed}\ket{\rm alive}\ket{\rm happy} \] We have erased the coefficient \(0.6\) in order to keep the state normalized.

The anti-quantum zealots would answer «No». If the previous «macroscopic superposition» is the right state to describe the system, another state just can’t be right. Except that from Wigner’s friend’s viewpoint, things are different. By that time, he has made an observation and he has either seen a dead cat (the probability is 64 percent) or he has seen a cat that was still alive (the probability is 36 percent).

The latter possibility – the cat is still alive (because the nucleus hasn’t decayed yet) – is a perfectly conceivable option. If this option is right, Wigner’s friend obviously uses the simple state vector including just one term because he has made a measurement and has eliminated the possibility that the cat has already died. So from the friend’s viewpoint, the «purely alive» state vector is the most accurate element of the Hilbert space to describe the physical system.

The «interpreters» think that there is a paradox in quantum mechanics because the wave function at that moment should either objectively be «collapsed» to the single («alive») term, or it should still be in the «superposition» state. They don’t like either option. If the «right state» is already collapsed, then quantum mechanics has to be supplemented with some non-unitary extra mechanisms that «collapse» the state vector if there are conscious observers, and – they believe – quantum mechanics therefore has to define the soul and becomes unscientific.

Or «the objectively right state» is the superposition which is also bad because it contradicts the fact that people observe «sharp» results – they see that the cat is either alive or dead.

However, quantum mechanics solves this situation completely differently – none of the options advocated by the «interpreters» is right. Quantum mechanics says that the state vector summarizes the observer’s knowledge about the system. Eugene Wigner uses the superposition; his friend who has already made the measurement uses the single-term «collapsed» or «purely alive» wave function.

They use different states and the point of Born’s rule is that there is no contradiction because these two different states are not mutually exclusive.

By showing that the right wave functions used by the two men at the given moment are different, we haven’t shown any paradox in quantum mechanics because the two wave functions don’t exclude one another. Their inner product is nonzero – they are non-orthogonal – which means that if we know that the system is found in one of these pure vectors, it may very well be in the other vector, too.

Only if we could prove that a physical system is found in two different and mutually orthogonal states \(\ket{\psi_1}\) and \(\ket{\psi_2}\), there would be a contradiction! A «contradiction in practice» would also emerge if the two states were nearly orthogonal – if the calculated probability were tiny enough so that the option may be «ruled out in practice».

Again, I want to emphasize that the explanation of the non-exclusiveness above isn’t open to any «interpretations», «arts», or other forms of bullšiting. It is nothing else than Born’s rule. There is no way to express Born’s rule that would make two nearby but different state vectors mutually exclusive. The very point of Born’s rule is that they are not mutually exclusive and the rule actually quantifies the probability that a physical system in one state is in another state.

One more point. In the thought experiment with Wigner’s friend, there seems to be an asymmetry between the two men. Wigner seems to be more free, superior, and (by avoiding «collapses»), he is performing a more accurate calculation that takes the (very small) possibility of future interference effects between the «alive» and «dead» parts of the wave function into account.

One may look at the situation in this asymmetric way. However, one may also see the situation of the two men in a symmetric way – only up to a moment, however. Let’s return to the experiment and recall that Eugene Wigner describes the state of the box using the state

\[ {
\ket\psi_{\rm nucleus+cat+friend} = 0.6\ket{\rm notdecayed}\ket{\rm alive}\ket{\rm happy} + 0.8i \ket{\rm decayed}\ket{\rm dead}\ket{\rm sad}.} \] If he has very fine gadgets capable of seeing some relative phases in large systems, he may in principle measure the observable

\[ \hat O_{\rm Wigner}=\ket\psi_{\rm nucleus+cat+friend} \bra\psi_{\rm nucleus+cat+friend} \] which is, once again, a linear Hermitian projection operator. Its eigenvalues are zero or one. If he gets «one», then he is sure that the wave function (after the measurement) is indeed the «macroscopic superperposition»

\[ \ket\psi_{\rm nucleus+cat+friend} \] written above (up to an irrelevant complex normalization factor). Now, his friend in the box has already seen that the cat was still alive – he has made the measurement – but he may still ask what would be the result of the measurement of

\[ \hat O_{\rm Wigner} \] performed by Eugene Wigner outside the box. Wigner’s friend does the calculation and he finds out that the probability is nonzero for Wigner to obtain \(\hat O_{\rm Wigner}=1\). In fact, the probability is determined by Born’s rule i.e. by the inner product so it is equal to 36 percent.

You may say that this result is wrong because Wigner’s own «superior» perspective seems to imply that the probability «should be» 100 percent to get \(\hat O_{\rm Wigner}=1\).

However, it’s important to realize that there is no contradiction. As long as the predicted probability of the outcomes that actually materialized was nonzero, things are just fine. Wigner’s friend’s «wrong» prediction of the probability is an artifact of his perspective that completely neglects the possible re-interference of the portions of the wave function. The very fact that he «perceives» one outcome at that time means that he is admitting that all his predictions that depend on the relative phases of the parts of the wave function may be wrong.

You may also try to «measure» the probability of \(\hat O_{\rm Wigner}\) accurately by repeating the same experiment many times. If Eugene Wigner prepares the whole box in the very same state many times and at the end, he measures \(\hat O_{\rm Wigner}\), he will always find that this operator’s value is equal to one.

Can Wigner’s friend «measure» that the probability is 36 percent by repeating the very same experiment many times? Well, he cannot. If the same experiment with the box is repeated many times, Wigner’s friend will sometimes see an alive cat, sometimes he will see a dead cat after the same amount of time.

More importantly, the asymmetry between Wigner and his friend enters at this stage. Eugene Wigner is the master of the whole box so he may be sure that the whole system is prepared in a specific state. But Wigner’s friend lives in the box and he has no control over the state of particles outside the box, e.g. electrons in Wigner’s brain. The friend can’t really measure Wigner’s brain, at least not with the required accuracy, so he can’t ever be sure that Wigner is found in a pure state etc. He will have to describe all objects outside the box using a mixed state, a density matrix.

A «sufficiently» mixed state predicts that the probability of nearly everything is nonzero. The exactly vanishing probability of some outcome is only possible if the system is an eigenstate of an operator, so it must be pure (or at least not near-maximally mixed).

Wigner’s friend’s perspective and predictions are inevitably inaccurate due to his limited abilities to measure the physical objects outside the box. However, quantum mechanics gives him a method to predict things in this context, too. There will never be a contradiction – two orthogonal (mutually exclusive) states are always evolving into two orthogonal (mutually exclusive) states – this is mathematically expressed by the unitarity of the evolution operator – so if there are two contradictory assumptions about the initial state of the system, the probability that these assumptions will be proven by the final state (after a sequence of manipulations and measurements) is zero.

At the same moment, quantum mechanics allows one to describe physical systems with an arbitrary accuracy that may be obtained by any gadgets that the theory allows. In particular, Wigner himself is supposed to have a complete control over all the degrees of freedom – and relative phases – defining the state of the box. And with this complete control, he can make perfect probabilistic predictions that just work.

At any rate, the wave function isn’t a collection of the «classical data» in any sense. A spectacular manifestation of this insight is that two different state vectors are not mutually exclusive – mutually contradicting – if their inner product is nonzero. Instead, the probability that one state «is» another is given by Born’s rule. You can’t understand quantum mechanics if you don’t accept its logical framework. Different points of the phase space in classical physics were mutually exclusive; different state vectors in the Hilbert space are not mutually exclusive because the Hilbert space has a fundamentally different interpretation than the phase space.

Another important fact is that a value of an observable may only be assumed to be known if it is measured, and a measurement of \(\hat L\) unavoidably disrupts the physical system and especially the (probabilities of) values of all observables \(\hat K\) that don’t commute with \(\hat L\), i.e. those for which \(\hat K \hat L -\hat L \hat K \neq 0\). People who are assuming that physical objects may be in pure states even though no measurements are performed are guaranteed to be led to wrong conclusions, too. But that point – measurements inevitably influence the system – may be discussed in a future blog post.