Quantum theory to model the meaning of words

Quantum phisics

Quantum theory explains in a wonderful way what happens in the Universe at the scale of elementary particles. Many of the effects that occur at such a scale (superposition, entanglement, …) sound weird and contra-intuitive to us, but are quite real and are what physicists have found in their labs since decades.

Since I started working with some Natural Language Processing-related topics years ago, I got the vague impression that there were analogies between some effects described by quantum physics and the completely different world of words and meanings: same as particles, words can be in several “states” at the same time (meanings) and only when they are observed (they are read or listened in a certain context) the superposition of states “collapses” in one of them (a well defined meaning). I remember a very interesting “coffe talk” with Núria Bel (UPF) a time ago who also shared similar intuitions!

WaveFunctionCollapse

Wave function collapse [figure taken from http://afriedman.org/AndysWebPage/BSJ/CopenhagenManyWorlds.html ]

The above picture illustrates the “wave function collapse”: the position of a particle is expressed as a likelihood or probability, which is the superposition of its many possible positions (left). Then, when a measurement (observation) is made, the function “collapses” and the position is well determined (right).

In analogy, let’s imagine the word “apple” and their possible meanings (as a tree, a fruit, the electronics firm, …). The notion of “apple” can be perceived as a superposition of such meanings or states. In isolation, one cannot say what is the state (meaning) of the word. However, when the word “apple” is observed in the context of a sentence about food (“He cooked an apple pie”), the ambiguity resolves (collapses) into the sense of apple as a fruit.

Actually, the mathematical formalism underlying quantum mechanics is not necessarily tied to physics (in fact, Hilbert proposed it many years before it was adopted by quantum mechanics) and can be potentially applied to any field in which uncertainty has to be handled.

It turns out that a group of researchers at University of Edinburgh (Blacoe et al.) have already proposed a formal framework for capturing lexical meaning featuring key quantum aspects such as superposition and entanglement, in a paper titled “A Quantum-Theoretic Approach to Distributional Semantics” [1]. Let’s review the features of quantum theory which make it a promising framework for modelling meaning, according to the authors.  I am copying here some lines from their paper’s introduction, which is very concise and informative:

“[…] Superposition is a way of modeling uncertainty, more so than in classical probability theory. […] An electron whose location in an atom is uncertain can be modeled as being in a superposition of locations. Analogously, words in natural language can have multiple meanings. […] The meanings of words in a semantic space are superposed in a way which is intuitively similar to the atom’s electron.

Entanglement concerns the relationship between probability distribution from the probability distributions of their constituent parts. With regard to word meanings, entanglement encodes (hidden) relationships between concepts. The different senses of a word “exist in parallel” until it is observed in some context. This reduction of ambiguity has effects on other concepts connected via entanglement.

The notion of incompatibility is fundamental to quantum systems. In classical systems, it is assumed by default that measurements are compatible, that is, independent, and as a result the order in which these take place does not matter. By contrast in quantum theory, measurements may share (hidden) order-sensitive inter-dependencies and the outcome of the first measurement can change the outcome of the second measurement. 

Interference is a feature of quantum probability that can cause classical assumptions such as the law of total probability to be violated. When concepts interact their joint representation can exhibit nonclassical behavior, e.g., with regard to conjunction and disjunction. An often cited example is the “guppy effect”. Although guppy is an example of a pet-fish it is neither a very typical pet nor fish. […]”

The idea of representing meanings as states in a Hilbert space resembles other distributional semantic models such as Latent Semantic Analysis [2] and Explicit Semantic Analysis [3]. Actually, Aerts and Czachor [4] demonstrated that LSA is essentially a Hilbert space formalism.

But quantum theory has been used not only to model semantic spaces. In fact, many effects related to human psychology and cognition have found in quantum theory a satisfactory explanation. Quantum theory is specially effective when modelling how humans make judgements and decisions under conflict and uncertainty. I encourage you to read an excellent review article on the matter by Bruza et al. “Quantum cognition: a new theoretical approach to psychology” [5].

References

[1] W. Blacoe, E. Kashefi, and M. Lapata, “A Quantum-Theoretic approach to distributional semantics,” in Proceedings of NAACL-HLT’13. ACL, 2013, pp. 847-857.
[2] T. K. Landauer and S. T. Dumais, “A solution to plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge,” Psychological Review, vol. 104, no. 2, pp. 211-240, Apr. 1997.
[3] E. Gabrilovich and S. Markovitch, “Computing semantic relatedness using wikipedia-based explicit semantic analysis,” in Proceedings of The Twentieth International Joint Conference for Artificial Intelligence, Hyderabad, India, 2007, pp. 1606-1611.
[4] D. Aerts and M. Czachor, “Quantum Aspects of Semantic Analysis and Symbolic Artificial Intelligence,” Journal of Physics A-Mathematical and General, vol. 37, pp. L123-L132, 2004.
[5] P. D. Bruza, Z. Wang, and J. R. Busemeyer, “Quantum cognition: a new theoretical approach to psychology,” Trends in Cognitive Sciences, vol. 19, no. 7, pp. 383-393, Jul. 2015.
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s