Artificial intelligence systems e.g., ChatGPT learn, as we humans do, the meaning of words by observing the linguistic contexts in which they are used. The science that studies all this (and more) is called distributional semantics, a discipline that has found its first comprehensive discussion in the book Distributional Semantics from Cambridge University Press written by Alessandro Lenci (photo) of the University of Pisa and Magnus Sahlgren of Artificial Intelligence Sweden.
“Like computers, we also learn many concepts by observing how words are used in different linguistic contexts, and this is one of the fundamental factors in the creative power of the human mind,” says Alessandro Lenci, Professor of Computational Linguistics at the Department of Philology, Literature and Linguistics.” This aspect makes distributional semantics a fascinating area of research that combines theoretical, computational, and cognitive perspectives for the study of language and its application, in order to develop intelligent artificial systems.”
But to understand how the context mechanism works, let us take the hypothetical word “blimp”. From the sentence: “I just drank some frozen blimp”, we can easily understand that it refers to some kind of liquid. If, on the other hand, the same word was in a difference sentence such as: “A blimp has been barking all night”, it is clear that “blimp” would have referred to an animal, most likely a dog. These simple examples illustrate the principle by which computational models of distributional semantics work. They basically learn the meaning of words and other linguistic expressions, analysing by statistical and mathematical methods the contexts in which they are used and other words that often recur with them.
“This kind of distributional method,” Lenci concludes, “is also part of the way children and adults learn the meanings of many words, thus enriching their vocabulary, and it is the same principle on which systems such as ChatGPT are based to acquire their knowledge, which they use to answer our questions accordingly.”