I have on my bookshelves a book I bought in the late 1980s, when I started experimenting with machine learning (or “artificial neural networks” or “connectionism”, as the field was then called). In this book, Thinking Machines, the authors described the Chinese Room thought experiment, proposed by the philosopher John Searle in 1980. Strings of Chinese characters (“input questions”) are passed under the room’s door. By following the instructions from a computer programme for correctly manipulating Chinese symbols, Searle, who does not speak Chinese, is able to send the appropriate sequence of Chinese characters back out under the door (“output answers”), thereby convincing observers outside the room that there is a Chinese speaker inside the room.
At the end of his thought experiment, Searle asks whether the computer programme could be said to understand Chinese (“strong AI”) or whether it just simulated that ability (“weak AI”). As my experience with Google Translate reveals, such a question is still relevant now, even if today’s data-driven ML algorithms are entirely different from the symbol-manipulating programmes of the early 1980s.
Within the ML community, the focus is almost entirely on building ever more impressive demonstrators, such as the work by DeepMind researchers in the UK on game-playing machines. In 2016, their AlphaGo ML algorithm was able to beat the world’s best Go player. AlphaGo Zero and AlphaZero then went beyond AlphaGo by generating their own training datasets, using a combination of deep neural networks, reinforcement learning and game-specific representations, to achieve “super-human” performance. As a result of two AlphaZero machines playing millions of games against each other, they explored a huge space of possibilities and were able to make moves that a human player could not have foreseen.
But AlphaZero has no more understanding of Go than Google Translate has of French – or than any other machine translation tool has of any other language or its grammar. The most powerful ML model today, GPT-3, is used in hundreds of text-generating apps, such as chatbots, producing nearly five billion words a day; but does GPT-3 understand the text it automatically generates?
There has been extraordinary progress in learning algorithms, computational hardware and size of training data, but are we any closer to building thinking machines than we were 30 years ago (whether we call these strong AI, Artificial General Intelligence or super-intelligence)? Does the ability to learn how to translate from one language to another, play intellectually demanding games, or generate text automatically in response to prompts demonstrate the remarkable success of weak AI or is it the first hint of strong AI?
Such a debate should really be taking place within higher education, especially as academics and graduate students resume in-person seminars and workshops. Emily Bender, a linguist from the University of Washington, last year updated the Chinese Room thought experiment with her “octopus test” to emphasise the importance of the link between form and meaning: two people living alone on remote islands send each other text messages through an underwater cable. An octopus listens in on the pulses, then cuts off one of the islanders and attempts to impersonate them by tapping on the cable. What happens when one of the islanders sends a message with instructions for how to build a coconut catapult but also asks the other islander for suggestions on how to improve the design?
Dialogue around such deep questions with ML researchers in computer science departments has been minimal, however, because most of them are too busy trying to keep up with the big tech companies while training PhD students – who are soon absorbed into the ever-growing labs of those very same companies.
In a world of chatbots and autonomous vehicles, fundamental questions about the limits of AI/ML need urgently to be re-visited, with insights from multiple disciplines. ML researchers in academia should engage in a new dialogue with colleagues in philosophy, linguistics and cognitive science. Reuben College, Oxford’s newest college, intends to play its part in promoting these multidisciplinary exchanges.
Lionel Tarassenko is president of Reuben College, University of Oxford.