ChatGPT's Trough of Disillusionment

3 years ago

If you don't recognize the phrase "trough of disillusionment," check out Gartner's Hype Cycles. It's a great way to conceptualize how new technology emerges and is received in the marketplace (and by society at large). Roughly speaking, the sequence is innovation, followed by inflated expectations, disillusionment, enlightenment and, finally, productivity. I think we are at the peak of inflated expectations and rapidly moving into the trough of disillusionment. This is all to be expected, except that the depth of the trough for ChatGPT – and the family of large language model products (LLMs) it represents – is likely to be deeper than most due to the unique nature of this technology and the way it has been brought to market.

I'm sorry Dave, I'm afraid I can't do that

The context for my claim starts with the term "AI." If you ask somebody inside the technology industry what AI is, you'll get a grab bag list of products and services: chat bots, image recognition, text-to-speech, speech-to-text, form recognizers, speaker recognition, offensive content detection, machine learning, etc. Now ask any non-technical lay-person what the term "AI" means and they will likely respond with Hal from 2001: A Space Odyssey, Data from Star Trek: The Next Generation or Agent Smith from The Matrix.

The gap between these two different definitions is immense – and, I believe, dangerous. The popular concept of "AI" is actually what industry people call artificial general intelligence (AGI) which doesn't exist yet, even as an experimental technology. The best forecasts are that it is still decades away from becoming reality.

The next piece of context is how LLM technology works. The best layperson analogy I've seen is that it is like a "stochastic parrot." Roughly speaking, LLMs are like parrots that have picked up many phrases of English and, when prompted, randomly form and utter phrases based on what they've picked up. The most critical aspect of this analogy is that the parrot doesn't understand the English it hears nor the phrases it utters. The remarkable aspect of LLMs comes from the "large" part of their name. Following the analogy, they are stochastic parrots that have picked up every phrase ever written and available on the Internet.

Indistinguishable From Magic

The result is astonishing. When prompted, LLMs generate text that looks and reads like a human understood the prompt and originated the response. And that's where the last piece of context gets us into trouble: humans' tendency to anthropomorphize. It's a very powerful force and difficult to resist. Even when you are fully aware the LLM has no actual understanding of the prompt it was given nor applied any reasoning to the text it generated (sometimes resulting in "hallucination" of facts), it is nearly impossible not to feel you are interacting with another human being.

I think this has revealed a remarkable aspect of human intelligence: while LLMs are not AGI, human intelligence, on a moment-to-moment basis, does express stochastic parrot-like behavior. We learn phrases (and more generally patterns) throughout our lives and reproduce them when prompted by circumstances – sometimes with little conscious thought. But make no mistake, LLMs are missing that ingredient that is critical to being human: reasoning.

When you put all of this together, it's a dangerous situation and an irresponsible way to introduce and market a novel technology. The danger is when laypeople interact with a technology that can't reason about what it produces while believing they are receiving a reasoned response. It has already cost one life and will likely result in more lives lost. And this disconnect appears to reside even within the companies producing the technology. What marketing department that understands the "stochastic parrot" nature of LLMs would risk brand damage with obvious mistakes like this?

The Consequences are Real

Bottom line: I believe the responsible delivery of LLMs to the masses must include continual reminders of their stochastic parrot nature and that no reasoning is taking place in the production of the output. This has clearly not been Silicon Valley's priority so far, but as we get closer to the end of the innovation phase of the hype cycle, they must address it. Otherwise, the trough of disillusionment and attendant backlash will be severe, resulting in yet another black eye for the same industry that plagued us with social media.