Having started my research on language and diffusion models last year, the hyperbolic turn we’re taking, as a society, is quite depressing. Many people in or adjacent to AI research and development seem to be getting more and more unhinged by the minute.
The people who study it as a flawed technology, and posit that the danger to society and the economy comes from a lack of appropriate functionality seem to be a fast shrinking minority—ostracised by AGI doomers and evangelists alike.
For the record, I’m of the school that thinks that the fact-free generative aspects of the technology are the one part where the functionality of the technology is dangerous, as it threatens to destroy our digital commons and the viability of pretty much all online media.
But the rest? People keep claiming that language models are capable of things they turn out to be very far from able to do. Building on those imagined capabilities—relying on them as cornerstones of our society—is as much of a disaster as a destroyed commons.
But the discourse as devolved into science-fiction clichés. Super-intelligent AIs will save us! No, destroy us! They’re able to fix healthcare! And education!
These tools are much narrower than they’re made out to be and much less stable than you think. Becoming reliant on them, even in areas where they’re fine 99% of the time, is a much bigger risk than most are realising.
Language and diffusion models are remarkable technologies, but they aren’t the magic solutions people claim, and the studies being done seems almost purpose-designed to work as marketing, not research.
Next to none of the productivity studies, whether for code or writing, touch on real world tasks. It’s almost entirely synthetic. Code studies are all testing algorithm, parsing, or data structure implementation tasks that 99% of coders just aren’t working on. Because of how well-documented and functional they are, these tasks lend themselves to code assistants. Writing studies are, essentially, business fan-fiction completely detached from collaboration or processes. In situ studies are also flawed as designed, because the only way to get a true control is to eject yourself into a parallel dimension where the company didn’t deploy AI assistants. For the record, a 15% bump in productivity is on the low end for the novelty and observer-expectancy effect combined, as is usually the case is AI research.
Beyond the advocacy research, everything else seems to be polarising into two religions:
- “AGI is coming, and it’ll be great!”
- “AGI might be coming, and it’ll be a disaster!”
We do have many writers and researchers—such as Emily M. Bender, Timnit Gebru, and El Mahdi El Mhamdi to name a few—who are pointing out that the problems with the technology is that of functionality. The functionality that it does have—fluent language synthesis—is more harmful that helpful. It’s also volatile and unstable—the opposite of the reliability you normally need in software. It doesn’t have the functionality it would need to have to do 90% of the tasks AI evangelists claim it does.
The reward for taking what is essentially a purely pragmatic stance towards assessing new technology?
They get demonised by their own field.
This was inevitable because the survival of AI as a field and as an industry depends on being aligned with power.
In Meredith Whittaker’s words:
The view that LLMs are powerful/capable isn’t wrong, necessarily. But it is a perspective that, whatever your individual positionality, aligns you w some of the most powerful corps on earth.
This doesn’t make you “a bad person”, nor does believing some of the wilder LLM hype. But irrespective, these are much safer positions to take than those that requires meaningfully opposing these institutions.
Kindness is imperative, we need to be gentle. I get that. & we can’t frame ~“pro LLM” v ~“LLM critical” as an interpersonal argument whose relationship to power is best understood by looking @ the LinkedIn of each participant. Stakes are high & they ramify far beyond this.
She’s warning us against valorising the tech, but the drive to valorise is going to feel irresistible in any technological field as they all have the notion of technological progress as manifest destiny built into their foundations. Any warning against it will feel like a direct attack on your character.
Hence, the polarisation. The ‘acceptable’ critics are those that make sure they defer to the myth of technology’s manifest destiny in all of their writing—even if it’s only done in an almost ritualistic, perfunctory way: Ceterum censeo Carthaginem esse delendam, words spoken solely to identify you in terms of allegiance.
By always including a nod to the core beliefs of a field, they make their views palatable to it. Belief becomes the common ground even as the discourse polarises into doom and evangelism. Even the doomers believe in technological progress, their disagreement is—fittingly—about the polarity of the result.
This leaves no space for critiques based on pragmatism or functionality.
It’s all a bit depressing.
I have to hope that the crowd that focuses on the politics, power, and functionality of language and diffusion models will win out in the end, but I’m worried that we’re going to see so much of our society descend into mania in the meantime.
For more of my writing on AI, check out my book The Intelligence Illusion: a practical guide to the business risks of Generative AI.