Notices by Emily M. Bender (she/her) (emilymbender@dair-community.social)

As OpenAI and Meta introduce LLM-driven searchbots, I'd like to once again remind people that neither LLMs nor chatbots are good technology for information access.

A thread, with links:

Chirag Shah and I wrote about this in two academic papers:
2022: https://dl.acm.org/doi/10.1145/3498366.3505816
2024: https://dl.acm.org/doi/10.1145/3649468

We also have an op-ed from Dec 2022:
https://iai.tv/articles/all-knowing-machines-are-a-fantasy-auid-2334

>>

Why can't more journalists reporting on "AI" straightforwardly say "This is bad, actually"? Case in point: In this article juxtaposing the grandiose claims of OpenAI et al with the massive environmental footprint of LLMs, Goldman still has to include this weird AI optimism:

https://fortune.com/2024/07/12/ai-bizarro-world-agi-electricity-carbon-emissions/

>>

It seems like there are just endless bad ideas about how to use "AI". Here are some new ones courtesy of the UK government.

... and a short thread because there is so much awfulness in this one article.
/1

https://www.ft.com/content/f2ae55bf-b9fa-49b5-ac0e-8b7411729539

Check it out! Stochastic Parrots is now available in audiopaper form, read by @timnitGebru @meg Angelina McMillan-Major and me and produced by Christie Taylor.

If you've never read the original and appreciate this format for reading, this is for you!

https://soundcloud.com/emily-m-bender/stochastic-parrots?si=94532ca828264f8b95527b32fab943e2&utm_source=clipboard&utm_medium=text&utm_campaign=social_sharing

Fellow educators:

1) Don't do cop shit.
2) Don't pay for a service that does cop shit.
3) Don't trust claims of system performance without documentation.
4) DO educate your students about what ChatGPT is and why they wouldn't want to use it.

Screen caps from some spam I received just now:

One of the ways in which the web is like an ecosystem is that a synthetic text spill in one part of it can leak into others. Here, someone has posted ChatGPT output (unclear to what end) and the Google indexed it.

I learned of this particular #GoogleFail via BlueSky user aviendha69's post:

https://bsky.app/profile/aviendha69.bsky.social/post/3k4yxz2zz7f2v

... and verified that I could find the same results. (For now, anyway. Sometimes things things get patched.)

Anyone noticed that when folks get even a little specific about the supposed benefits of "AI" the call outs are frequently "curing disease" (sometimes specifically cancer and/or Alzheimer's) and "addressing climate change"? I'm hearing it enough to wonder where these talking points come from.

>>

Policymakers: Don't waste your time on the fantasies of the techbros saying "Oh noes, we're building something TOO powerful." Listen instead to those who are studying how corporations (and governments) are using technology (and the narratives of "AI") to concentrate and wield power.

Start with the work of brilliant scholars like Ruha Benjamin, Meredith Broussard, Safiya Noble, Timnit Gebru, Sasha Costanza-Chock and journalists like Karen Hao and Billy Perrigo.

Yes, there should be robust public funding but I'd priortize non-CS fields that look at the impacts of these things over "technical AI safety research".

Also "the dramatic economic and political disruptions that AI will cause". Uh, we don't have AI. We do have corporations and VCs looking to make the most $$ possible with little care for what it does to democracy (and the environment).

>>

Yes, we should have regulation that requires provenance and watermarking systems. (And it should ALWAYS be obvious when you've encountered synthetic text, images, voices, etc.)

Yes, there should be liability --- but that liability should clearly rest with people & corporations. "AI-caused harm" already makes it sound like there aren't *people* deciding to deploy these things.

>>

Some of these policy goals make sense:

>>

They then say: "AI research and development should be refocused on making today's powerful, state-of-the-art systems more accurate, safe, interpretable, transparent, robust, aligned, trustworthy, and loyal."

Uh, accurate, transparent and interpretable make sense. "Safe", depending on what they imagine is "unsafe". "Aligned" is a codeword for weird AGI fantasies. And "loyal" conjures up autonomous, sentient entities. #AIhype

>>

Just sayin': We wrote a whole paper in late 2020 (Stochastic Parrots, 2021) pointing out that this head-long rush to ever larger language models without considering risks was a bad thing. But the risks and harms have never been about "too powerful AI".

Instead: They're about concentration of power in the hands of people, about reproducing systems of oppression, about damage to the information ecosystem, and about damage to the natural ecosystem (through profligate use of energy resources).

>>

Okay, calling for a pause, something like a truce amongst the AI labs. Maybe the folks who think they're really building AI will consider it framed like this?

>>

I'm mean, I'm glad that the letter authors & signatories are asking "Should we let machines flood our information channels with propaganda and untruth?" but the questions after that are just unhinged #AIhype, helping those building this stuff sell it.

>>

On the "sparks" paper:
https://twitter.com/emilymbender/status/1638891855718002691?s=20

On the GPT-4 ad copy:
https://twitter.com/emilymbender/status/1635697381244272640?s=20

On "general" tasks:
https://datasets-benchmarks-proceedings.neurips.cc/paper/2021/hash/084b6fbb10729ed4da8c3d3f5a3ae7c9-Abstract-round2.html

>>

Next paragraph. Human-competitive at general tasks, eh? What does footnote 3 reference? The speculative fiction novella known as the "Sparks paper" and OpenAI's non-technical ad copy for GPT4. ROFLMAO.

>>

And could the creators "reliably control" #ChatGPT et al. Yes, they could --- by simply not setting them up as easily accessible sources of non-information poisoning our information ecosystem.

And could folks "understand" these systems? There are plenty of open questions about how deep neural nets map inputs to outputs, but we'd be much better positioned to study them if the AI labs provided transparency about training data, model architecture, and training regimes.

>>

Footnote 1 there points to a lot of papers, starting with Stochastic Parrots. But we are not talking about hypothetical "AI systems with human-competitive intelligence" in that paper. We're talking about large language models.

https://faculty.washington.edu/ebender/stochasticparrots/

And the rest of that paragraph. Yes, AI labs are locked in an out-of-control race, but no one has developed a "digital mind" and they aren't in the process of doing that.

>>

There a few things in the letter that I do agree with, I'll try to pull them out of the dreck as I go along.

So, into the #AIhype. It starts with "AI systems with human-competitive intelligence can pose profound risks to society and humanity, as shown by extensive research[1]".

>>