Emily M. Bender (she/her)
Put this up on Twitter over the weekend, but I guess it should go here, too:
Okay, I read it so you don't have to. Here's a reaction thread to OpenAI / Sam Altman's blog post from Friday "Planning for AGI and beyond":
https://openai.com/blog/planning-for-agi-and-beyond/
From the get-go this is just gross. They think they are really in the business of developing/shaping "AGI". And they think they are positioned to decide what "benefits all of humanity".
>>
Emily M. Bender (she/her)
But oh noes -- the magical imagined AGI also has downsides! But it is so so tempting and important to create, that we can't not create it. Note the next rhetorical sleight of hand here. Now AGI is an unpreventable future.
>>
Emily M. Bender (she/her)
Then Altman invites the reader to imagine that AGI ("if successfully created") is literally magic. Also, What does "turbocharging the economy" mean, if there is already abundance? More $$$ for the super rich, has to be.
Also, note the rhetorical sleight of hand there. Paragraph 1 has AGI as a hypothetical ("if successfully created") but by para 2 it already is something that "has potential".
>>
Emily M. Bender (she/her)
Then a series of principles for how to ensure that AGI is "beneficial". This includes "governance of AGI" as something that is "widely and fairly shared", but I've seen exactly nothing from OpenAI about or advocating for building shared governance structures.
Meanwhile, "continuously learn and adapt by deploying less powerful versions of the technology" suggests that they think that the various GPTs are "less powerful versions of AGI".
>>
Emily M. Bender (she/her)
What's in fn1? A massive presupposition failure: The GPTs are learning information about word distributions in lots and lots of text + what word patterns are associated with higher scores (from human raters). That's it.
>>
Emily M. Bender (she/her)
<recordscratch> hang on: did he just say "maximarlly flourish in the universe"? What kind of weirdo longtermist, space colonizing fantasy is that coming from?
>>
Emily M. Bender (she/her)
As I said in my earlier short thread on this blog post, I wish I could just laugh at these people, but unfortunately they are attempting (and I think succeeding) to engage the discussion about regulation of so-called AI systems.
>>
Emily M. Bender (she/her)
Okay, back to Altman. "As our systems get closer to AGI" -- here's a false presupposition again. Your system isn't AGI, it isn't a step towards AGI, and yet you're dropping that in as if the reader is just supposed to nod along.
Oh, and did you all catch that shout out to xrisk? Weirdo longertermist fantasy indeed.
>>
Emily M. Bender (she/her)
Hey OpenAI, I'm speaking to you from 2018 to say: DOCUMENT YOUR DAMN DATASETS. Also, to everyone else: If you don't know what's in it, don't use it.
Source: https://aclanthology.org/Q18-1041.pdf
>>
Emily M. Bender (she/her)
Then there's a glib paragraph about how "most expert predictions have been wrong so far" ending in footnote 2.
Paraphrasing: "Our experts thought we could do this as a non-profit, but then we realized we wanted MOAR MONEY. Also we thought we should just do everything open source but then we decided nah. Also, can't be bothered to even document the systems or datasets."
>>
Emily M. Bender (she/her)
Similarly here, this seems designed to promote the idea that the models they have already put into their API (GPT-2, GPT-3, ChatGPT) are the early stages of "AGI" being "stewarded into existence".
>>
Emily M. Bender (she/her)
The problem isn't regulating "AI" or future "AGI". It's protecting individuals from corporate and government overreach using "AI" to cut costs and or deflect accountability.
>>
Emily M. Bender (she/her)
What's needed is regulation about: how data can be collected and used, transparency of datasets, models and the deployment of text/image generation systems, recourse and contestability of any automated decision making, etc.
Talking about text synthesis machines as if they were "AI" muddies the waters and hampers effective discussions about data rights, transparency, protection from automated decision systems, surveillance, and all the rest of the pressing issues.
>>
Emily M. Bender (she/her)
"Continuum of intelligence" is gross, not least for the suggestions of ableism, eugenics, transhumanism etc. But also "rate of progress [of] the past decade"-? Progress towards what? Ever larger carbon footprints? More plausible fake text?
>>
Emily M. Bender (she/her)
Here's a bunch of promises about future oversight by unnamed independent auditors and also "major world governments" (who counts as major? who decides?). Also, how about just DOCUMENTING YOUR DAMN DATA for everyone to see?
>>
Emily M. Bender (she/her)
Wait what -- now they're talking seriously about "late-stage AGI development"?
>>
Emily M. Bender (she/her)
[This is exhausting, but I started. Might as well finish.]
>>
Emily M. Bender (she/her)
LOLOL -- calling something a "ratio" doesn't make measurable or, ahem, real.
>>
Emily M. Bender (she/her)
The contradiction in these next two paragraphs is stunning: We think you should be able to do whatever you want with our systems, because "diversity of ideas" but also we think we can align the systems with "human values". So, assholes can create fake revenge porn, but that's okay because-?
>>
Emily M. Bender (she/her)
And, more to the point: There are harms NOW. Harms to privacy, theft of creative output, harms to our information ecosystems, and harms from the scaled reproduction of biases. An organization that cared about "benefiting humanity" wouldn't be developing and disseminating tech that does those things.
>>
Emily M. Bender (she/her)
No, they don't want to address actual problems in the actual world (which would require ceding power). They want to believe themselves gods who can not only create a "superintelligence" but have the beneficence to do so in a way that is "aligned" with humanity.
/fin
Fifi Lamoura
@emilymbender The whole "the results will be beyond even our wildest dreams but let's not even consider the very foreseeable real world problems" vibe is strong. I'm sure this press release is adding to global warming.
phryk 🏴
From what I've seen of the "AI" community, I fully expect that *if* AGI is created, it won't be planned.
An engineer will just connect one more subsystem and things will emerge from that.
Also if consciousness (however weakly defined) emerges, it probably won't be spotted for quite a while, and with these systems being products there's a financial incentive to explain anything resembling consciousness away, not to mention how impossibly hard it would be to *prove* consciousness…
phryk 🏴
@emilymbender After reading both Altmans text and your thread, I gotta say you pretty much nailed it.
One additional thing I noticed tho is that Altman, especially given how much of a given he seems to think AGI is, does not say a single word about AGI rights, but talks about them as "products" and about "pulling them from production".
０７６萌ＳＮＳ is a social network, courtesy of ０７６. It runs on GNU social, version 2.0.2-beta0, available under the GNU Affero General Public License.
All ０７６萌ＳＮＳ content and data are available under the Creative Commons Attribution 3.0 license.