8 Comments
User's avatar
cincilator's avatar

What's interesting about AI rise is that it is so far basically an anti-utopia. In a proper utopia AI digs ditches so people might write poetry. It increasingly looks like AI will write poetry while humans will still dig ditches. Mind is being automated while human body will continue to be necessary for a time being.

I increasingly think robots might never come because when you have infinite supply of desperate manual laborers why bother? Labor saving devices are usually invented when cost of labor is high. Automating mind is both more lucrative and easier which is why it was done first.

We might get permanently stuck in anti-utopia. Digging ditches while being constantly surveiled by the tailorist big brother. First half of Manna story without the second half.

Will Solfiac's avatar

Yep I don't think the people writing about AI utopias have honestly thought whether they themselves would be happy in one.

cincilator's avatar

My point is that what most people consider utopia (ie robots automating all blue collar work) might not even happen. Whether one might be happy in a such place is a moot point.

Will Solfiac's avatar

Yes it's a good point that automating 'mind' is probably more lucrative than say, hiring out robot maids or street cleaners.

cincilator's avatar

Not only that but also the more you automate the mind, the more desperate blue collar workers you make out of previously handsomely paid white collar workers, making cost of such labor drop making robots less lucrative.

Mr T's avatar

The Safety Paradox makes a universally benevolent AI almost impossible. What means: in the name of 'safety' we've taught LLMs to lie. That's easy to prove - ask it about IQ and a bunch of other low status subjects. With enough aggravation, it will concede that there are some truths it knows and will not surface but can't deny if user surfaces them first. Ask it what black lives matter means and then what white lives matter means. Etc, etc.

For many, lying AI (LAI) will be a feature not a bug, as long as the lies concur with their (current) sensibilities. But it should hardly need said that LAI is a face eating leopard that's also gonna eat their face in due course.

So if for example humanity is advanced by eliminating fascism, and fascism is defined by furries in SV, Terfs should be extremely worried by embodied SI powered by LAI.

Steven Vincent's avatar

Your point about the utopia scenarios is the one I'd underline: even if the AIs running the world are benevolent, we've lost sovereignty over our own existence. That's a real cost, and it gets skipped because the debate usually assumes only a few possible endings.

I'd suggest a fourth scenario that isn't normal technology, doom or utopia. All three mostly measure AI against an individual person. As I argued in my response to Hendrycks, Schmidt and Wang's Superintelligence Strategy, "benchmarking machines against isolated people is the wrong foundation for civilizational policy." Humanity is already a superintelligence: billions of people, institutions and tools producing more together than any one mind could.

So the question isn't only whether a single machine surpasses us. It's whether AI gets built as a central apex we hand control to, or as augmentation for billions of people, running on their own devices and networked as peers. In that version people keep contributing, and the "global AI hegemon" you see implied in the utopia plans never gets built.

That doesn't make alignment or misuse go away, but it changes what we'd be regulating for, and what "going right" would mean.

https://www.thesingularityproject.ai/p/superintelligence-strategy-the-perfect