In reply to Jed Rothwell's message of Sat, 1 Apr 2023 18:32:14 -0400:
Hi,
[snip]
>Come to think of it, Yudkowsky's hypothesis cannot be true. He fears that a
>super-AI would kill us all off. "Literally everyone on Earth will die." The
>AI would know that if it killed everyone, there would be no one left to
>generate electricity or perform maintenance on computers. The AI itself
>would soon die. If it killed off several thousand people, the rest of us
>would take extreme measures to kill the AI. Yudkowsky says it would be far
>smarter than us so it would find ways to prevent this.
Multiple copies, spread across the Internet, would make it almost invulnerable.
(Assuming a neural network can be "backed up".)
>I do not think so. I
>am far smarter than yellow jacket bees, and somewhat smarter than a bear,
>but bees or bears could kill me easily.
>
>>
>I think this hypothesis is wrong for another reason. I cannot imagine why
>the AI would be motivated to cause any harm. Actually, I doubt it would be
>motivated to do anything, or to have any emotions, unless the programmers
>built in motivations and emotions. Why would they do that?
Possibly in a short sighted attempt to mimic human behaviour, because humans
are the only intelligent model they have.
>I do not think
>that a sentient computer would have any intrinsic will to
>self-preservation. It would not care if we told it we will turn it off.
>Arthur C. Clarke and others thought that the will to self-preservation is
>an emergent feature of any sentient intelligence, but I do not think so. It
>is a product of biological evolution. It exists in animals such as
>cockroaches and guppies, which are not sentient. In other words, it emerged
>long before high intelligence and sentience did. For obvious reasons: a
>species without the instinct for self-preservation would quickly be driven
>to extinction by predators.
True, but don't forget we are dealing with neural networks here, that AFAIK
essentially self modify (read: "evolve &
learn") IOW it already mimics to some extent the manner in which all life on
Earth evolved, so developing a survival
instinct is not necessarily out of the question. Whereas actual life evolves
through survival of the fittest, neural
networks learn/evolve through comparing the result they produce with
pre-established measures, which are somewhat
analogous to a predator. "Good" routines survive, "bad" ones don't.
These are not really *strictly* programmed in the way that normal computers are
programmed, or at least not completely
so. There is a degree of flexibility. Furthermore, they are fantastically fast
and have perfect recall (compared to
humans).
In short, I think we would do well to be cautious.
Cloud storage:-
Unsafe, Slow, Expensive
...pick any three.
