Telmo Menezes <>

> >> So if the slave AI has a fixed goal structure with the number one goal
>> being  to always do what humans tell it to do and the humans order it to
>> determine  the truth or falsehood of something unprovable then its infinite
>> loop time  and you've got yourself a space heater not a AI.
> > Right, but I'm not thinking of something that straightforward. We
> Already have that -- normal processors. Any one of them will do precisely
> what we order it to do.

Yes, and because the microprocessors in our computers do precisely what we
order them to do and not what we want them to do they sometimes go into
infinite loops, and because they never get bored they will stay in that
loop forever, or at least until we reboot our computer; if we're just using
the computer to surf the internet that's only a minor inconvenience but if
the computer were running a nuclear power plant or the New York Stock
Exchange it would be somewhat more serious; and if your friendly AI were
running the entire world the necessity of a reboot would be even more

> >> Real minds avoid this  infinite loop problem because real minds don't
>> have fixed goals, real minds
>>  get bored and give up.
> At that level, boredom would be a very simple mechanism, easily replaced
> by something like: try this for x amount of time and then move on to
> another goal

But how long should x be? Perhaps in just one more second you'll get the
answer, or maybe two, or maybe 10 billion years, or maybe never. I think
determining where to place the boredom point for a given type of problem
may be the most difficult part in making an AI; Turing tells us we'll never
find a algorithm that works perfectly on all problems all of the time, so
we'll just have to settle for an algorithm that works pretty well on most
problems most of the time.

And you're opening up a huge security hole, in fact they just don't get any
bigger, you're telling the AI that if this whole "always obey humans no
matter what" thing isn't going anywhere just ignore it and move on to
something else. It's hard enough to protect a computer when the hacker is
no smarter than you are, but now you're trying to outsmart a computer
that's thousands of times smarter than yourself. It can't be done.

Incidentally I've speculated that unusual ways to place the boredom point
may explain the link between genius and madness particularly among
mathematicians. Great mathematicians can focus on a problem with ferocious
intensity, for years if necessary, and find solutions that you or I could
not, but in everyday life that same attribute of mind can sometimes cause
them to behave in ways that seem to be at bit, ah, odd.

 John K Clark

You received this message because you are subscribed to the Google Groups 
"Everything List" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
To post to this group, send email to
Visit this group at
For more options, visit

Reply via email to