This raises another point for me though. In another post (2008-03-06
14:36) you said:
"It would *NOT* be Friendly if I have a goal that I not be turned into
computronium even if <your clause> (which I hereby state that I do)"
Yet, if I understand our recent exchange correctly, it is possible for
this to occur and be a Friendly action regardless of what sub-goals I may
or may have. (It's just extremely unlikely given ..., which is an
important distinction.)
You are correct. There were so many other points flying around during the
earlier post that I "approximated" the "extremely unlikely" to an absolute
*NOT* for clarity (which then later obviously made it less clear for you).
Somehow I need to clearly state that even where it looks like I'm using
absolutes, I'm really only doing it to emphasize greater unlikeliness than
usual, not absolutehood.
It would be nice to have some ballpark probability estimates though to
know what we mean by extremely unlikely. 10E-6 is a very different beast
than 10E-1000.
Yeah. It wuld be nice but a) I don't believe that I can do it accurately at
all, b) I strongly believe that the estimates vary a lot from situation to
situation, and c) it would be a distraction and a diversion if my estimates
weren't pretty darn good.
Argh! I would argue that Friendliness is *not* that distant. Can't you
see how the attractor that I'm describing is both self-interest and
Friendly because **ultimately they are the same thing** (OK, so maybe
that *IS* enlightenment :-)
Well, I was thinking of the region of state space close to the attractor
as being a sort of "approaching perfection" region in terms of certain
desirable qualities and capabilities, and I don't think we're really close
to that. Having said that, I'm by temperament a pessimist and a skeptic,
but I would go along with "heading in the right direction".
You'll probably like the part after the next part (society) which is either
"The nature of evil" or "The good, the bad, and the evil". I had a lot of
fun with it.
-------------------------------------------
agi
Archives: http://www.listbox.com/member/archive/303/=now
RSS Feed: http://www.listbox.com/member/archive/rss/303/
Modify Your Subscription:
http://www.listbox.com/member/?member_id=8660244&id_secret=95818715-a78a9b
Powered by Listbox: http://www.listbox.com
