Josh,
Thank you very much for the pointers (and replying so rapidly).
You're very right that people misinterpret and over-extrapolate econ and
game
theory, but when properly understood and applied, they are a valuable tool
for analyzing the forces shaping the further evolution of AGIs and indeed
may
be our only one.
No. I would argue that there is a lot of good basic research into human and
primate behavior that is more applicable since it's already been tested and
requires less extrapolation (and visibly shows where a lot of current
extrapoloation is just plain wrong).
But in the long run, slightly nicer programs can out-compete slightly
nastier
ones, and then in turn be out-competed by slightly nicer ones yet. For
example, in a simulation with ``noise,'' meaning that occasionally at
random
a ``cooperate'' is turned in to a ``defect,'' tit-for-tat gets hung up in
feuds, and a generous version that occasionally forgives a defection does
better--but only if the really nasty strategies have been knocked out by
tit-for-tat first. Even better is a strategy called Pavlov, due to an
extremely simple form of learning. Pavlov repeats its previous play if it
``won,'' and switches if it ``lost.'' In particular, it cooperates
whenever
both it and its opponent did the same thing the previous time--it's a
true,
if very primitive, ``cahooter.'' Pavlov also needs the underbrush to be
cleared by a ``stern retaliatory strategy like tit-for-tat.''
Actually, I've seen this presented as a good clock analogy as well . . . .
slightly nicer out-competes slightly nastier but once it hits a nasty enough
program, it loses.
The true question is, how do you raise the niceness of *all* players and
prevent defection -- because being the single bad guy is a winning strategy
while being just one among many is horrible for everyone.
So, in simplistic computer simulations at least, evolution seems to go
through
a set of phases with different (and improving!) moral character.
So why do so many people think evolution favors the exactly the opposite?
-------------------------------------------
agi
Archives: http://www.listbox.com/member/archive/303/=now
RSS Feed: http://www.listbox.com/member/archive/rss/303/
Modify Your Subscription:
http://www.listbox.com/member/?member_id=8660244&id_secret=103754539-40ed26
Powered by Listbox: http://www.listbox.com
