Read the appendix, p37ff. He's not making arguments -- he's explaining,
with a
few pointers into the literature, some parts of completely standard and
accepted economics and game theory. It's all very basic stuff.
The problem with "accepted economics and game theory" is that in a proper
scientific sense, they actually prove very little and certainly far, FAR
less than people extrapolate them to mean (or worse yet, "prove").
All of the scientific experiments in game theory are very, VERY limited and
deal with entities with little memory in small, toy systems. If you
extrapolate their results with no additional input and no emergent effects,
you can end up with arguments like Omohundro's BUT claiming that this
extrapolation *proves* anything is very poor science. It's just
speculation/science fiction and there are any number of reasons to believe
that Omohundro's theories are incorrect -- the largest one, of course, being
"If all goal-based systems end up evil, why isn't every *naturally*
intelligent entity evil?
----- Original Message -----
From: "J Storrs Hall, PhD" <[EMAIL PROTECTED]>
To: <[email protected]>
Sent: Sunday, May 25, 2008 8:14 AM
Subject: Re: [agi] Goal Driven Systems and AI Dangers [WAS Re: Singularity
Outcomes...]
The paper can be found at
http://selfawaresystems.files.wordpress.com/2008/01/nature_of_self_improving_ai.pdf
Read the appendix, p37ff. He's not making arguments -- he's explaining,
with a
few pointers into the literature, some parts of completely standard and
accepted economics and game theory. It's all very basic stuff.
On Sunday 25 May 2008 06:26:59 am, Jim Bromer wrote:
----- Original Message ----
From: Richard Loosemore <[EMAIL PROTECTED]>
Richard Loosemore said:
If you look at his paper carefully, you will see that at every step of
the way he introduces assumptions as if they were obvious facts ... and
in all the cases I have bothered to think through, these all stem from
the fact that he has a particular kind of mechanism in mind (one which
has a goal stack and a utility function). There are so many of these
assertions pulled out of think air that I found it gave me a headache
just to read the paper. ...
But this is silly: where was his examination of the systems various
motives? Where did he consider the difference between different
implementations of the entire motivational mechanism (my distinction
between GS and MES systems)? Nowhere. He just asserts, without
argument, that the system would be obsessed, and that any attempt by us
to put locks on the system would result in "an arms race of measures and
countermeasures."
That is just one example of how he pulls conclusions out of thin air.
-------------------------------------------
Your argument about the difference between a GS and an MES system is a
strawman argument. Omohundro never made the argument, nor did he touch on
it
as far as I can tell. I did not find his paper very interesting either,
but
you are the one who seems to be pulling conclusions out of thin air.
You can introduce the GS vs MES argument if you want, but you cannot then
argue from the implication that everyone has to refer to it or else stand
guilty of pulling arguments out of thin air.
His paper Nature of Self Improving Artificial Intelligence September 5,
2007, revised January 21, 2008 provides a lot of reasoning. I don't find
the
reasoning compelling, but the idea that he is just pulling conclusions out
of
thin air is just bluster.
Jim Bromer
-------------------------------------------
agi
Archives: http://www.listbox.com/member/archive/303/=now
RSS Feed: http://www.listbox.com/member/archive/rss/303/
Modify Your Subscription:
http://www.listbox.com/member/?&;
Powered by Listbox: http://www.listbox.com
-------------------------------------------
agi
Archives: http://www.listbox.com/member/archive/303/=now
RSS Feed: http://www.listbox.com/member/archive/rss/303/
Modify Your Subscription:
http://www.listbox.com/member/?&;
Powered by Listbox: http://www.listbox.com
-------------------------------------------
agi
Archives: http://www.listbox.com/member/archive/303/=now
RSS Feed: http://www.listbox.com/member/archive/rss/303/
Modify Your Subscription:
http://www.listbox.com/member/?member_id=8660244&id_secret=103754539-40ed26
Powered by Listbox: http://www.listbox.com
