On 6/7/06, Eliezer S. Yudkowsky [EMAIL PROTECTED] wrote:
But -as I said in the chapter conclusion - imagine how careful you would haveto be if you wanted to survive as an *individual*; and that is howcareful humanity must be to survive existential risks.
*nods* I know where you're coming from
Hi Eli,
First, as discussed in the chapter, there's a major context change
between the AI's prehuman stage and the AI's posthuman stage. I can
think of *many* failure modes such that the AI appears to behave well as
a prehuman, then wipes out humanity as a posthuman. What I fear from
this
The AI system could be built as more of an advisor of actions that we might
take.
The investment field has already progressed into automated program trading.
I would bet that the investment brokers have some human monitors watching
and maybe even approving the trading.
But you have heard the
Wow! Reading this, I must
say that I was struck by the Geddes-like proportion of *claims* to *reasonable
proofs* (or even true discussionswhere you've deigned to share even the
beginnings of a proof).
Claims like "AGI understanding
will always run ahead of FAI understanding" are
AWESOME post . . . .
- Original Message -
From:
Russell Wallace
To: agi@v2.listbox.com
Sent: Wednesday, June 07, 2006 2:20
AM
Subject: Re: [agi] Two draft papers: AI
and existential risk; heuristics and biases
On 6/7/06, Eliezer S. Yudkowsky
[EMAIL
On Wed, Jun 07, 2006 at 11:32:36AM -0400, Mark Waser wrote:
I think that we as a community need to get off our butts and start
building consensus as to what even the barest framework of friendliness is.
I think that we've seen more than enough proof that no one here can go on for
more
There will always be some hostile, virus or Trojan spreading elements.
I can envision AI wars.
AI can do a lot for mankind, yet the development may get bogged down in
more security versus productive development.
Interesting, very interesting...
Dan Goe
On 6/6/06, Eliezer S. Yudkowsky [EMAIL PROTECTED] wrote:
I espouse the
Proactionary Principle for everything *except* existential risks.
The Proactionary Principle is a putative optimum strategy for progress
within an inherently risky and uncertain environment. How do you
reconcile your
I think that we as a community need to get off our butts and start
building consensus as to what even the barest framework of friendliness
is. I think that we've seen more than enough proof that no one here can
go on for more than twenty lines without numerous people objecting
vociferously to
Eliezer,
I don't think it
inappropriate to cite a problem that is general to supervised learning
and reinforcement, when your proposal is to, in general, use supervised
learning and reinforcement. You can always appeal to a different
algorithm or a different implementation that, in some
On Wed, 2006-06-07 at 11:32 -0400, Mark Waser wrote:
I think that we as a community need to get off our butts and start
building consensus as to what even the barest framework of
friendliness is. I think that we've seen more than enough proof that
no one here can go on for more than
The point is that you're unlikely to murder the human race, if given the
chance, and neither is Kass. In fact, if given the chance, you will
protect them.
But what about all of those lovely fundamentalist Christians or Muslims
who see no problem with killing infidels (see Crusades, Jihad,
On Wed, Jun 07, 2006 at 03:31:10PM -0400, Mark Waser wrote:
But what about all of those lovely fundamentalist Christians or Muslims
who see no problem with killing infidels (see Crusades, Jihad, etc.)? They
won't murder the human race as a whole but they will take out a major piece
of
And if you build a consensus, what will you have accomplished?
I will have accomplished some degree of error-checking (and
reality-checking) better than just trying to do it myself. I will have
started a process towards peaceful acceptance rather than attempting to
impose my flawed will
What was your operational definition of friendliness, again?
My personal operational definition of friendliness is simply what my current
self would be willing to see implemented as the highest level goal of an
AGI.
Obviously, that includes being robust enough that it doesn't evolve into
Hi,
When reading this nice survey paper of Eliezer's
_Cognitive biases potentially affecting judgment of global risks_
http://singinst.org/Biases.pdf
I was reminded of some of the heuristics and biases that exist in the
Novamente system right now.
For instance, consider the case of
On Wed, 2006-06-07 at 16:13 -0400, Mark Waser wrote:
I'm pretty sure that I've got the science and math that I need and, as I
Okay. I supposed the opposite, not because of anything you said, but
because the base rate is so low.
said above, I don't feel compelled to listen to everyone.
Mark Waser wrote:
What was your operational definition of friendliness, again?
My personal operational definition of friendliness is simply what my
current self would be willing to see implemented as the highest level
goal of an AGI.
Obviously, that includes being robust enough that it
Example: Linda is 31, outspoken, and very bright. She majored in
philosophy in college. As a student, she was deeply concerned with
discrimination and other social issues, and she participated in
antinuclear demonstrations. Which statement is more likely to be
true?
a. Linda is a bank
On Jun 7, 2006, at 5:52 PM, Mike Ross wrote:
I think its actually correct to say that (b) is more likely than (a).
Humans dont get this wrong because they are bad at reasoning. They
get this wrong because of the ambiguities of natural language.
Unlike mathematical language, human speech has
I don't think this has been raised before, the only similar suggestion
is that we should start by understanding systems that might be weak
and then convert it to a strong system rather than aiming for weakness
that is hard to convert to a strong system.
Caveats:
1) I don't believe strong
Whether the common response to the Linda example is a fallacy or not
depends on the normative theory that is used as the standard of
correct thinking.
The traditional probabilistic interpretation is purely extensional,
in the sense that the degree of belief for L is a C is interpreted
as the
On Thu, 2006-06-08 at 07:56 +0800, Pei Wang wrote:
Whether the common response to the Linda example is a fallacy or not
depends on the normative theory that is used as the standard of
correct thinking.
The paper I linked described an experiment in which subjects were paid
for making correct
Peter,
I don't mean that humans never make errors (see my papers). When in a
situation a normative theory should be applied, but the subjects
failed to do so, it is an error.
However, it is not always the case, or I should say that it is usually
not the case. For example:
*. according to
24 matches
Mail list logo
