At the risk of oversimplifying or misinterpreting your position, here
are some thoughts that I think follow from what I understand of your
position so far. But I may be wildly mistaken. Please correct my mistakes.
There is one unique attractor in state space. Any individual of a
species that develops in a certain way -- which is to say, finds itself
in a certain region of the state space -- will thereafter necessarily be
drawn to the attractor if it acts in its own self interest. This
attractor is friendliness (F). [The attractor needs to be sufficiently
distant from present humanity in state space that our general
unfriendliness and frequent hostility towards F is explainable and
plausible. And it needs to be sufficiently powerful that coming under
its influence given time is plausible or perhaps likely.]
Since any sufficiently advanced species will eventually be drawn towards
F, the CEV of all species is F. Therefore F is not species-specific, and
has nothing to do with any particular species or the characteristics of
the first species that develops an AGI (AI). This means that genuine
conflict between friendly species or between friendly individuals is not
even possible, so there is no question of an AI needing to arbitrate
between the conflicting interests of two friendly individuals or groups
of individuals. Of course, there will still be conflicts between
non-friendlies, and the AI may arbitrate and/or intervene.
The AI will not be empathetic towards homo sapiens sapiens in
particular. It will be empathetic towards f-beings (friendly beings in
the technical sense), whether they exist or not (since the AI might be
the only being anywhere near the attractor). This means no specific acts
of the AI towards any species or individuals are ruled out, since it
might be part of their CEV (which is the CEV of all beings), even
though they are not smart enough to realize it.
Since the AI empathizes not with humanity but with f-beings in general,
it is possible (likely) that some of humanity's most fundamental beliefs
may be wrong from the perspective of an f-being. Without getting into
the debate of the merits of virtual-space versus meat-space and
uploading, etc., it seems to follow that *if* the view that everything
of importance is preserved (no arguments about this, it is an assumption
for the sake of this point only) in virtual-space and *if* turning the
Earth into computronium and uploading humanity and all of Earth's beings
would be vastly more efficient a use of the planet, *then* the AI should
do this (perhaps would be morally obligated to do this) -- even if every
human being pleads for this not to occur. The AI would have judged that
if we were only smarter, faster, more the kind of people we would like
to be, etc., we would actually prefer the computronium scenario.
You might argue that from the perspective of F, this would not be
desirable because ..., but we are so far from F in state space that we
really don't know which would be preferable from that perspective (even
if we actually had detailed knowledge about the computronium scenario
and its limitations/capabilities to replace our wild speculations). It
might be the case that property rights, say, would preclude any f-being
from considering the computronium scenario preferable, but we don't know
that, and we can't know that with certainty at present. Likewise, our
analysis of the sub-goals of friendly beings might be incorrect, which
would make it unlikely that our analysis of what a friendly being will
actually believe is mistaken.
It's become apparent to me in thinking about this that 'friendliness' is
really not a good term for the attitude of an f-being that we are
talking about: that of acting solely in the interest of f-beings
(whether others exist or not) and in consistency with the CEV of all
sufficiently ... beings. It is really just acting rationally (according
to a system that we do not understand and may vehemently disagree with).
One thing I am still unclear about is the extent to which the AI is
morally obligated to intervene to prevent harm. For example, if the AI
judged that the inner life of a cow is rich enough to deserve protection
and that human beings can easily survive without beef, would it be
morally obligated to intervene and prevent the killing of cows for food?
If it would not be morally obligated, how do you propose to distinguish
between that case (assuming it makes the judgments it does but isn't
obligated to intervene) and another case where it makes the same
judgments and is morally obligated to intervene (assuming it would be
required to intervene in some cases).
Thoughts?? Apologies for this rather long and rambling post.
joseph
-------------------------------------------
agi
Archives: http://www.listbox.com/member/archive/303/=now
RSS Feed: http://www.listbox.com/member/archive/rss/303/
Modify Your Subscription:
http://www.listbox.com/member/?member_id=8660244&id_secret=95818715-a78a9b
Powered by Listbox: http://www.listbox.com
