Eliezer,
As a counter to my own previous argument
about the risk of the
simultaneous failure of AGIs, your argument is likely to be closest to
being right in certain circumstances after the time dimension is taken
into account.
Our previous argument has been around
the black and white yes/no
notion of whether simultaneous failure is likely in an AGI population.
I have argued (to my own satisfaction
:) ) that simultaneous failure in
the most important areas of mentation and psychological health is likely
to be very low. ie. that failure is likely to have a normal distribution
(bell curve form).
But the practical question is, even
if I'm right technically, how
temporally compressed is the bell curve likely to be? Will we get
enough time from when the first failures occur in a population to when
the majority of the failures occur for corrective action to be taken by
humans and the non-failed AGIs working together?
It seems to me that if the bell curve
is compressed temporally then the
message of your argument has practical significance. So we need to
look carefully at design inadequacies and early childhood education
inadequacies to see where temporally bunched failures might occur
(and given that AGI minds will be so complex that precise anticipation
of temporally bunched failures is likely in many cases to be
impossible) then we probably need to implement AGI architectures,
training programs and monitoring and improvement regimes that have
a precautionary preventive effect.
The people from Boeing, Airbus and
NASA might have some
experience in trying to make fail-safe super-complex systems - and
they might be prepared to fund research into this area. Maybe AGI
researchers/developers could get some $$s to further their work
through this channel. There is an interesting loop here - AGIs might be
useful entities on aircraft as part of an anti-terrorism strategy - but you
would need to guard against AGI failure. So the civil aircraft industry
might be interested in general AGI development as well as the safety
issue.
Cheers, Philip
