But the idea of
having just one Novamente seems somewhat unrealistic and quite risky to
me.
If the Novamente design is
going to enable boostraping as you plan then your one Novamente is going to
end up being very powerful. If you try to be the gatekeeper to this one
powerful AGI then (a) the rest of the world will end up considering your
organisation as worse than Microsoft and many of your clients are not going to
want to be held to ranson by being dependent on your one AGI for their mission
critical work and (b) the one super-Novamente might develop ideas if it own
that might not include you or anyone else being the
gatekeeper.
The idea of one
super-Novamente is also dangerous because this one AGI will develop its own
perspecitive on things and given its growing power that perpective or bias
could become very dangerous for any one or anything that didn't fit in with
that perspective.
I think an AGI
needs other AGIs to relate to as a community so that a community of leaning
develops with multiple perspectives available. This I think is the only way
that the accelerating bootstraping of AGIs can be handled with any possibility
of being safe.
******
That
feels to me like a lot of anthropomorphizing...
Clearly there are going to be a fair number of
commercial partial-Novamente software systems in use before we finish the real
uber-Novamente.... But, I don't see why there necessarily has to be more
than one Novamente taught to be a true AGI.
To
me, it's an unanswered question whether it's a better use of,
say, 10^5 computers to make them all one Novamente, or to partition them
into a society of Novamente's....
* ***********
> Philip: So why not proceed to develop Novamentes
down two different
> paths simultaneously - the path you have already
designed - where
> experience-based learning is virtually the only
strategy, and a variant
> where some Novamentes have a modicum of carefully
designed pre-wiring
> for ethics.........
(coupled with a
major program of experience-based learning)?
On reflection I
can well imagine that you are not ready to make any commitment to my
suggestion to give the dual (simultaneous) development path approach a
go. But would you be prepared to explore the possibility of
dual
(simultaneous) development path approach? I think there would be much to
be learned from at least examining the dual approach prior to making any
commitment.
What do you
think?
*******************
I guess I'm accustomed to working in a
limited-resources situation, where you just have to make an intuitive call as
to the best way to do something and then go with it ... and then try the next
way on the list, if one's first way didn't work...
Of course, if there are a lot of resources available,
one can explore parallel paths simultaneously and do more of a breadth-first
rather than a depth-first search through design space !
-- Ben G
