dear AGI group,
those two questions keep me awake at night - they probably are rather
stupid (and anthropomorphic) and therefore i failed to manage to find
useful input on the web ;-)
1) will all sufficiently intelligent AGI agents ultimately share the
same behaviour everywhere in our universe just like zero intelligence
behaves the same everywhere? could it be that all AGI systems end up
with the same utility function because the mathematical definition of
joy, joice and growth (or whatever the universal virtues of life might
be) are the same to every superintelligent AGI agent? (thank you for
your brilliant book ben!) to me this would seem extremely elegant - all
the innumerable configurations of life throughout the universe branching
out with one common goal. just like our universe branches out from
minimum entropy to maybe end up in a ~maximum entropy state.
2) what could be the incentive for such a superintelligent AGI to stay
"alive"? if joy is defined as a reduction of entropy or an increase in
unity (as ben suggests) what is there left to learn or to unite with?
how much complexity would any piece of art need to embody in order to be
of any interest to such an agent? there would be nothing left to invent,
nothing left to research? i don't see how simulating new universes would
be as interesting or fun as many people suggest? i don't know how
getting super intelligent would make it more desirable to watch regimes
kill millions of conscious agents ... but maybe that's just me? of
course such an agent would not need to actually feel bored but existing
just for the sake of it seems not very satisfactory to me. it could help
out other civilizations in its cone of light or answer their arecibo
message with "enjoy your barbaric life, being superintelligent sucks"
and shut itself down afterwards. fermi's paradox solved!
-- just another camel
-------------------------------------------
AGI
Archives: https://www.listbox.com/member/archive/303/=now
RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-c97d2393
Modify Your Subscription:
https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-2484a968
Powered by Listbox: http://www.listbox.com
