On Tue, Nov 28, 2017 at 5:03 AM, Telmo Menezes <[email protected]> wrote:
> > > If you have some time/patience, let me know what you think of my arguments > here: > https://arxiv.org/abs/1609.02009 > > Telmo, Interesting read. In general I have a lot of sympathy for this view. I think there may be an inverse relationship between intelligence and confidence in actions. That is, the more intelligence the super intelligence becomes, the less certain it may be about whether a given course of action is correct, and this could lead to a paralysis of sorts. I've also read a few science fiction stories where upon being uploaded, people modify their brains to activate their pleasure centers and effectively become zombies thereafter. I wonder though, and perhaps this relates to the nature of possible conscious experiences, would a super-intelligence prefer to exist and continually stimulate its utility function, or would it be equally (or more?) happy to define its utility function as being maximized by not existing and then kill itself? E.g. with the choice between an eternal heroine trip/orgasm vs. suicide, what would a rational agent choose? Another question, what if a super intelligence agreed with the ideas expressed in the one-self paper and it determined its self interest extends to all conscious beings. Would it, acting under such a belief, seek to help (and not modify) existing conscious life realize their utility functions, or would it instead decide to modify the utility functions of those other conscious life forms it has the power to change? Would it modify their utility functions to seek to stop existing and then kill them? If it does so instantaneously, it doesn't seem like it really ever modified their utility functions in the first place and instead of assisting their suicides, is murdering them. It seem to me, that under computationalism, realizing conscious states requires computation, and in our universe computation requires time. Therefore maximizing the types and kinds of conscious states one wants to exist requires persistence over time. I think for a conscious super intelligence, utility functions must somehow be based of the perceived utility of various conscious experiences. Ceasing to exist (or ceasing to realize new conscious states) serves only to eliminate your own contribution of experiences to the total set of experiences that exist. Therefore the super intelligence that kills itself, is in effect, deciding a preference for the other already extant conscious life forms and their experiences over its own. If you look at everything that motivates all human endeavors, it is ultimately, all about realizing and maximizing good experiences while avoiding and minimizing bad experiences. Another consideration is that so long as the ratio of superintelligences that clone themselves remains greater than the ratio of superintelligences that modify their utility function to become inert (over some period of time) remains greater than 1, it seems they will be subject to darwinian forces and will be selected for those with lower rates of modifying their utility function to become inert. Overall your paper leads to a great number of interesting topics that deserve further exploration. Thanks for sharing it. Jason -- You received this message because you are subscribed to the Google Groups "Everything List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/everything-list. For more options, visit https://groups.google.com/d/optout.
