On Wed, Feb 11, 2015 at 4:25 AM, Stathis Papaioannou <[email protected]> wrote:
> > > On Wednesday, February 11, 2015, Jason Resch <[email protected]> wrote: > >> >> >> On Tue, Feb 10, 2015 at 8:15 PM, Stathis Papaioannou <[email protected]> >> wrote: >> >>> >>> >>> On Wednesday, February 11, 2015, Jason Resch <[email protected]> >>> wrote: >>> >>>> >>>> >>>> On Tue, Feb 10, 2015 at 3:30 PM, Stathis Papaioannou < >>>> [email protected]> wrote: >>>> >>>>> >>>>> >>>>> On Wednesday, February 11, 2015, Jason Resch <[email protected]> >>>>> wrote: >>>>> >>>>>> If you define increased intelligence as decreased probability of >>>>>> having a false belief on any randomly chosen proposition, then >>>>>> superintelligences will be wrong on almost nothing, and their beliefs >>>>>> will >>>>>> converge as their intelligence rises. Therefore nearly all >>>>>> superintelligences will operate according to the same belief system. We >>>>>> should stop worrying about trying to ensure friendly AI, it will either >>>>>> be >>>>>> friendly or it won't according to what is right. >>>>>> >>>>>> I think chances are that it will be friendly, since I happen to >>>>>> believe in universal personhood, and if that belief is correct, then >>>>>> superintelligences will also come to believe it is correct. And with the >>>>>> belief in universal personhood it would know that harm to others is harm >>>>>> to >>>>>> the self. >>>>>> >>>>> >>>>> Having accurate beliefs about the world and having goals are two >>>>> unrelated things. If I like stamp collecting, being intelligent will help >>>>> me to collect stamps, it will help me see if stamp collecting clashes with >>>>> a higher priority goal, but it won't help me decide if my goals are >>>>> worthy. >>>>> >>>>> >>>>> >>>> Were all your goals set at birth and driven by biology, or are some of >>>> your goals based on what you've since learned about the world? Perhaps >>>> learning about universal personhood (for example), could lead one to >>>> believe that charity is a worthy goal, and perhaps deserving of more time >>>> than collecting stamps. >>>> >>> >>> The implication is that if you believe in universal personhood then even >>> if you are selfish you will be motivated towards charity. But the >>> selfishness itself, as a primary value, is not amenable to rational >>> analysis. There is no inconsistency in a superintelligent AI that is >>> selfish, or one that is charitable, or one that believes the single most >>> important thing in the world is to collect stamps. >>> >>> >>> >> But doing something well (regardless of what it is) is almost always >> improved by having greater knowledge, so would not gathering greater >> knowledge become a secondary sub goal for nearly any supintelligence that >> has goals? Is it impossible that it might discover and decide to pursue >> other goals during that time? After all, capacity to change one's mine >> seems to be a requirement for any intelligence process, or any process on >> the path towards superintelligence. >> > > Sure, but the AI may still decide to do evil, perverse or self destructive > things. There is no contradiction in superintelligence behaving this way. > > > It's an assumption to say there is no contradiction. If it's beliefs are defined to be almost completely correct, why would its actions not follow its beliefs and also be almost completely correct? Unless we are talking about a superintelligence with some kind of malfunction, I would think its actions would be driven by its beliefs. Do you think morality is relative or universal? Jason -- You received this message because you are subscribed to the Google Groups "Everything List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/everything-list. For more options, visit https://groups.google.com/d/optout.
