On Tue, Nov 28, 2017 at 3:33 PM, Jason Resch <[email protected]> wrote: > > > On Tue, Nov 28, 2017 at 5:03 AM, Telmo Menezes <[email protected]> > wrote: >> >> >> >> If you have some time/patience, let me know what you think of my arguments >> here: >> https://arxiv.org/abs/1609.02009 >> > > > Telmo, > > Interesting read.
Thanks for reading, and for the comments. > In general I have a lot of sympathy for this view. > > I think there may be an inverse relationship between intelligence and > confidence in actions. That is, the more intelligence the super > intelligence becomes, the less certain it may be about whether a given > course of action is correct, and this could lead to a paralysis of sorts. That is an interesting idea. My initial intuition is to argue that in a purely probabilistic system, the more intelligent actors might assume (correctly) that they are more likely to predict the future correctly than the less intelligent ones. A bit like an expert poker player: they know they can't win them all, but they also know that they will win in the long term. > I've also read a few science fiction stories where upon being uploaded, > people modify their brains to activate their pleasure centers and > effectively become zombies thereafter. I wonder though, and perhaps this > relates to the nature of possible conscious experiences, would a > super-intelligence prefer to exist and continually stimulate its utility > function, or would it be equally (or more?) happy to define its utility > function as being maximized by not existing and then kill itself? E.g. with > the choice between an eternal heroine trip/orgasm vs. suicide, what would a > rational agent choose? I agree, this is a deep question. I would say that it goes into they mystery of qualia. I would say that the arguments that I present in the paper are valid from the third-person, but we have to take a grain of salt because we don't understand qualia/consciousness. > Another question, what if a super intelligence agreed with the ideas > expressed in the one-self paper and it determined its self interest extends > to all conscious beings. Would it, acting under such a belief, seek to help > (and not modify) existing conscious life realize their utility functions, or > would it instead decide to modify the utility functions of those other > conscious life forms it has the power to change? Would it modify their > utility functions to seek to stop existing and then kill them? If it does > so instantaneously, it doesn't seem like it really ever modified their > utility functions in the first place and instead of assisting their > suicides, is murdering them. Ok, I see we have similar thoughts. I don't write about this because these are things that I see almost as my personal faith, not as something that I can address scientifically. Bruno might disagree, if assuming comp. My personal faith: we are all the same person, including animals and who knows what else. I cannot show that this to be true, and I further think that it is beyond the Gödelian veil. Even doing introspection (and enhanced introspection, let's leave it at that...), I fluctuate between "yes, we are all the same person" and "bullshit". To defend my faith: independently of the truth, if everyone operates on this belief we are all better off. Of course I am not claiming to be a saint or even close, I am just saying that this seems like an overall benevolent belief system. So, as you say, the AI might reach this same conclusion: it's better to bet that the well-being of all is equivalent to my well-being. I think our morality is constrained by evolution -- in the same way that some people suspect that even our perception of reality is constrained, a sort of Darwinian-Plato-Cave. Most people naturally feel that a human life is more valuable than the life of other sentient beings. I feel that myself, but is this fundamentally justifiable or is it just the outcome of kinship selection? What would the AI think about this? To go further: not so long ago, most people would freely defend that the lives of people from their ethnicity are more valuable than those of other ethnicities. It seems to me that only recently did the civilization process start to oppose this way of thinking, and it seems clear that there is still a long way to go. > It seem to me, that under computationalism, realizing conscious states > requires computation, and in our universe computation requires time. > Therefore maximizing the types and kinds of conscious states one wants to > exist requires persistence over time. I think for a conscious super > intelligence, utility functions must somehow be based of the perceived > utility of various conscious experiences. Ceasing to exist (or ceasing to > realize new conscious states) serves only to eliminate your own contribution > of experiences to the total set of experiences that exist. Therefore the > super intelligence that kills itself, is in effect, deciding a preference > for the other already extant conscious life forms and their experiences over > its own. Well put, I agree. Even without AI, bit assuming comp: would it make sense to kill yourself if you figure that you are significantly less happy than most other conscious beings? > If you look at everything that motivates all human endeavors, it is > ultimately, all about realizing and maximizing good experiences while > avoiding and minimizing bad experiences. I read replies to this, but I agree with you. People sacrifice for their kids because ultimately they bet that this makes them more happy than any short-term pleasure. It might seem cold to discuss such calculations, but within the "we are all the same person" faith it is perfectly benevolent. > Another consideration is that so long as the ratio of superintelligences > that clone themselves remains greater than the ratio of superintelligences > that modify their utility function to become inert (over some period of > time) remains greater than 1, it seems they will be subject to darwinian > forces and will be selected for those with lower rates of modifying their > utility function to become inert. Agreed. When I talk about superintelligences becoming inert, I am not making a prediction. I am just trying to take a certain way of thinking to its ultimate consequences. > Overall your paper leads to a great number of interesting topics that > deserve further exploration. Thanks for sharing it. Thanks for saying that! Telmo. > Jason > > -- > You received this message because you are subscribed to the Google Groups > "Everything List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/everything-list. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Everything List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/everything-list. For more options, visit https://groups.google.com/d/optout.
