Hi Brent, On Sat, Sep 10, 2016 at 8:29 PM, Brent Meeker <[email protected]> wrote: > Good paper.
Thanks! > Many of the thoughts I've had about the subject too. But I > think your use of persistence is misleading. There are different ways to > persist. Bacteria persist, mountains persist - but very differently. Ok, I talk about persistence in the very specific sense of Dawkin's selfish gene. Forward propagation of information in a system of self-replicators. > The > AI that people worry about is one that modifies it's utility function to be > like humans, i.e. to compete for the same resources and persist by > replicating and by annihilating competitors. That is one type of worry. The other (e.g.: the "paper clip" scenario) does not require replication. It is purely the worry that side-effects of maximizing the utility function will have catastrophic consequences, while the AI is just doing exactly what we ask of it. > You may say that replicating > isn't necessarily a good way to persist and a really intelligent being would > realize this; but I'd argue it doesn't matter, some AI can adopt that > utility function, just as bacteria do, and be a threat to humans, just as > bacteria are. I don't say that replication is the only way to persist. What I say is that evolutionary pressure is the only way to care about persisting. Cheers, Telmo. > Brent > > > > On 9/10/2016 6:06 AM, Telmo Menezes wrote: >> >> Hi everyone, >> >> I published this working paper on arxiv, same title as this email: >> http://arxiv.org/abs/1609.02009 >> >> Criticisms most welcome! >> >> Best, >> Telmo. >> > > -- > You received this message because you are subscribed to the Google Groups > "Everything List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/everything-list. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Everything List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/everything-list. For more options, visit https://groups.google.com/d/optout.
