On Wed, Aug 27, 2014 at 12:21 PM, Telmo Menezes <[email protected]> wrote:
> > On Wed, Aug 27, 2014 at 1:53 PM, Terren Suydam <[email protected]> > wrote: > >> >> The space of possibilities quickly scales beyond the wildest imaginings >> of computing power. Chess AIs are already better than humans, because they >> more or less implement this approach, and it turns out you "only" need to >> computer a few hundred million positions per second to do that. Obviously >> that's a toy environment... the possibilities inherent in the real world >> are even be enumerable according to some predefined ontology (i.e. that >> would be required to specify in a minimax type AI). >> > > Ok, but of course minimax was also a toy example. Several algorithms that > already exist could be combined: deep learning, bayesian belief networks, > genetic programming and so on. A clever combination of algorithms plus the > still ongoing exponential growth in available computational power could > soon unleash something impressive. Of course I am just challenging your > intuition, mostly because it's a fun topic :) Who knows who's right... > I think these are overlapping intuitions. On one hand, there is the idea that given enough computing/data resources, something can be created that - regardless of how limited its domain of operation - is still a threat in unexpected ways. On the other hand is the idea that AIs which pose real threats - threats we are not capable of stopping - require a quantum leap forward in cognitive flexibility, if you will. Although my POV is aligned with the latter intuition, I actually agree with the former, but consider the kinds of threats involved to be bounded in ways we can in principle control. Although in practice it is possible for them to do damage so quickly we can't prevent it. Perhaps my idea of intelligence is too limited. I am assuming that something capable of being a real threat will be able to generate its own ontologies, creatively model them in ways that build on and relate to existing ontologies, simulate and test those new models, etc., generate value judgments using these new models with respect to overarching utility function(s). It is suspiciously similar to human intelligence. The difference is that as an *artificial* intelligence with a different embodiement and different algorithms, the modeling they would arrive at could well be strikingly different from how we see the world, with all the attendant problems that could pose for us given the eventually superior computing power. > Another interesting/scary scenario to think about is the possibility of a > self-mutating computer program proliferating under our noses until it's too > late (and exploiting the Internet to create a very powerful meta-computer > by stealing a few cpu cycles from everyone). > I think something like this could do a lot of damage very quickly, but by accident... in a similar way perhaps to the occasional meltdowns caused by the collective behaviors of micro-second market-making algorithms. I find it exceedingly unlikely that an AGI will spontaneously emerge from a self-mutating process like you describe. Again, if this kind of thing were likely, or at least not extremely unlikely, I think it suggests that AGI is a lot simpler than it really is. > > >> >> >>> >>> >>>> You're talking about an AI that arrives at novel solutions, which >>>> requires the ability to invent/simulate/act on new models in new domains >>>> (AGI). >>>> >>> >>> Evolutionary computation already achieves novelty and invention, to a >>> degree. I concur that it is still not AGI. But it could already be a >>> threat, given enough computational resources. >>> >> >> AGI is a threat because it's utility function would necessarily be >> sufficiently "meta" that it could create novel sub-goals. We would not >> necessarily be able to control whether it chose a goal that was compatible >> with ours. >> >> It comes down to how the utility function is defined. For Google Car, the >> utility function probably tests actions along the lines of "get from A to B >> safely, as quickly as possible". If a Google Car is engineered with >> evolutionary methods to generate novel solutions (would be overkill but >> bear with me), the novelty generated is contained within the utility >> function. It might generate a novel route that conventional map algorithms >> wouldn't find, but it would be impossible for it to find a solution like >> "helicopter the car past this traffic jam". >> > > What prevents the car from transforming into an helicopter and flying is > not the utility function but the set of available actions. I have been > playing with evolutionary computation for some time now, and one thing I > learned is to not trust my intuition on the real constraints implied by > such set of actions. > I was actually talking about contracting a helicopter ride which seems easier :-) The set of actions available to an AI is limited to the way it models the world. Without a capacity for intelligently expanding its world model, no AI is going to do anything outside of the domain it is defined in. Google Car won't ever think to contract a helicopter ride until either A) Google engineers program it to consider that as an option or B) Google engineers give the Car the ability to start modelling the world on its own terms. If B then it could be a long time before the Car discovers what a helicopter is, what it's capable of, how it could procure one, etc. The helicopter example is a bad one actually because it's a solution you or me can easily conceive of, so it seems mundane or easy. Terren -- You received this message because you are subscribed to the Google Groups "Everything List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/everything-list. For more options, visit https://groups.google.com/d/optout.
