On Thu, Aug 28, 2014 at 10:05 PM, Terren Suydam <terren.suy...@gmail.com> wrote:
> > On Thu, Aug 28, 2014 at 7:30 AM, Telmo Menezes <te...@telmomenezes.com> > wrote: > >> >> Although my POV is aligned with the latter intuition, I actually agree >>> with the former, but consider the kinds of threats involved to be bounded >>> in ways we can in principle control. Although in practice it is possible >>> for them to do damage so quickly we can't prevent it. >>> >>> Perhaps my idea of intelligence is too limited. I am assuming that >>> something capable of being a real threat will be able to generate its own >>> ontologies, creatively model them in ways that build on and relate to >>> existing ontologies, simulate and test those new models, etc., generate >>> value judgments using these new models with respect to overarching utility >>> function(s). It is suspiciously similar to human intelligence. >>> >> >> I wonder. What you describe seems like the way of thinking of a person >> trained in the scientific method (a very recent discovery in human >> history). Is this raw human intelligence? I suspect raw human intelligence >> is more like a kludge. It is possible to create rickety structures of order >> on top of that kludge, by a process we call "education". >> >> > > I don't mean to imply formal learning at all. I think this even applies to > any animal that dreams during sleep (say). Modeling the world is a very > basic function of the brain, even if the process and result is a kludge. > With language and the ability to articulate models, humans can get very > good indeed at making them precise and building structures, rickity or > otherwise, upon the basic kludginess you're talking about. > > >> I think something like this could do a lot of damage very quickly, but by >>> accident... in a similar way perhaps to the occasional meltdowns caused by >>> the collective behaviors of micro-second market-making algorithms. >>> >> >> Another example is big societies designed by humans. >> > > Big societies act much more slowly. But they are their own organisms, we > don't design them anymore than our cells design us. We are not really that > good at seeing how they operate, for the same reason we find it hard to > perceive how a cloud changes through time. > > >> >> >>> I find it exceedingly unlikely that an AGI will spontaneously emerge >>> from a self-mutating process like you describe. Again, if this kind of >>> thing were likely, or at least not extremely unlikely, I think it suggests >>> that AGI is a lot simpler than it really is. >>> >> >> This is tricky. The Kolmogorov complexity of AGI could be relatively low >> -- maybe it can be expressed in 1000 lines of lisp. But the set of programs >> expressible in 1000 lines of lisp includes some really crazy, >> counter-intuitive stuff (e.g. the universal dovetailer). Genetic >> programming has been shown to be able to discover relatively short >> solutions that are better than anything a human could come up with, due to >> counter-intuitiveness. >> > > I suppose it is possible and maybe my estimate of how likely it is is too > low. All the same I would be rather shocked if AGI could be implemented in > 1000 lines of code. And no cheating - each line has to be less than 80 > chars ;-) Bonus points if you can do it in Arnold > <https://github.com/lhartikk/ArnoldC>. > Arnold is excellent! :) I raise you Piet: http://www.dangermouse.net/esoteric/piet.html > > T > > -- > You received this message because you are subscribed to the Google Groups > "Everything List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to everything-list+unsubscr...@googlegroups.com. > To post to this group, send email to everything-list@googlegroups.com. > Visit this group at http://groups.google.com/group/everything-list. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Everything List" group. To unsubscribe from this group and stop receiving emails from it, send an email to everything-list+unsubscr...@googlegroups.com. To post to this group, send email to everything-list@googlegroups.com. Visit this group at http://groups.google.com/group/everything-list. For more options, visit https://groups.google.com/d/optout.
