Sorry, Ben, I wrote most of my last post several days ago, before you emailed me more detailed issues for me to address. So I will try to address these new issues quickly now. Ben>>>>>>> So there is no real plan for how to achieve abstract symbolic reasoning as needed for human level general intelligence within a purely formal-NN type approach Ed>> Deep neural nets do create abstract symbols, in the form of neural codes for high level invariant concepts. Hassabis’s discusses ways to get more powerful symbolic reasoning and behavior with the use of neural generative models. He also suggests that the system will be more capable a developing the types of symbolic generalization that are useful for many types of learning, reasoning, and behavior models. He does so by suggesting that neural nets be more capable of learning generalizations that are combinations of concepts at many different levels in the hierarchical representation and which span pattern elements across different patches of the cortex and across not only the cortex, but also subcortical nets. He also talks about the use of episodic memory nets to increase the power of deep neural net learning and behaving.
Plus, Shruti has already shown how to do symbolic reasoning with neural nets. Ben>>>>>>> Obviously in opencog we are taking more of a symbolic-neural approach so we don't have issues with abstraction Ed>> That’s great if they can also do deep learning well, or can intelligently interface with nets created by such learning. Ben>>>>>>> Also if you look at the recent Markram et al paper on algebraic topology and mesoscopic brain structure, there is nothing in the Hassabis etc. universe that seems to address how such structures would be learned or would emerge Ed>> “Algebraic topology and mesoscopic brain structure”, wow, sounds cool. I will have to check it out. Thanks for the pointer. Ben>>>>>>> But sure in a big-picture historical sense the progress happening these days on "narrow AI verging toward AGI" and on "making complex cognitive architectures finally do stuff" is super exciting. We are on the verge of multiple breakthroughs within the next few years. Woo hoo !! Ed>>>Woo hoo, Dooooooooooooooooooooooooode!!! If I had what Hassabis has behind him, and if I could persuade a major company to put a billion dollars a year behind me and help me recruit talent, I think I could make a roughly human level brain in six years. It wouldn’t yet be as good as a human at everything, but it would be hundreds to billions of times faster at other perceptual, cognitive, and behavior tasks, meaning it could do many intellectual tasks better and faster than a hundred people. But Hassabis could do it a lot quicker and better than I can, and he’s already plugged in to an impressive team helping him to do it. Ed Porter On Mon, Jul 31, 2017 at 1:18 PM, EdFromNH . <[email protected]> wrote: > Ben, > > Thanks for sending a copy of your interesting article. I have read it > twice and am still thinking about it. Here are a few brief thoughts. > > I don't think variable binding is that big a problem for neural nets for, > at least, the following reasons. > > 1--Shruti by Shastri, as you acknowledge, shows how variable binding can > be represented by synchronies in neural nets for logic-like functions. > > 2-- I agree with you that we currently don't understand how Shruti-like > mechanism could allow a neural net operating at the frequency of the > brain's gama waves to provide all the detail of binding we sense in > conscious experience. Perhaps substantially more complex forms of > synchronous timing could be used. Electronic brains could have > frequencies hundreds of time higher, than biological brains, meaning it > could have hundreds of time more Shruiti-like binding. > > 3--It is believed by many the prefrontal cortex has the ability to store > variable sequences of the activation states of mid and high level > representational concepts, which map into recent hierarchical activation > patterns in the cortex's generalizational and compositional hierachies. > These activation patterns stored in the hippocampus contain bindings of low > level instantiations of higher level activated concepts. This functions > like a binding, because it stores temporal relationships between concepts > and their instantiation, and their associations in episodic perceptual > state. This is a very complex and powerful form of binding. > > 4-- In electronic neural nets it's easy to use variables to set weights > in the net, change net architecture, or feed variables into pipes -- such > as in LSTM memory or neural turing machines. > > 5-- Lastly, neural net artificial brains will be able to interface to a > lot of traditional computing hardware and software, and to take advantage > of the powerful forms of binding that can be executed efficiently by > traditional computing. > > > I don't understand combinatorial logic well enough to be certain I > understood your point about it. To me it seems like you were talking > about something similar to functional programming (about which I know not > much, but more). As I understand your argument, there ARE things acting > somewhat like variables in the combinatorial logic you describe. They are > what is passed along the one or more pipes connected into each of a network > of transformations (e.g.,functions). The signal passing along each pipe > would be a variable bound to the function it is piped into. This is > somewhat similar to the binding of low level instantiation of high level > concepts I mentioned above under "3". Of course, in the brain the neural > net would be recurrent, making things more complex, but presumably > functional programming can have recurrencies. > > Net-net, your article doesn't make me think any less of Hassabis's paper > -- or any less of the probability we are close to near-human-level AGI. He > provides promising approaches for solving every problem he cites in the > paper. Even I have ideas, based on brain science, of how to address each > of them. My optimism is increased by the understanding that most of the > problems he suggest are interrelated, and progress on almost any one will > help progress on the other. This means that the rate of their collective > progress will tend to grow at an exponential of the average rate of > progress on each of them alone, if separated from the benefit of the > advances on others.. > > My optimism for DeepMind is increased by the facts that: Demis is a rare > genius, who has already shown unusual success in multiple intellectual > chalenges: he has hundreds of world-class people working for him; his > project is backed by the resources of one of the world's largest, richest, > and most advanced computing companies; and by the fact his papers show he > is properly focusing on some of the most important capabilities required to > make artificial brains. > > Ben, you once told me, that when you tried to build your webmind AI near > the end of the dot.com boom, that one of the biggest problems you faced > was tuning the combinatorial parameter space required to get a complex > semantic network to work well. In my mind one of the major problem > standing between humanity and powerful AGI is the amount of experimentation > required to tune brain architectures, and their parameters. But even with > far suboptimal tuning, the archtectural changes Hassabis proposes could be > very commercially valuable -- even if that are substantially sub-human in > many respects. > > Google and every other company that wants to be a major player in AGI, and > has the money to do so, should spend billions of dollars following > Hassabis's general roadmap. It would be a very wise decision for any > company that could get the talent to do so. > > Ed Porter > > On Thu, Jul 27, 2017 at 8:50 PM, Mike Archbold <[email protected]> > wrote: > >> I read the article (but not yet the paper) and, no disrespect to the >> researchers, it sounds like recycled arguments and suggestions from >> the last 5,000 posts I've read about AGI. As a side note, the kind of >> issues we used to talk about, formerly considered crackpot, are now >> the basis of runaway hype. People I know are convinced that strong AI >> is imminent. I don't think the field of AGI is hyped, by the way. >> It's more like the media and general climate of opinion. >> >> On 7/27/17, Ben Goertzel <[email protected]> wrote: >> > http://goertzel.org/Neural_Foundations_Symbolic_Thought.pdf >> > >> > On Thu, Jul 27, 2017 at 9:54 PM, EdFromNH . <[email protected]> wrote: >> > >> >> Ben, could you please send me a free author's copy of the paper at >> >> http://ieeexplore.ieee.org/document/6889662/ . Ed Porter >> >> >> >> On Thu, Jul 27, 2017 at 12:44 AM, Nanograte Knowledge Technologies < >> >> [email protected]> wrote: >> >> >> >>> Ben >> >>> >> >>> Conceptually, I like where you are going with this. Your team's work >> >>> with >> >>> human-language-based robotic communication is astounding. >> >>> >> >>> I think your idea of a universal attractor has merit. I suppose, in >> the >> >>> end, when matter exists, it generates an elcetro-magnetic field. In a >> >>> genetic sense, the flux of such a field would act as an open and >> >>> closed-loop communications network. In this sense, the relevant data, >> >>> information, and a relative perspective of knowledge, would all be >> >>> packaged >> >>> within relative, genomic code. In other words, we are imagining a >> >>> relative >> >>> system of relative systems from which reality would functionally >> emerge. >> >>> >> >>> Given my systems methodology, what remains to be done in order to >> >>> visualize a model of human-like machine reasoning, is to be able to >> link >> >>> your "attractor" value to the information, from which it should become >> >>> possible to systematically emerge any informational concept at any >> level >> >>> of >> >>> abstraction within any, dimension of reasoning. The genetics of >> >>> resultant >> >>> information would in theory make forward and backchaining possible, >> and >> >>> much more. >> >>> >> >>> The completeness schema of functional, attractor values seems to be a >> >>> critical machine-reasoning component to pursue. It would probably also >> >>> assume the role of a priority systems constraint. I've been doing much >> >>> thinking about this as a next-step for my own research. >> >>> >> >>> I think you've got this. Keep up the great work. >> >>> >> >>> Rob >> >>> >> >>> ------------------------------ >> >>> *From:* Ben Goertzel <[email protected]> >> >>> *Sent:* 27 July 2017 04:57 AM >> >>> *To:* AGI >> >>> *Subject:* Re: [agi] Neuroscience-Inspired AI >> >>> >> >>> >> >>> Well I would say that none of the work done at Deep Mind and also none >> >>> of >> >>> the ideas in Demis etc.'s paper address the questions I raised in this >> >>> paper >> >>> >> >>> http://ieeexplore.ieee.org/document/6889662/ >> >>> How might the brain represent complex symbolic knowledge? - IEEE >> Xplore >> >>> Document <http://ieeexplore.ieee.org/document/6889662/> >> >>> ieeexplore.ieee.org >> >>> A novel category of theories is proposed, providing a potential >> >>> explanation for the representation of complex knowledge in the human >> >>> (and, >> >>> more generally, >> >>> >> >>> >> >>> (sorry for the paywall ... use sci-hub.cc ...) >> >>> >> >>> So there is no real plan for how to achieve abstract symbolic >> reasoning >> >>> as needed for human level general intelligence within a purely >> formal-NN >> >>> type approach >> >>> >> >>> >> >>> Obviously in opencog we are taking more of a symbolic-neural approach >> so >> >>> we don't have issues with abstraction >> >>> >> >>> Also if you look at the recent Markram et al paper on algebraic >> topology >> >>> and mesoscopic brain structure, there is nothing in the Hassabis etc. >> >>> universe that seems to address how such structures would be learned or >> >>> would emerge >> >>> >> >>> >> >>> >> >>> But sure in a big-picture historical sense the progress happening >> these >> >>> days on "narrow AI verging toward AGI" and on "making complex >> cognitive >> >>> architectures finally do stuff" is super exciting. We are on the >> verge >> >>> of >> >>> multiple breakthroughs within the next few years. Woo hoo !! >> >>> >> >>> - -Ben >> >>> >> >>> >> >>> On Thu, Jul 27, 2017 at 5:55 AM, EdFromNH . <[email protected]> >> wrote: >> >>> >> >>>> About the above linked Hassabis paper, Ben said, "It's sort of a high >> >>>> level inspirational paper... it does lay down pretty clearly what >> sort >> >>>> of >> >>>> thinking and approach Deep Mind is likely to be taking in the next >> >>>> years >> >>>> ... there are no big surprises here though as this has been Demis's >> >>>> approach, bias and interest all along, right?" >> >>>> >> >>>> From my knowledge of several articles and videos by, or about, >> Hassabis >> >>>> -- >> >>>> I totally agree. But I am a little less ho-hum than Ben, perhaps >> >>>> because >> >>>> I'm not as up on the current state of AGI as Ben. >> >>>> >> >>>> Reading Hassabis's paper makes me bullish about how close we are to >> >>>> powerful, if not fully human-level AGI, within 5 years. >> >>>> >> >>>> Why? Because all of the unsolved challenges Hassabis discusses seem >> >>>> like they could be easily solved if enough engineering and >> programming >> >>>> talent was thrown at them. I feel like I could relatively easily >> >>>> -- within a few months -- weave plausible high level architectural >> >>>> descriptions for solving all of these problems, as, presumably, >> people >> >>>> like Demis and Ben could do even better. (Perhaps that is why Ben is >> so >> >>>> ho-hum about the paper.) With the money that's being thrown into >> AGI, >> >>>> and >> >>>> the much greater ease of doing cognitive architectural experiments >> made >> >>>> possible with Neural Turing Machines -- which allow programmable, >> >>>> modular >> >>>> plug-and-play with pre-designed and pre-trained neural net modules -- >> >>>> the >> >>>> world is going to get weird fast. >> >>>> >> >>>> Tell me why I am wrong. >> >>>> >> >>>> On Sun, Jul 23, 2017 at 8:29 PM, Ed Pell <[email protected]> >> wrote: >> >>>> >> >>>>> https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5467749/ >> >>>>> >> >>>>> >> >>>>> On 7/23/2017 4:18 PM, Giacomo Spigler wrote: >> >>>>> >> >>>>>> >> >>>>>> An Approximation of the Error Backpropagation >> >>>>>> Algorithm in a Predictive Coding Network >> >>>>>> with Local Hebbian Synaptic Plasticity >> >>>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> ------------------------------------------- >> >>>>> AGI >> >>>>> Archives: https://www.listbox.com/member/archive/303/=now >> >>>>> RSS Feed: https://www.listbox.com/member >> /archive/rss/303/8630185-a57a7 >> >>>>> 4e1 >> >>>>> Modify Your Subscription: https://www.listbox.com/member/?&; >> >>>>> Powered by Listbox: http://www.listbox.com >> >>>>> >> >>>> >> >>>> *AGI* | Archives <https://www.listbox.com/member/archive/303/=now> >> >>>> <https://www.listbox.com/member/archive/rss/303/19237892-5029d625> | >> >>>> Modify <https://www.listbox.com/member/?&;> Your Subscription >> >>>> <http://www.listbox.com> >> >>>> >> >>> >> >>> >> >>> >> >>> -- >> >>> Ben Goertzel, PhD >> >>> http://goertzel.org >> >>> >> >>> "I am God! I am nothing, I'm play, I am freedom, I am life. I am the >> >>> boundary, I am the peak." -- Alexander Scriabin >> >>> *AGI* | Archives <https://www.listbox.com/member/archive/303/=now> >> >>> <https://www.listbox.com/member/archive/rss/303/26941503-0abb15dc> | >> >>> Modify <https://www.listbox.com/member/?&;> Your Subscription >> >>> <http://www.listbox.com> >> >>> >> >>> *AGI* | Archives <https://www.listbox.com/member/archive/303/=now> >> >>> <https://www.listbox.com/member/archive/rss/303/8630185-a57a74e1> | >> >>> Modify <https://www.listbox.com/member/?&;> Your Subscription >> >>> <http://www.listbox.com> >> >>> >> >> >> >> *AGI* | Archives <https://www.listbox.com/member/archive/303/=now> >> >> <https://www.listbox.com/member/archive/rss/303/19237892-5029d625> | >> >> Modify >> >> <https://www.listbox.com/member/?&;> >> >> Your Subscription <http://www.listbox.com> >> >> >> > >> > >> > >> > -- >> > Ben Goertzel, PhD >> > http://goertzel.org >> > >> > "I am God! I am nothing, I'm play, I am freedom, I am life. I am the >> > boundary, I am the peak." -- Alexander Scriabin >> > >> > >> > >> > ------------------------------------------- >> > AGI >> > Archives: https://www.listbox.com/member/archive/303/=now >> > RSS Feed: https://www.listbox.com/member/archive/rss/303/11943661- >> d9279dae >> > Modify Your Subscription: >> > https://www.listbox.com/member/?&; >> > Powered by Listbox: http://www.listbox.com >> > >> >> >> ------------------------------------------- >> AGI >> Archives: https://www.listbox.com/member/archive/303/=now >> RSS Feed: https://www.listbox.com/member/archive/rss/303/8630185-a57a74e1 >> Modify Your Subscription: https://www.listbox.com/member >> /?& >> Powered by Listbox: http://www.listbox.com >> > > ------------------------------------------- AGI Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-f452e424 Modify Your Subscription: https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-58d57657 Powered by Listbox: http://www.listbox.com
