Robert, I'm afraid I haven't followed this list closely, so I don't know Prof Honeycutt or what your discussions might have been.
To be fair, I don't think theoretical consensus comes easily. There are people reaching for one. Most recently we have had Geoff Hinton asking for a reboot and offering capsule networks. Gary Marcus is untiring pushing for a more psychological approach. Any number of people are trying to rehabilitate GOFAI and Bayes from before the deep learning boom. There are those pushing embodiment, artificial organisms, people working up from neuroscience. I'm only smug because I see a solution. What's good is that I now discover OpenCog has moved in the same direction. Some years ago they were more committed to symbolism, at least as a short cut. Another shortcut, then symbolism, now deep learning? But the pdf Linas Vepstas posted, for all his having a PhD in theoretical particle physics and being unable to understand anything said to him (sorry Linas, I couldn't resist. I do know the frustration of feeling people are failing to understand or think clearly), actually scored a couple of home runs with me. He has identified linearities in vector models as a key weakness, resolved the problem as one of reassembling parts (jigsaw pieces?), and even dropped a mention of category theory, which speaks to formal incompleteness. (Though I'm not sure he's thought that through, because it actually becomes an argument for distributed representation and against symbolism, against any fixed symbolism, anyway.) Formal incompleteness might actually be the biggest of those from a theoretical point of view. It can be a kind of theoretical unifying factor, reaching down even to deep learning, then extending it. I don't think Linas would see it that way. But that makes it even more a situation which is ripe to have more eyes on it, to examine exactly what it means. So I see a short distance, and concrete changes which could be made, to make a big difference. I'm criticizing the status quo, definitely the deep learning work which sparked this thread, but also even more what I now learn OpenCog has come to, not because it is bad, but because I see the solution as very close, and I want to span that last gap. -Rob P. S. Linas, I see you have posted more documentation. As a first impression I think I'm going to suggest you simplify the way your "atoms" connect, very, very simply, and ring it like a bell to find sheaves among them simply too. It may just not work yet because serial hardware will be too slow. But I'll look in more detail. -Rob On Thu, Feb 21, 2019 at 6:34 PM Nanograte Knowledge Technologies < [email protected]> wrote: > Rob > > So much of what you're saying makes so much sense, that it's almost scary. > The years of theoretical discussions I had with Prof Honeycutt bears out > what you're saying. Why I constantly say it remains an issue of design, is > exactly what I see in your communications. It is because if one designs and > implements, one simply becomes stuck with that design and trying to make it > work. Very few social groups are willing to throw away a design that is > clearly inappropriate - even after a few years' of investment - to use that > learning for a radical redesign. Karl Mannheim referred to such elitist > practices as being rooted in Ideological and Utopian thinking. Clearly, a > society of developers need new ways of thinking about AGI. > > My contention is; to develop AGI, one must first become AGI. To my mind, > the artifact of such a transformation would be represented by an AGI > blueprint. A strong truth is often most unpopular. I know my perspective is > disturbing a number of staid veterans in the field, but it does not > conclude that what I'm saying has no relevance. > > Rob, I'd like to venture to say that even what you're discussing, I have > theoretically researched beyond with my mentor. There are a number of > levels still above what you propose, which we have scant theory for. > However, you are already thinking about what the integration and outcomes > of such integration would mean. Such is systems thinking. > > I'm grateful for those who are diligently clawing away - with seemingly > bare hands at times - at the AGI coalface, but I think it would've been > much better had "we" first agreed on a the continuous development approach > of an appropriate toolkit. There remains a chronic absence in consensus and > I see no end to it. A failure of society to organize? > > One cannot use methodology, which has proven to be limited (eventually) to > try to address futuristic solutions that we have to still develop "new" > theory for. As such, I propose, at first, a radical redesign, supported by > a pragmatic, next-step approach. We need case-based successes to learn > from. > > In its absence, a number of us would simply follow our own idea of what an > AGI system would become. > > Last, a critical thought. I have no idea why anyone would spend a useful > life on building submarines, when those have already been perfected and are > not what is needed in the world. To my mind, that is just playing it safe. > What it is not, is assuming industry leadership and stepping out to define > what AGI should become. To do so takes very-specific personality, and > character. It requires a historical bigness, not petty nit picking and > ridicule. This, I'm pointing to at our learned friend (and similar others > who I have encountered here) who seem to think no one else in this whole, > wide world has much use to contribute to their version(s) of AGI. I think > they're sorely mistaken, but only time would tell. > > Rgds > > Robert Benjamin > > > > ------------------------------ > *From:* Rob Freeman <[email protected]> > *Sent:* Thursday, 21 February 2019 12:38 AM > *To:* AGI > *Subject:* Re: [agi] openAI's AI advances and PR stunt... > > OK, that makes sense Ben. So long as you have a clear picture of how to > progress the theory beyond temporary expediency, temporarily using the > state-of-the-art may be strategic. > > So long as you are moving forward with some strong theoretical candidates > too. If we get trapped without theory, we're blind. There are too few > people with any broad theoretical vision for how to move forward. Too many > script kiddies just tweaking blindly, viz, the "important step" this thread > began with. > > I'm encouraged that it now appears you are deconstructing grammar and > resolving it to a raw network level. That Linas is seeing the relevance of > maths like category theory, which is motivated by formal incompleteness, > speaks to this realization. (Though he may not be aware of the full import.) > > Deep learning does not realize this. It does not realize that formal > description above the network level will be incomplete. I'm sure that is > the key theoretical failure holding it back. I wish there were more people > talking about it. If deep learning realized this they wouldn't still be > trying to "learn" representations, whether in intermediate layers or other. > (What was that article recently about the representation "bottle neck" idea > in deep learning needing to be revised?) > > It's actually ironic that deep learning does not realize this idea that > formal description (above the network) must always be incomplete, because > it is also the key to the success of deep learning! The whole success of > distributed representation is due to this. The field moved to distributed > representation blindly, without theory, just because things started working > better that way! But you still see articles where people say no-one knows > why distributed representation works better! The failure of theoretical > vision is extraordinary. > > But if you've deconstructed your dictionaries (throwing out your hand > coded dictionaries?) and arrived back at the level of observation in a > sequence network. And done it because of the theoretical realization that > complete representation above the network level is impossible (or was it > just an accident, trying to deconstruct symbolism to connectionism, and > then accidentally noticing the relevance to variational theories of maths?) > Then your group would be the only ones I've come across who have done (I > think the Oxford thread of variational formalization, around Coecke et al. > Grefenstette, were also seduced away by the short term effectiveness of > deep learning on GPUs.) > > We need to keep (or get!) the theoretical vision. > > Even given a vision of formal incompleteness, you (and Pissanetzky?) may > still be lacking a totally clear conception that the key problem is > assembling elements in new ways all the time. > > Still, some focus on assembling elements in different ways (from a > sequence network) is encouraging. There is scope to move forward. > > As a concrete, immediate, idea to explore moving forward, I hope you'll > look at the idea of using oscillations to structure your sequence network > representations. For it to be meaningful your networks will need to be > connected in ways which directly reflect the ideas behind embedding vectors > (without their linearities.) I don't know if that is true for your > networks. But given that, implementation should be simple, if practically > slow without parallel hardware. > > -Rob > > On Thu, Feb 21, 2019 at 12:03 AM Ben Goertzel <[email protected]> wrote: > > It's not that it's hard to feed data into OpenCog, whose > representation capability is very flexible > > It's simply that deep NNs running on multi-GPU clusters can process > massive amounts of text very very fast, and OpenCog's processing is > much slower than that currently... > > *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + delivery > options <https://agi.topicbox.com/groups/agi/subscription> Permalink > <https://agi.topicbox.com/groups/agi/T581199cf280badd7-M630f3e9428c378ac25185812> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T581199cf280badd7-M2dceba07a96e44cdbca6ecd0 Delivery options: https://agi.topicbox.com/groups/agi/subscription
