M$ has done great stuff applying deep learning to speech-to-text, probably that's the NLP stuff they're referring to...
On Tue, Jul 15, 2014 at 6:10 PM, Chris Nolan via AGI <[email protected]> wrote: > It's interesting that they say it's been applied to NLP but provide no > description on results, which makes one think it didn't do as well as a NL > classifier as it did for the image classifying application. I wonder if > that's a result of training size, or there's something more inherent in > undering NL that just makes it much harder, and that DNNs won't do as well > on in comparison to Image classification? That later point is interesting > for AGI, because it implies that there's still a larger ML paradigm that's > needed for NL, before we'll see real breakthroughs. > > They also talk about these big improvements in accuracy, but give no > figures. It'd be nice to know what it's relative to. Guess we have to wait > for the paper... > > -Chris > > > On Tuesday, July 15, 2014 6:07 AM, Jim Bromer via AGI <[email protected]> > wrote: > > > It is scalable they say. OK, but in how many cognitive categories? Are the > neural networks able to use that information for combination and selection > at a higher cognitive order? I am asking if they are able to use their > capabilities in a creative and imaginative application of knowledge. If you > could get a neural network to detect imagination then you might be able to > train it to use directed imagination in problem solving. But you can't > because imagination is too broad a concept. > > Jim Bromer > > > On Tue, Jul 15, 2014 at 5:51 AM, Jim Bromer <[email protected]> wrote: > > It is scalable they say. OK, but in how many cognitive dimensions? > (Throwing a term like "cognitive dimensions" out in front of you guys may > be a mistake.) Perhaps I should have asked, in how many cognitive > categories? This has been a persistent problem with neural networks. The > point of view that Chilimbi seems to be taking is that the [parallel] > system is able to detect different features and combine them somehow. > However, that does not mean, for example, that the neural networks are able > to use that information for combination and selection at a higher cognitive > order. Just because I cannot word my criticism in just the right way does > not mean that the criticism is completely invalid. A counter argument is > that since the algorithms are effectively making some kinds of combinations > and some kinds of selections then that proves that they are theoretically > capable of making the kinds of subsequent combinations and selections that > I am thinking of. However, this (kind of) argument still hasn't been > established experimentally. It's the same old same old. > > Jim Bromer > > > On Tue, Jul 15, 2014 at 12:22 AM, Ben Goertzel via AGI <[email protected]> > wrote: > > > ;) > > Hmm.. the quote about quantum physics is > > *** > “It’s like in quantum physics at the beginning of the 20th century,” > Chilimbi says. “The experimentalists and practitioners were ahead of the > theoreticians. They couldn’t explain the results. We appear to be at a > similar stage with DNNs. We’re realizing the power and the capabilities, > but we still don’t understand the fundamentals of exactly how they work. " > *** > > It is true that with deep learning, practice is way ahead of theory these > days. OTOH, quantum mechanics has some mysterious confusing-ness too it, > which deep learning doesn't really have.... What modern DL algorithms are > doing has made sense conceptually to a lot of people for a long time, and > the recent breakthrough results are not based so much on new conceptual > breakthroughs, as on the new availability of massive amounts of hardware > and associated operational software.... > > Still, "solving object recognition via DL" is a fair bit more AGI-ish than > "solving chess via alpha-beta pruning", for example These DL > architectures have a strong conceptual resemblance to one aspect of the > human mind-brain (parts of our visual and auditory cortices), whereas > alpha-beta pruning has a significantly lesser resemblance to stuff that the > human mind does (the human mind does some search like that, but on a far > lesser scale and only in tight interaction with a lot of other > processes).... > > I think these modern DL architectures bear moderately close resemblance > to something that could serve as a significant component of an AGI system. > In their details, though, they are not architected for integration with > other AGI components, unlike how the human visual and auditory cortices, > with their pronounced hierarchical sensory pattern recognition > functionality, are architected (well, evolved) for interaction with the > rest of the brain... > > -- Ben > > > On Mon, Jul 14, 2014 at 9:04 PM, David Hart <[email protected]> wrote: > > I skimmed it. Is it a joke? DNN like quantum physics? Really? Do they > really think the two are in the same league? > > -dave > > > On Tue, Jul 15, 2014 at 11:52 AM, Ben Goertzel <[email protected]> wrote: > > > > This time from Microsoft... > > > > http://research.microsoft.com/en-us/news/features/dnnvision-071414.aspx > > > > -- > > Ben Goertzel, PhD > > http://goertzel.org > > > > "In an insane world, the sane man must appear to be insane". -- Capt. > James > > T. Kirk > > > > "Emancipate yourself from mental slavery / None but ourselves can free > our > > minds" -- Robert Nesta Marley > > > > -- > David Hart | +852 9559 9580 > OpenCog Foundation | http://opencog.org > > > > > -- > Ben Goertzel, PhD > http://goertzel.org > > "In an insane world, the sane man must appear to be insane". -- Capt. > James T. Kirk > > "Emancipate yourself from mental slavery / None but ourselves can free our > minds" -- Robert Nesta Marley > *AGI* | Archives <https://www.listbox.com/member/archive/303/=now> > <https://www.listbox.com/member/archive/rss/303/24379807-f5817f28> | > Modify <https://www.listbox.com/member/?&;> Your Subscription > <http://www.listbox.com/> > > > > *AGI* | Archives <https://www.listbox.com/member/archive/303/=now> > <https://www.listbox.com/member/archive/rss/303/20347893-f72b365c> | > Modify <https://www.listbox.com/member/?&;> Your Subscription > <http://www.listbox.com/> > > > *AGI* | Archives <https://www.listbox.com/member/archive/303/=now> > <https://www.listbox.com/member/archive/rss/303/212726-deec6279> | Modify > <https://www.listbox.com/member/?&;> > Your Subscription <http://www.listbox.com> > -- Ben Goertzel, PhD http://goertzel.org "In an insane world, the sane man must appear to be insane". -- Capt. James T. Kirk "Emancipate yourself from mental slavery / None but ourselves can free our minds" -- Robert Nesta Marley ------------------------------------------- AGI Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-f452e424 Modify Your Subscription: https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-58d57657 Powered by Listbox: http://www.listbox.com
