Hi Ivan, I hope you don't interpret my comments as a criticism of you or your thoughts on this. Quite the opposite. I'm trying to help steer the focus towards a clearly shared understanding of what we're trying to achieve with this exercise. I have very little experience of other approaches, and rely heavily on reading up and having discussions with people who know far more about it than I do!
I was deliberately being a little harsh on traditional AI; it's more a cultural problem than anything, because the norm seems to be one of announcing "breakthroughs" which turn out to have no general applicability. That being said, I absolutely recognise their progress in understanding cognition which has come from 50 years of hard work by some very smart people, and I also appreciate the importance of having a computational model of cognition which we can use to describe the phenomena we're studying. >From talking with Francisco (on and off this list - see the recent planning video), it seems that the CEPT SDR's are indeed being generated using a similar approach to that used in the SP. The sparse bitmaps we're seeing are derived from a grayscale "image" which itself is a map of the semantic values associated with each word. It may be worth exploring how this process works in order to see how well it matches the needs of the CLA (this will be part of the feedback and improvement process I mentioned in the previous email). Again, we have no idea what this setup is going to do. We have some hope that it will achieve some kind of linguistic performance, but we'll most likely have to do some evolution on it before we're sure we're onto something. If you read Pinker's discussions on this, it seems that there are a number of areas (ie CLA's) in the brain which specialise in certain language function, and that damage to one area often leaves the other functions intact, while disrupting the connections between them seems to have more complicated consequences. All these areas are being fed some kind of semantic SDR's from other regions in the brain (see the beautiful images and animations on http://www.neuroscienceblueprint.nih.gov/connectome/ for examples of this plumbing). Regards, Fergal Byrne On Sun, Sep 15, 2013 at 6:37 PM, Ivan Sytenko <[email protected]> wrote: > Hi Fergal, > > Let me stand up for the AI. In my opinion a lot more done here than in NN. > For example, a hierarchical analysis of scenes based on models and the > corresponding inductive-deductive learning process and analysis. Much like > the HTM. Your concepts were born in AI. > > On the other hand, how far you've gone from summer in the modeling of a > neuron? > Your scheme of the brain in the same distance. > > Distance from the word "cat" to the word "dog" is defined by the evolution > of language. This distance corresponds to the semantic relations of cats > and dogs. When you add semantic information to the morphological level you > make those words much closer. You only change the pattern of the word, but > its meaning will be determined by the texts or other modalities. > > Sorry to bother you. I have some experience in NN. Now I've done a > first-order logic as an example NLP with fast learning. My NN fits into the > part of the cortex column. Of course, I followed the publications of Jeff > and I was looking at your experience with the columns. My opinion of CEPT > related to my study of its use in my NN. Method CEPT is well done, but it > should be inside the NN system. I just wanted to share that opinion. > > Regards, > Ivan > > > ON:Date: Sun, 15 Sep 2013 13:08:34 +0100 > From: Fergal Byrne <[email protected]> > To: "NuPIC general mailing list." <[email protected]> > Subject: Re: [nupic-dev] (no subject) > Message-ID: > < > cad2q5yex+jr28sbx9tutubzgnad1jhdpv8zr1oy8c31xu0o...@mail.gmail.com> > Content-Type: text/plain; charset="iso-8859-1" > > Hi Ivan, > > I'm not sure what you're driving at here, but let me just try to answer > your concerns. > > While a huge amount of (largely fruitless IMHO) work has been done in > classical AI research on NLP, this work has a fatal flaw. It is looking at > the problem as one which can be solved by combining ever more complex > programs in ever more complex ways. The only good thing it has achieved is > that it has asked a lot of questions about the structure of language, and > also allowed computers to analyse huge amounts of text in order to get good > statistical understandings of what we actually say. > > Loads of clever software cannot be the answer. Only one system in the > universe has been built which does NLP, and that system uses no programs. > It has some inbuilt structure, but it's essentially a set of simple CLA's > wired up in a particular way. > > So, what we're attempting to do here is to assess what happens when you > look at one (hypothetical) piece of this system. All regions in our > language systems communicate by passing SDR's up the hierarchy. So we've > got some of these from CEPT and we're going to do some experiments to see > can we observe anything similar to what we see in the brain. > > The classifiers we use are just implementation artefacts which allow us to > measure and observe the activation patterns and predictions that the CLA is > generating. These classifiers are like electrode arrays or fMRI pictures, > and are not part of the simulation. > > We aren't going to be writing classifiers to perform the tasks you listed - > in fact the opposite is the case. We plan to build hierarchies of CLA's > which perform those tasks, with only very simple (but massively parallel) > plumbing between them providing the "cleverness". The key to figuring out > how the brain achieves all these wonderful feats is to figure out how to > connect the regions and combine them together in the hierarchy. > > The research begins with this simple question: How do we (or just can we) > connect the CEPT SDR's, representing words, to a "word-level" region of > CLA, in order to produce some meaningful linguistic performance? > > The answer will be dependent on both a) whether the CEPT data is a good > analogue for how words are passed around in the brain, and b) whether we > can design a way to connect up the CEPT SDR's to the CLA which works. We > know neither of these for sure, but by varying the connection strategies, > we should be able to provide an answer to the former question. This should > allow us (if we fail) to improve the CEPT data and try again. > > It is possible that we're barking up the wrong tree with the CEPT SDR's. If > so, we should find out as soon as we can, and then see if a pivot will > yield fruit. Humans are doing something like this, so let's see if we can > generate the appropriate analogue with just a single region, or if we need > to add hierarchy to get there. > > Regards > > Fergal Byrne > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > > -- Fergal Byrne ExamSupport/StudyHub [email protected] http://www.examsupport.ie Dublin in Bits [email protected] http://www.inbits.com +353 83 4214179 Formerly of Adnet [email protected] http://www.adnet.ie
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
