Hideaki and all, I'm glad to hear that someone else in the community is considering this possibility!
I agree that bodily motor outputs are not the only thing that could be looked at, I only wonder if activities like keystrokes are not separated from EEG activity by so many degrees of abstraction that their effective analysis by the CLA is impossible. Given the problems already associated with the EEG's noisy inputs and resolution, I was even wondering if this would be the case for direct spatial/kinematic data, and played around with the idea predicting a range of EKG information instead, since hopefully there would be a more direct relationship to be observed. What we know for sure is that at some level, the concept works. I think we've all seen videos of more invasive systems (direct brain monitoring) being used to control robot arms, it's just a question of whether the system can be made feasible for general consumers. I've heard of Emotiv, and their research kit might be a good platform to experiment with. Another contender is OpenBCI, although in that case a standard 8-channel EEG cap would be needed, and no software for things like concentration/key inputs would be provided (though since Emotiv's systems are proprietary, these might have been of limited usefulness anyway). And although the work Emotiv does is interesting, it doesn't really seem to have caught on as far as a consumer BCI is concerned. I wonder if the approach of trying to map out high-level computer functions like direct key inputs and mouse tracking are simply not responsive and intuitive enough to be truly useful, though maybe you can confirm or deny this based on your experiences with the EPOC. I've got a few ideas for making brain-computer interfaces more attractive in this regard, but it's not really related to NuPIC so discussion here doesn't seem appropriate. Feel free to e-mail me directly if you want to talk. Regards, Marcos On Wed, Jan 1, 2014 at 10:25 PM, Hideaki Suzuki <[email protected]> wrote: > Happy new year, community! > > Hi, Marcos > > Probably Yes. > I've been fascinated by EEG input to CLA since 2011 probably with the > mixture of other inputs, not only from body parts but also from keystrokes, > browser crawling activities, ...,etc. > > I'm afraid that CLA can associate such blurry? multi-stream inputs with > motor outputs very well. CLA's features (online learning, real-time, > special pooling, temporary pooling) seem a best match to me. There are a > number of prediction & abnormally detection methods, but I believe not so > many are online AND real-time. > > A hypothetical construction of CLA for that purpose could be: > - A CLA region receives EEG and other input streams and convert them to > an SDR at a specific time. > - Temporal changes of successive SDRs are stored in TP. > - Significant motor outputs (I could call "events") are associated with > related SDRs. > - TP predicts SDRs in the near future which indicates the corresponding > motor output with confidence probability. > > > > I believe that the best publicly available EEG scanner is Emotiv EPOC, > which can produce 14 channel data streams with every 8 ms (plus additional > head gyro data so it can detect when you tilt your head). > If SP and TP with enough size of regions can handle this amount/speed of > data stream in every 5ms with commodity hardware, I'm wondering we can play > a lot with our EEG. In my opinion the prediction speed is the key > differentiator to enable this game. > > Emotiv also provides several proprietary EEG-to-digital-data converters by > themselves; EmoKey to produce key input, Expressiv to get facial > expressions like blinking, Congnitiv to associate thoughts with commands, > and Affective to guess your feeling like sad or concentrated. The thing is > the user needs some training to use these software. i.e. the software rely > on the plasticity of your brain. I'm wondering if we can replace these > software with CLAs. Then, CLAs can learn the associations of inputs and > events for you, instead that your brain learns them for computers. ;-) > > Also, the conversion has some time gap probably .1 sec or so with the > provided software. I'm suspecting that the delay implies the capturing > window for the software to extract the meaningful digital data from EEG > wave patterns. This can be fatal if one wants to play an action game with > EEG. We need sub-frame response time (<16.7ms) to feed input data. Then, > there is no such capturing window for CLA: truly real-time. IIUC. > > BTW I have an EPOC scanner in my room and I like the product, so I'm not > saying the provided software is bad. I'm just talking about the possible > application of CLA. For clarity. > > --- > B.R. > Hideaki Suzuki > > > PS. > I've been messed up with daytime job since the last October which is over > now (very happy) > I hope I can enjoy this OSS activity this year. I'm now reading the ML > archive... > > _______________________________________________ > nupic mailing list > [email protected] > http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org > >
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
