On 2012-05-15, at 3:23 PM, Andreas Mueller <[email protected]> wrote:
> I am not sure if we want to support sparse data. I have no experience with > using MLPs on sparse data. > Could this be done efficiently? The weight vector would need to be > represented explicitly and densely, I guess. > > Any ideas? People can and do use neural nets with sparse inputs, dense-sparse products aren't usually too bad in my experience. Careful regularization and/or lots of data (a decent number of examples where each feature is non-zero) will be necessary to get good results, but this goes for basically any parametric model operating on sparse inputs. Aside: there was interesting work on autoencoder-based pre-training of MLPs with sparse (binary, I think) inputs done by my colleagues here in Montreal. They showed that in the reconstruction step, you can get away with reconstructing the non-zero entries in the original input and a small random sample of the zero entries, and it works just as well as doing the (much more expensive, when the input is high-dimensional) exhaustive reconstruction. Neat stuff. David ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
