No problem ;) One other autoencoder architecture you might find interesting is Yann Lecun's "deep belief network": http://yann.lecun.com/exdb/publis/pdf/ranzato-nips-07.pdf (his most recent publication).
Deep belief network's are basically stacked feedforward autoencoders, learned with backprop, with a sparse coding mechanism on top. Yann Lecun's networks are based on traditional feed-forward neural nets, and in general much faster to learn then boltzmann machines. I agree with you that this idea of autoencoders / deep belief networks could be interesting for AGI, since they provide a natural way of automatically finding compact, usefull representations of otherwise very obscure data such as vision or speech. In the above paper, some pretty impressive results are published in the context of general vision. Currently LeCun's architecture is the best (simplest) solution for general object recognition... Durk On Thu, Mar 6, 2008 at 5:13 PM, Ed Porter <[EMAIL PROTECTED]> wrote: > Durk, > > I am indebted to you for bringing this very interesting Hinton lecture to > the attention of this list. > > It is highly relevant to AGI, since, if it is to be believed, it provides > a > general architecture for learning invariant hierarchical representations > (which are currently in vogue--for good reason), from presumably any type > of > data. It can perform both unsupervised and supervised learning. Hinton > claims this architecture scales well. He does not mention how his system > would learn temporal patterns, but presumably it could be expanded to do > so, > such as by the use of temporal buffers to store sequences of inputs over > time. If it could learn temporal patterns it would seem to be able to > generate behaviors as well as recognizing and generating patterns. > > Of course it would require considerably more to become a full AGI, such as > motivational, reinforcement-learning-like, mental behavior, goal > selecting, > goal pursuing, and novel pattern formation features. But it would seem to > provide a system for automatically learning and generating a significant > percent of the patterns and behaviors an AGI would need. > > I think the AGI community should be open to adopting such a potentially > powerful idea from machine learning, if it is shown to be as powerful as > Hinton says, because, if so, it would add credence to the possibility of > AGI > by making the task of building an AGI seem considerably less complex. > > Ed Porter > > -----Original Message----- > From: Kingma, D.P. [mailto:[EMAIL PROTECTED] > Sent: Sunday, March 02, 2008 12:08 PM > To: [email protected] > Subject: [agi] interesting Google Tech Talk about Neural > Nets > > Gentlemen, > For guys interested in vision, neural nets and the like, > there's a very interesting talk by Geoffrey Hinton about unsupervised > learning of low-dimensional codes: > It's been on Youtube since December, but somehow it escaped > my attention for some months. > > http://www.youtube.com/watch?v=AyzOUbkUf3M > > BTW, the back of Peter Norvig's head makes a guest > appearance throughout most of the video ;) > > As an academic I'm quite excited about this technique > because it has the potential of solving non-trivial parts of problems in > perception in a clean, practical, understandable way. > > Greets from Utrecht, Netherlands, > Durk > > agi | Archives <http://www.listbox.com/member/archive/303/=now> > <http://www.listbox.com/member/archive/rss/303/> | Modify > < > http://www.listbox.com/member/?&; > > > Your Subscription <http://www.listbox.com> > > ------------------------------------------- agi Archives: http://www.listbox.com/member/archive/303/=now RSS Feed: http://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: http://www.listbox.com/member/?member_id=8660244&id_secret=98558129-0bdb63 Powered by Listbox: http://www.listbox.com
