Hi Issam.
Sorry to break this to you, but sklearn will not add any GPU code in the
near future.
Also, we will probably not use numba in quite a while.
I think it is possible that we want to replace some cython with numba,
but I don't see this happening this year.
For your proposal: actually Deep Boltzmann machines are strictly more
general than Deep Belief Networks.
If you don't know this, I'm not sure you know enough about these
algorithms to implement them.
Also, stacked denoising autoencoders are synonymous with deep
autoencoders (modulo modifying the input).
You write "Learn and implement GPU accelerated Python techniques (eg.
shared variables) to improve speed".
Are you talking about theano there? So you want to add a
theano-dependency to sklearn?
No, sorry, that won't happen either.
I don't think there is any point in submitting your proposal.
Sorry.
Andy
On 05/02/2013 09:33 AM, Issam wrote:
Hi Scikit-learn,
Apologies for not replying to the comments I received in the
preproposal, for some reason I had the mailing-list notification
turned off :(. AsYou
- Its very related to MLP and RBM
- For Performance, I will be exploiting numba and GPU techniques.
-For fear of duplicating Theano:
Scikit-learn enjoys a very friendly, flexible API that allows for
simple, black-box usage to perform any complex algorithm. Eventually,
scikit-learn would need easy-to-use neural networks package, that this
project aims to fulfill. Deep learning is just an extension to
primitive neural network techniques.
Implementing the basis of deep learning algorithms in scikit has a lot
of future benefits, since the research area is evolving quickly and
many algorithms are being proposed. Theano might not have all these
algorithms. In addition, scikit allows for easy, robust scalability,
thus, extending deep learning algorithms to meet the current research
demands becomes an ease.
I submitted a proposal to GSOC 2013 which is structured as follows,
*Title,*
(scikit-learn) Deep Learning capabilities for scikit-learn
*Short Description*,
Here I am proposing to work on deep learning to augment its algorithms
in scikit-learn. Deep learning is a relatively new research area that
is progressing fast with a lot of potential for contributions. It
involves an interesting idea by trying to imitate the brain, as it
uses many levels (hidden layers) of processing. Where the levels are
at decreasing order of abstractions!
In this project, I'm planning to work on each step carefully, first I
look into "Deep Boltzmann machines", then "Deep belief
networks","Deep auto-encoders", "Stacked denoising auto-encoders", and
more.
My plan is to establish an easy to use, fast (GPU enhanced),
black-box, plugin, to the scikit-learn library.
This is necessary since Deep Learning is evolving quickly, and having
it in an attractive API such as of scikit's would allow the library to
easily adapt to new changes in this research area
*
**Content,*
The plan is as follows,
Please note that all implementations are done in consistence with
scikit's-learn convention, while exploiting numba and GPU techniques
for high-speed performance
*Until Jun 17*
Familiarize self with deep learning and scikit-learn-'s API
Analyze related algorithms in the pull request (scikit-learn github)
and ensure they are bug free
Implement/reuse dependent algorithms (eg. MLP, RBM)
*Jun 17 - Jun 24*
Read and sketch out "Deep directed networks" in pseudo code
*Jun 24 - Jul 1*
Implement and document "Deep directed networks"
*Jul 1 - Jul 8*
Read and sketch out "Deep Boltzmann machines" in pseudo code
*Jul 8 - Jul 15*
Implement and document " Deep Boltzmann machines "
*Jul 15 - Jul 22*
Learn and implement GPU accelerated Python techniques (eg. shared
variables) to improve speed
Add examples to the aforementioned algorithms
*Jul 22 - Jul 29*
Inspect implemented code thoroughly and submit for mid-term evaluation
*Jul 29 - Aug 5*
Read and sketch out " Deep belief networks" in pseudo code
*Aug 5 - Aug 12*
Implement and document " Deep belief networks"
*Aug 12 - Aug 19*
Read and sketch out " Greedy layer-wise learning of DBNs" in pseudo code
*Aug 19 - Aug 26*
Implement and document " Greedy layer-wise learning of DBNs"
*Aug 26 - Sep 2*
Read and sketch out " Deep multi-layer perceptrons" in pseudo code
Read and sketch out " Deep auto-encoders" in pseudo code
*Sep 2 - Sep 9*
Implement and document " Deep multi-layer perceptrons"
Implement and document " Deep auto-encoders "
*Sep 9 -Sep 16*
Inspect all code thoroughly, and fix defects
Add proper examples:
* Handwritten digit classification using DBNs
* Data visualization and feature discovery using deep auto-encoders
* Information retrieval using deep auto-encoders
* Learning audio features using 1d convolutional DBNs
* Learning image features using 2d convolutional DBNs
*Sep 16 -Sep 23*
Write a user-friendly global documentation giving a step-by-step
guidance for getting started with deep learning in scikit-learn
*Link to a patch/code sample:*
http://deeplearning.net/tutorial/gettingstarted.html**
*Additional Information:*
[1]/Book/, Machine Learning a Probabilistic Perspective,/author/,
Kevin P. Murphy
[2]Lee, Honglak, et al. "Convolutional deep belief networks for
scalable unsupervised learning of hierarchical
representations."/Proceedings of the 26th Annual International
Conference on Machine Learning/. ACM, 2009.
[3] Hinton, Geoffrey E., Simon Osindero, and Yee-Whye Teh. "A fast
learning algorithm for deep belief nets."/Neural computation/ 18.7
(2006): 1527-1554.
[4]Lee, Honglak, et al. "Convolutional deep belief networks for
scalable unsupervised learning of hierarchical
representations."/Proceedings of the 26th Annual International
Conference on Machine Learning/. ACM, 2009.
------------------------------------------------------------------------------
Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET
Get 100% visibility into your production application - at no cost.
Code-level diagnostics for performance bottlenecks with <2% overhead
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap1
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite
It's a free troubleshooting tool designed for production
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap2
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
