Sorry, but I'm not an DL expert and can't do such recommandation.
Maybe someone else here can, but you can also ask on the pylearn2 mailing
list.
Fred
On Thu, May 2, 2013 at 11:12 AM, Issam <issamo...@gmail.com> wrote:
> Hi,
>
> Thanks a lot for the comment, I hope this doesn't open a new thread :),
> I'm pretty new to using mailing list.
>
> You are right that I'm underestimating the development time to craft
> efficient,usable DL algorithms.
>
> For this I would like to ask your opinion on which deep models do you
> recommend I should focus on within the given time frame?
>
> Thanks a lot,
> yours truly
> --Issam
>
> On 5/2/2013 5:20 PM, Frédéric Bastien wrote:
>
> Hi,
>
> I have no dough you are a great programmer, but even people in my lab
> that is specialized in deep learning won't be able to do the full list
> while respecting the scikit-learn code quality, documentation and
> performance/efficiency level.
>
> I think you should keep 1 or 2 deep model. Just look at the time it took
> for the MLP and RBM PR to be done. I don't expect less time for yours.
>
> Other people mentioned the problem of usability of deep learning
> techniques. I think you should focus on that instead of doing many models.
> That is the what will difference your implementation from
> Theano/Pylearn2/DLT. For this, you could check James Bergstra email on this
> list that talk about automatic hyper-parameter selection. I that could
> solve a big part of the usability problem of deep learning. I suppose good
> documentation could do the rest.
>
> Also, shared variable is a Theano only thing. For GPU without Theano, you
> can look at Numba Pro, PyCUDA or PyOpenCL. scikit-learn don't want Theano
> as a dependency (and I understand that).
>
> HTH
>
> Frédéric Bastien
>
> Disclaimer: I'm a core Theano developer. I never contributed to
> scikit-learn, so take other people comment from this list more important
> then mine.
>
>
> On Thu, May 2, 2013 at 5:34 AM, Issam <issamo...@gmail.com> wrote:
>
>> Hi Vladn,
>>
>> Here is the updated proposal, I have added the current challenges and
>> proposed solutions on the abstract,
>>
>>
>> https://google-melange.appspot.com/gsoc/proposal/review/google/gsoc2013/issamou/1#
>>
>> Thank you!
>>
>> On 5/2/2013 11:34 AM, Vlad Niculae wrote:
>> > Sorry, I just saw that your submission is on Melange.
>> >
>> > I think the proposal could use some discussion on what issues might be
>> > faced. Many people here have expressed concerns about including "deep
>> > stuff", the difficulty to have sensible defaults, the difficulty to
>> > having a general-purpose efficient implementation that can be used on
>> > different domains without hacking the code. In the very simple RBM,
>> > the example is still unsatisfactory because it is hard to show off the
>> > algorithm on too small a dataset. This might be even trickier with
>> > deeper things.
>> >
>> > In tuning a good neural model some know-how and tricks are needed,
>> > many times you need to look over the training process and measure
>> > statistics. It would be useful to describe this kind of difficulties
>> > and how we might be able to avoid them, what kind of hyperparameter
>> > heuristics / initialization should be used, etc. It is early to go
>> > into it too deeply (pun intended) but I think the proposal can benefit
>> > by your embracing the skeptic side.
>> >
>> > Hope this helps,
>> > Vlad
>> >
>> >
>> > On Thu, May 2, 2013 at 5:20 PM, Vlad Niculae <zephy...@gmail.com>
>> wrote:
>> >> Hi Issam,
>> >>
>> >> The deadline is fast approaching. How is your proposal going? Could
>> >> you share a version so we can give some feedback?
>> >>
>> >> Yours,
>> >> Vlad
>> >>
>> >> On Sat, Apr 20, 2013 at 3:57 AM, amir rahimi <noname01....@gmail.com>
>> wrote:
>> >>> Sorry, I didn't see Andy's note ;)
>> >>>
>> >>>
>> >>> On Fri, Apr 19, 2013 at 11:23 PM, amir rahimi <noname01....@gmail.com
>> >
>> >>> wrote:
>> >>>> Hi,
>> >>>> I recommend Theano if you want to use python with GPU for deep
>> learning.
>> >>>> It is tightly integrated with numpy....
>> >>>>
>> >>>> Best,
>> >>>> Amir
>> >>>>
>> >>>>
>> >>>> On Thu, Apr 18, 2013 at 9:21 PM, Wei LI <kuant...@gmail.com> wrote:
>> >>>>> @Andy What do you mean by "blackbox" algorithm? Does that mean
>> something
>> >>>>> similar to pylearn2?
>> >>>>>
>> >>>>> @Issam, It seems to me that scalablity is a key factor to train deep
>> >>>>> models and make them work. Do you have any suggestion how to make it
>> >>>>> scalable while still fits in sklearn framework? I think sklearn
>> cannot
>> >>>>> supports GPU easily. I wanna know is training a deep model for a
>> mid-level
>> >>>>> scale(maybe like cifar?) painful on CPU only with numpy?
>> >>>>>
>> >>>>> Best,
>> >>>>> Wei
>> >>>>>
>> >>>>> On Fri, Apr 19, 2013 at 12:27 AM, Andreas Mueller
>> >>>>> <amuel...@ais.uni-bonn.de> wrote:
>> >>>>>> Hi Issam.
>> >>>>>> Thank you for your interest. Have you looked at the
>> >>>>>> MLP and RBM pull requests that are currently open?
>> >>>>>> How would your project relate to those?
>> >>>>>>
>> >>>>>> A real problem is that we don't want to replicate theano
>> >>>>>> and rather have a somewhat "black box" algorithm that people can
>> >>>>>> apply....
>> >>>>>>
>> >>>>>> Cheers,
>> >>>>>> Andy
>> >>>>>>
>> >>>>>>
>> >>>>>> On 04/18/2013 06:07 PM, Issam wrote:
>> >>>>>>> Hi scikit,
>> >>>>>>>
>> >>>>>>> Here I am proposing to work on deep learning topic for GSOC 2013.
>> Deep
>> >>>>>>> learning is a relatively new research area that is progressing
>> fast
>> >>>>>>> with a lot of potential for contributions. It involves an
>> intersting
>> >>>>>>> idea by trying to imitate the brain, as it uses many levels
>> (hidden
>> >>>>>>> layers) of processing. Where the levels are at decreasing order of
>> >>>>>>> abstractions!
>> >>>>>>>
>> >>>>>>> In this project, I'm planning to work on each step carefully,
>> first I
>> >>>>>>> look into "Deep Boltzmann machines", then "Deep belief
>> >>>>>>> networks","Deep
>> >>>>>>> auto-encoders", "Stacked denoising auto-encoders", and more. I
>> could
>> >>>>>>> create a complete plan for this, once I get your feedback :)
>> >>>>>>>
>> >>>>>>> I have been involved in quite a number of machine learning
>> projects,
>> >>>>>>> from dealing with imbalanced datasets (software quality
>> prediction),
>> >>>>>>> to
>> >>>>>>> XML classification, from recognizing gender out of handwriting, to
>> >>>>>>> breast cancer prediction using mammograms. I'm in my second
>> semester
>> >>>>>>> as
>> >>>>>>> a graduate student (MSc), and machine learning is my research
>> area. My
>> >>>>>>> thesis would involve deep learning, which i will apply on
>> >>>>>>> bioinformatics
>> >>>>>>> and face recognition.
>> >>>>>>>
>> >>>>>>> I would be more than happy to work with a mentor on this!
>> >>>>>>>
>> >>>>>>> Thank you!
>> >>>>>>>
>> >>>>>>> Best regards,
>> >>>>>>> --Issam Laradji
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> ------------------------------------------------------------------------------
>> >>>>>>> Precog is a next-generation analytics platform capable of advanced
>> >>>>>>> analytics on semi-structured data. The platform includes APIs for
>> >>>>>>> building
>> >>>>>>> apps and a phenomenal toolset for data science. Developers can use
>> >>>>>>> our toolset for easy data analysis & visualization. Get a free
>> >>>>>>> account!
>> >>>>>>> http://www2.precog.com/precogplatform/slashdotnewsletter
>> >>>>>>> _______________________________________________
>> >>>>>>> Scikit-learn-general mailing list
>> >>>>>>> Scikit-learn-general@lists.sourceforge.net
>> >>>>>>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> ------------------------------------------------------------------------------
>> >>>>>> Precog is a next-generation analytics platform capable of advanced
>> >>>>>> analytics on semi-structured data. The platform includes APIs for
>> >>>>>> building
>> >>>>>> apps and a phenomenal toolset for data science. Developers can use
>> >>>>>> our toolset for easy data analysis & visualization. Get a free
>> account!
>> >>>>>> http://www2.precog.com/precogplatform/slashdotnewsletter
>> >>>>>> _______________________________________________
>> >>>>>> Scikit-learn-general mailing list
>> >>>>>> Scikit-learn-general@lists.sourceforge.net
>> >>>>>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> LI, Wei
>> >>>>> Tsinghua/CUHK
>> >>>>> http://kuantkid.github.com/
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> ------------------------------------------------------------------------------
>> >>>>> Precog is a next-generation analytics platform capable of advanced
>> >>>>> analytics on semi-structured data. The platform includes APIs for
>> >>>>> building
>> >>>>> apps and a phenomenal toolset for data science. Developers can use
>> >>>>> our toolset for easy data analysis & visualization. Get a free
>> account!
>> >>>>> http://www2.precog.com/precogplatform/slashdotnewsletter
>> >>>>> _______________________________________________
>> >>>>> Scikit-learn-general mailing list
>> >>>>> Scikit-learn-general@lists.sourceforge.net
>> >>>>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>> >>>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>>
>> ----------------------------------------------------------------------
>> >>>> #include <stdio.h>
>> >>>> double d[]={9299037773.178347,2226415.983937417,307.0};
>> >>>> main(){d[2]--?d[0]*=4,d[1]*=5,main():printf((char*)d);}
>> >>>>
>> ----------------------------------------------------------------------
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> ----------------------------------------------------------------------
>> >>> #include <stdio.h>
>> >>> double d[]={9299037773.178347,2226415.983937417,307.0};
>> >>> main(){d[2]--?d[0]*=4,d[1]*=5,main():printf((char*)d);}
>> >>> ----------------------------------------------------------------------
>> >>>
>> >>>
>> ------------------------------------------------------------------------------
>> >>> Precog is a next-generation analytics platform capable of advanced
>> >>> analytics on semi-structured data. The platform includes APIs for
>> building
>> >>> apps and a phenomenal toolset for data science. Developers can use
>> >>> our toolset for easy data analysis & visualization. Get a free
>> account!
>> >>> http://www2.precog.com/precogplatform/slashdotnewsletter
>> >>> _______________________________________________
>> >>> Scikit-learn-general mailing list
>> >>> Scikit-learn-general@lists.sourceforge.net
>> >>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>> >>>
>> >
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET
>> Get 100% visibility into your production application - at no cost.
>> Code-level diagnostics for performance bottlenecks with <2% overhead
>> Download for free and get started troubleshooting in minutes.
>> http://p.sf.net/sfu/appdyn_d2d_ap1
>> _______________________________________________
>> Scikit-learn-general mailing list
>> Scikit-learn-general@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>>
>
>
>
------------------------------------------------------------------------------
Introducing AppDynamics Lite, a free troubleshooting tool for Java/.NET
Get 100% visibility into your production application - at no cost.
Code-level diagnostics for performance bottlenecks with <2% overhead
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap1
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
