Re: [mlpack] Queries regarding GSoC

[Marcus Edel]

Hello Rajiv,

> I did a bit of research and read a few papers related to CNNs. I found the
> following to be very interesting:

Yes, these are some really neat ideas, especially Spatial Transformer Networks
would be a great addition.

> For this, following improvements can also be done: 1. Adding image training
> support in MLPack(If it already exists, I would love to work to simplify the
> process of training images by adding features like Command Line Interface,
> train.txt, test.txt parsing).

There is an open issue https://github.com/mlpack/mlpack/issues/1254 that is
directly related to the CLI idea.

> 2. Also, for working on images, GPU support is needed. Hence, I can work on
> adding GPU support for training CNNs.

There is some work in this direction; https://github.com/conradsnicta/bandicoot-
code is a GPU accelerator add-on for the Armadillo. In the meantime, we could 
use
NVBLAS which is a GPU-accelerated implementation of BLAS.

> I also read a few papers on RBFNs which were really interesting. If possible, 
> I
> would love to work on it as well.

My recommendation is to focus on one or two ideas, each takes time to implement
and writing tests often takes more time as you might anticipate.

I hope this is helpful, let me know if I should clarify anything.

Thanks,
Marcus


> On 25. Feb 2018, at 19:06, Rajiv Vaidyanathan  
> wrote:
> 
> Hi Marcus,
> 
> I did a bit of research and read a few papers related to CNNs. I found the 
> following to be very interesting:
> 1. R-CNN
> 2. Spatial Transformer Networks
> 
> They improve the CNN results without directly affecting the core 
> architecture. They act as an additional layer before the CNN input layer and 
> I felt it would be great addition to the MLPack library which already has the 
> core architecture 
> ready(https://github.com/mlpack/mlpack/tree/master/src/mlpack/methods/ann 
> ).
> 
> Also, I looked into the implementation of Digit 
> Recogniser(https://github.com/mlpack/models/ 
> ). The main application of CNNs is in the 
> domain of images and thus I want to work on improving the process of working 
> with images and that's why I decided to go with R-CNN and Spacial Transformet 
> Networks.
> 
> For this, following improvements can also be done:
> 1. Adding image training support in MLPack(If it already exists, I would love 
> to work to simplify the process of training images by adding features like 
> Command Line Interface, train.txt, test.txt parsing).
> 2. Also, for working on images, GPU support is needed. Hence, I can work on 
> adding GPU support for training CNNs.
> 
> I also read a few papers on RBFNs which were really interesting. If possible, 
> I would love to work on it as well.
> 
> Please let me know if anything more could be added to this or if any of this 
> is feasible. It would be great if you could give your suggestions.
> 
> Regards,
> Rajiv
> 
> 
> On 19 February 2018 at 00:44, Rajiv Vaidyanathan 
> > wrote:
> Hi Marcus,
> 
> The paper "Learning Methods for radial basis function networks" is already 
> present in the ideas page(There is where I found it initially :p). Has RBFN 
> already been implemented in MLPack? Also, I have a small query. In the ideas 
> page, it has been said that we have to implement deep learning modules. Does 
> this necessarily mean more than one? For example, I want to implement RBFN as 
> well something in CNN. But, I am not sure about its feasibility.
> 
> As you said, I'll look into the CNN implementation in MLPack. I'll research 
> for ideas related to it. If possible, could you give me an example of idea 
> which is feasible to work on? As CNN is a vast domain, I would be 
> specifically able to search for ideas in those lines and then start a 
> discussion based on it.
> 
> Regards,
> Rajiv
> 
> On 18 February 2018 at 21:29, Marcus Edel  > wrote:
> Hello Rajiv,
> 
> thanks for the references, I will add the "Learning methods for radial basis
> function networks" paper to the ideas page.
> 
>> Also, I have worked before on CNNs using AlexNet have also worked on a few 
>> Caffe
>> Models like SegNet, etc. I found CNNs to be amazingly creative in the way 
>> they
>> work. Is there a possibility of working on CNN? Has it already been 
>> implemented?
>> Convolutional Neural Networks would be a great addition to MLPack and maybe 
>> if
>> we have a simple interface like Caffe(maybe a simpler one with great 
>> tutorials
>> :p) for using it with image dataset(and for tuning the parameters), then 
>> MLPack
>> would be a really popular choice for learning images.
> 
> Yes, mlpack provides an CNN implementation. Eugene Freyman wrote a really nice
> implementation of a network that learns the MNIST digits for a Kaggle
> 

Re: [mlpack] Queries regarding GSoC

[Marcus Edel]

Hello Rajiv,

> I initially had 3 topics of choice(in the previous mail), but after going
> through the mailing list archive, mlpack blog and other links provided by 
> you, I
> have decided to go ahead with "ESSENTIAL DEEP LEARNING MODULES". I like the 
> list
> of suggested algorithms. I'll also see if I can think of anything. Please let 
> me
> know if you have any suggestions in mind.

Sounds good, we will update the project description in the next days and
probably add more suggestions.

> So, how should I go about from here? Is there any open ticket to work on? 
> Should
> I start thinking about the proposal?

If you don't mind I'd like to finish the open PR first, before opening another
PR, how does this sound? About the proposal, if you like you can start working
on it, the most important part is the timeline.

I hope anything I said is helpful.

Thanks,
Marcus

> On 13. Feb 2018, at 08:07, Rajiv Vaidyanathan  
> wrote:
> 
> Hi Marcus,
> 
> Firstly, congratulations for making into GSoC 2018! 
> 
> I initially had 3 topics of choice(in the previous mail), but after going 
> through the mailing list archive, mlpack blog and other links provided by 
> you, I have decided to go ahead with "ESSENTIAL DEEP LEARNING MODULES". I 
> like the list of suggested algorithms. I'll also see if I can think of 
> anything. Please let me know if you have any suggestions in mind.
> 
> So, how should I go about from here? Is there any open ticket to work on? 
> Should I start thinking about the proposal?
> 
> Also, I thought I should to finish the SPSA 
> optimizer(https://github.com/mlpack/mlpack/pull/1153 
> ) before I start working on a new 
> issue. I am stuck up in the implementation of the test code. I have posted 
> the latest error in the comments. It would be great if you could help me out.
> 
> Regards,
> Rajiv
> 
> On 3 February 2018 at 20:37, Rajiv Vaidyanathan 
> > wrote:
> 
> 
> Hi Marcus,
> 
> As of now, I am working on writing a test for SPSA optimizer which I have 
> implemented... I'll try to finish it ASAP. As of now, I cannot think of any 
> DL model good enough to replace the suggested ones... Also, as you said, lets 
> wait until Google officials confirms :)
> 
> Regards,
> Rajiv
> 
> On 3 February 2018 at 03:22, Marcus Edel  > wrote:
> Hello Rajiv,
> 
> Nice to hear from you again how are things going?
> 
> > I am interested in the following topics(listed in the order of interest):
> > 1. Reinforcement Learning
> > 2. Essential Deep Learning Modules
> > 3. Particle Swarm Optimization
> >
> > How should I go about it? I read a few Mailing List archives for 1 and 3. 
> > What
> > should I do after that? Can I start working on the proposal submission?
> 
> 
> Going through the mailing list archive is definitely a good starting point, 
> also
> the weekly updates from Kris and Shangtong
> (http://www.mlpack.org/gsocblog/index.html 
> ) could be interesting too.
> 
> The models listed for the Essential Deep Learning Modules idea are just
> suggestions, if you like to work on an interesting network model over the 
> summer
> please feel free to start a discussion.
> 
> A good starting point, in general, is to get familiar with the codebase, if 
> you
> have any question please don't hesitate to ask. About the submission that's up
> to you, but note Google hasn't announced the accepted organizations yet and
> there is plenty of time to prepare the proposal and get feedback.
> 
> I hope this is helpful, let us know if we should clarify anything.
> 
> Thanks,
> Marcus
> 
> > On 2. Feb 2018, at 16:19, Rajiv Vaidyanathan  > > wrote:
> >
> > Hi Marcus,
> >
> > I'm N Rajiv Vaidyanathan(github handle: rajiv2605). I have contributed to 
> > mlpack in the past and I really like this organisation. I am very 
> > interested in participating in GSoC.
> >
> > I am interested in the following topics(listed in the order of interest):
> > 1. Reinforcement Learning
> > 2. Essential Deep Learning Modules
> > 3. Particle Swarm Optimization
> >
> > How should I go about it? I read a few Mailing List archives for 1 and 3. 
> > What should I do after that? Can I start working on the proposal submission?
> >
> > Thanking you.
> >
> > Regards,
> > Rajiv
> >
> > ___
> > mlpack mailing list
> > mlpack@lists.mlpack.org 
> > http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack 
> > 
> 
> 
> 
> 
> ___
> mlpack mailing list
> mlpack@lists.mlpack.org
> http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack

Re: [mlpack] Queries regarding GSoC

[Marcus Edel]

Hello Rajiv,

Nice to hear from you again how are things going?

> I am interested in the following topics(listed in the order of interest):
> 1. Reinforcement Learning
> 2. Essential Deep Learning Modules
> 3. Particle Swarm Optimization
> 
> How should I go about it? I read a few Mailing List archives for 1 and 3. What
> should I do after that? Can I start working on the proposal submission?


Going through the mailing list archive is definitely a good starting point, also
the weekly updates from Kris and Shangtong
(http://www.mlpack.org/gsocblog/index.html) could be interesting too.

The models listed for the Essential Deep Learning Modules idea are just
suggestions, if you like to work on an interesting network model over the summer
please feel free to start a discussion.

A good starting point, in general, is to get familiar with the codebase, if you
have any question please don't hesitate to ask. About the submission that's up
to you, but note Google hasn't announced the accepted organizations yet and
there is plenty of time to prepare the proposal and get feedback.

I hope this is helpful, let us know if we should clarify anything.

Thanks,
Marcus

> On 2. Feb 2018, at 16:19, Rajiv Vaidyanathan  
> wrote:
> 
> Hi Marcus,
> 
> I'm N Rajiv Vaidyanathan(github handle: rajiv2605). I have contributed to 
> mlpack in the past and I really like this organisation. I am very interested 
> in participating in GSoC.
> 
> I am interested in the following topics(listed in the order of interest):
> 1. Reinforcement Learning
> 2. Essential Deep Learning Modules
> 3. Particle Swarm Optimization
> 
> How should I go about it? I read a few Mailing List archives for 1 and 3. 
> What should I do after that? Can I start working on the proposal submission?
> 
> Thanking you.
> 
> Regards,
> Rajiv
> 
> ___
> mlpack mailing list
> mlpack@lists.mlpack.org
> http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack

___
mlpack mailing list
mlpack@lists.mlpack.org
http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack