Hello Vivek, > I went through a lot of text on deep learning in the past few days starting > with > covering chapter 20 from Deep Learning book by Ian Goodfellow and following > up with the > papers I mentioned earlier.
There are many really good resources, also I often hear that the coursera machine learning courses are worth a look. > Also, while going through the text and looking at the codebase simulatenously > I discovered > that there's scope for implementing atleast a couple more activation > functions, e.g. softplus > and a more recent one; ELU (https://arxiv.org/abs/1511.07289 > <https://arxiv.org/abs/1511.07289>). > > I believe ELU can be implemented similar to the existing implementation of > Leaky ReLU layer > and softplus can be simply added as a new activation function in > ann/actication_functions > directory. Absolutely, I was going to open an issue for the ELU function, but sounds like I don't have to do this anymore. If you are going to implement one or both functions, don't feel obligated, please open a single PR for each function and also include some tests. Thanks, Marcus > On 9 Feb 2017, at 13:19, Vivek Pal <[email protected]> wrote: > > Hi Marcus, > > I went through a lot of text on deep learning in the past few days starting > with > covering chapter 20 from Deep Learning book by Ian Goodfellow and following > up with the > papers I mentioned earlier. > > Also, while going through the text and looking at the codebase simulatenously > I discovered > that there's scope for implementing atleast a couple more activation > functions, e.g. softplus > and a more recent one; ELU (https://arxiv.org/abs/1511.07289 > <https://arxiv.org/abs/1511.07289>). > > I believe ELU can be implemented similar to the existing implementation of > Leaky ReLU layer > and softplus can be simply added as a new activation function in > ann/actication_functions > directory. > > Would that be a useful first contribution? > > It'll help me gain first hand experience with mlpack development process and > a better > understanding of ann modules internals. > > > also I've added some papers to the Essential Deep Learning Modules project > > idea, that > > might be interesting too. > > Yeah, I feel I could use some more ideas or details on certain architectures > from some other > papers that you have added there. Thanks, will get started once I'm done > reading all papers > at hand. > > Thanks, > Vivek Pal > _______________________________________________ > mlpack mailing list > [email protected] > http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack
_______________________________________________ mlpack mailing list [email protected] http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack
