piiswrong opened a new pull request #11953: do not regularize beta and bias URL: https://github.com/apache/incubator-mxnet/pull/11953 In Module we only put weight decay on variables ending in "_weight" or "_gamma" while in gluon we are regularizing everything. This PR removes regularization on bias and beta. Further issues to discuss: 1. should we regularize Embedding layer's weight? (this is currently regularized in module) 2. should we regularize alpha in PReLU?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
