sxjscience commented on issue #9363: Incorrect weight_decay implementation in AdaGrad URL: https://github.com/apache/incubator-mxnet/issues/9363#issuecomment-358345278 For SGD, not merging in the wd term will linearly scale the ?real? wd and the impact is smaller compared to Adagrad. However, the change may break lots of our current examples and I?m not sure whether to revise it or not. We need to investigate how the other packages implement the WD term. Get Outlook for iOS<https://aka.ms/o0ukef> ________________________________ From: Ziyue Huang <[email protected]> Sent: Wednesday, January 17, 2018 7:01:29 AM To: apache/incubator-mxnet Cc: Xingjian SHI; Mention Subject: Re: [apache/incubator-mxnet] Incorrect weight_decay implementation in AdaGrad (#9363) wd term is also not merged into grad in SGD. Should we fix them? Will it cause some problems for backward compatibility? cc @sxjscience<https://github.com/sxjscience> ? You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<https://github.com/apache/incubator-mxnet/issues/9363#issuecomment-358330988>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AE8D7tu5nAPXmIkEMh35dR1rS6yrQ5odks5tLgtJgaJpZM4RYksA>.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
