thomelane commented on a change in pull request #11296: Adding 2 tutorials on Learning Rate Schedules URL: https://github.com/apache/incubator-mxnet/pull/11296#discussion_r196614808
########## File path: docs/tutorials/gluon/learning_rate_schedules.md ########## @@ -0,0 +1,317 @@ + +# Learning Rate Schedules + +Setting the learning rate for stochastic gradient descent (SGD) is crucially important when training neural network because it controls both the speed of convergence and the ultimate performance of the network. One of the simplest learning rate strategies is to have a fixed learning rate throughout the training process. Choosing a small learning rate allows the optimizer find good solutions but this comes at the expense of limiting the initial speed of convergence. Changing the learning rate over time can overcome this tradeoff. Review comment: Corrected. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
