solin319 commented on issue #8005: add warmup lr_scheduler URL: https://github.com/apache/incubator-mxnet/pull/8005#issuecomment-332713673 I have added additional argument in MultiFactorScheduler class. Warmup can be used when 'warmup_step' > 1. I added a test file in test/commom/test_lr_sceduler.py. Following is a test result ``` python test_lr_scheduler.py --lr-factor=0.1 --warmup_step=5 --lr-steps='8,10' INFO:root:step 1 lr = 0.020000 INFO:root:step 2 lr = 0.040000 INFO:root:step 3 lr = 0.060000 INFO:root:step 4 lr = 0.080000 INFO:root:Update[5]: now learning rate arrived at 1.00000e-01, will not warm up in the future INFO:root:step 5 lr = 0.100000 INFO:root:step 6 lr = 0.100000 INFO:root:step 7 lr = 0.100000 INFO:root:step 8 lr = 0.100000 INFO:root:Update[9]: Change learning rate to 1.00000e-02 INFO:root:step 9 lr = 0.010000 INFO:root:step 10 lr = 0.010000 INFO:root:Update[11]: Change learning rate to 1.00000e-03 INFO:root:step 11 lr = 0.001000 INFO:root:step 12 lr = 0.001000 INFO:root:step 13 lr = 0.001000 INFO:root:step 14 lr = 0.001000 ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
With regards, Apache Git Services
