piiswrong commented on a change in pull request #8107: "add warmup
lr_scheduler" create a new pr
URL: https://github.com/apache/incubator-mxnet/pull/8107#discussion_r151878202
##########
File path: python/mxnet/lr_scheduler.py
##########
@@ -100,16 +100,27 @@ class MultiFactorScheduler(LRScheduler):
Then calculate the new learning rate by::
- base_lr * pow(factor, k+1)
+ base_lr * pow(factor, k+1)
+
+ When warmup_step>1, warmup the learning rate by a const value for first
warmup_step steps.
+ It returns a new learning rate by::
+
+ begin_lr + (num_update - 1) * const_update
Parameters
----------
step: list of int
- The list of steps to schedule a change
+ The list of steps to schedule a change.
factor: float
The factor to change the learning rate.
+ warmup_step : int
+ Changes the learning rate for first 'warmup_step' updates.
+ begin_lr : float, optional
+ The learning rate at begin.
+ stop_lr : float, optional
+ Stop updating the learning rate if it is less than this value.
Review comment:
begin_lr and stop_lr are confusing names and the does doesn't explain
exactly what they mean
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
