onomatet commented on issue #17207: LR schedulers do not work in R URL: https://github.com/apache/incubator-mxnet/issues/17207#issuecomment-570788495 ## Minimal example I hope it will be of some help. Modified after the tutorial. ### Code ``` data(BostonHousing, package="mlbench") train.ind = seq(1, 506, 3) train.x = data.matrix(BostonHousing[train.ind, -14]) train.y = BostonHousing[train.ind, 14] data <- mx.symbol.Variable("data") fc1 <- mx.symbol.FullyConnected(data, num_hidden=1) lro <- mx.symbol.LinearRegressionOutput(fc1) cat("Without scheduler\n") mx.set.seed(0) model1 <- mx.model.FeedForward.create(lro, X=train.x, y=train.y, ctx=mx.cpu(), num.round=20, array.batch.size=20, learning.rate=2e-6, momentum=0.9, eval.metric=mx.metric.rmse) cat("\nWith scheduler\n") lr_scheduler <- mx.lr_scheduler.FactorScheduler( step = 5 * ceiling(length(train.y)/20), factor = 0.1, stop_factor_lr = 1e-10) mx.set.seed(0) model2 <- mx.model.FeedForward.create(lro, X=train.x, y=train.y, ctx=mx.cpu(), num.round=20, array.batch.size=20, learning.rate=2e-6, momentum=0.9, eval.metric=mx.metric.rmse, lr_scheduler = lr_scheduler) ``` ### Output ``` Without scheduler Start training with 1 devices [1] Train-rmse=18.0516391330295 [2] Train-rmse=13.9097522099813 [3] Train-rmse=10.666042221917 [4] Train-rmse=10.0117386711968 [5] Train-rmse=9.59162097507053 [6] Train-rmse=9.80227173699273 [7] Train-rmse=9.56405830383301 [8] Train-rmse=9.39033126831055 [9] Train-rmse=9.33245415157742 [10] Train-rmse=9.31073543760512 [11] Train-rmse=9.27919896443685 [12] Train-rmse=9.24656009674072 [13] Train-rmse=9.206680615743 [14] Train-rmse=9.17186906602648 [15] Train-rmse=9.14681609471639 [16] Train-rmse=9.12289328045315 [17] Train-rmse=9.09742567274306 [18] Train-rmse=9.0733421113756 [19] Train-rmse=9.05105861028036 [20] Train-rmse=9.02993933359782 With scheduler Start training with 1 devices [1] Train-rmse=18.0516391330295 [2] Train-rmse=13.9097522099813 [3] Train-rmse=10.666042221917 [4] Train-rmse=10.0117386711968 [5] Train-rmse=9.59162097507053 Update[46]: learning rate is changed to 2e-07 [6] Train-rmse=9.80227173699273 [7] Train-rmse=9.56405830383301 [8] Train-rmse=9.39033126831055 [9] Train-rmse=9.33245415157742 [10] Train-rmse=9.31073543760512 Update[91]: learning rate is changed to 2e-08 [11] Train-rmse=9.27919896443685 [12] Train-rmse=9.24656009674072 [13] Train-rmse=9.206680615743 [14] Train-rmse=9.17186906602648 [15] Train-rmse=9.14681609471639 Update[136]: learning rate is changed to 2e-09 [16] Train-rmse=9.12289328045315 [17] Train-rmse=9.09742567274306 [18] Train-rmse=9.0733421113756 [19] Train-rmse=9.05105861028036 [20] Train-rmse=9.02993933359782 Warning messages: 1: In mx.model.select.layout.train(X, y) : Auto detect layout of input matrix, use rowmajor.. 2: In mx.model.select.layout.train(X, y) : Auto detect layout of input matrix, use rowmajor.. ``` Train-rmse is the same in every iteration regardless of the scheduler used.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
