sxjscience commented on issue #7942: Adam optimizer consistent with paper URL: https://github.com/apache/incubator-mxnet/pull/7942#issuecomment-331651717 @formath I feel that the `rho` has the effect to gradually transform the gradient estimator from biased to unbiased, which may have some advantages. However, I've checked the Adam paper and haven't found the rho hyper-parameter. Could you help point out the section in the paper? Also, we need to better document its usage. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
With regards, Apache Git Services
