[
https://issues.apache.org/jira/browse/SINGA-46?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangwei resolved SINGA-46.
--------------------------
Resolution: Fixed
> Fix a bug in updater.cc to scale the gradients
> ----------------------------------------------
>
> Key: SINGA-46
> URL: https://issues.apache.org/jira/browse/SINGA-46
> Project: Singa
> Issue Type: Bug
> Reporter: wangwei
>
> The Updater's Update function has an argument grad_scale (default value is
> 1.0), which is used to scale the gradients of parameters. For instance, when
> n workers compute over one mini-batch (each worker is assigned 1/n records),
> then their gradients should be averaged. We can do the average by passing
> grad_scale=1/n.
> Some updaters in updater.cc, e.g., AdaGradUpdater, forget to scale the
> gradients by grad_scale. The bug can be fixed by
> {code}
> if (grad_scale != 1)
> grad *= grad_scale;
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
