[
https://issues.apache.org/jira/browse/SINGA-8?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangwei resolved SINGA-8.
-------------------------
Resolution: Fixed
> Implement distributed Hogwild
> -----------------------------
>
> Key: SINGA-8
> URL: https://issues.apache.org/jira/browse/SINGA-8
> Project: Singa
> Issue Type: New Feature
> Reporter: wangwei
> Assignee: wangwei
> Labels: distributed, features, hogwild
>
> Generally, both the Downpour framework of Google Brain [1] and the Caffe's
> distributed Hogwild implementation are extensions of the shared memory
> Hogwild training. In this ticket, we refer to the second one.
> In specific, each server group masters a subset of parameters (i.e., Param
> objects) when synchronizing with other server groups. It aggregates all
> updates for its subset and sends back (e.g., broadcast) the updated
> parameters back to all other server groups. The synchronization is conducted
> asynchronously. The frequency can be fixed in the first implementations.
> Finally, it should be tuned automatically to fully utilize the network
> bandwidth.
> [1]J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, Q. V. Le, M. Z. Mao, M.
> Ranzato, A. W. Senior, P. A. Tucker, K. Yang, and A. Y. Ng. Large scale
> distributed deep networks. In NIPS, pages 1232{1240, 2012.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
