Oh wait. a DNN BSPJob partitions the input data (examples) into the number of tasks, then each task executes SGD graph job using LocalBSPRunner as a mini-batch, and uses external pm server to avoid the heavy cost of synchronization of shared variables such as gradients or parameters. Does it make sense?
On Wed, May 27, 2015 at 9:34 AM, Edward J. Yoon <[email protected]> wrote: > According to their architecture, DNN job is consist of multiple (mini-batch) > SGD job groups and 'ParameterServer' is used for swapping parameters among > them. Since these mini-batch jobs need to be work asynchronously, I guess we > can't do this in single job of Hama BSP framework. Hama can be used as a sub > component. > > -- > Best Regards, Edward J. Yoon > > -----Original Message----- > From: Chia-Hung Lin [mailto:[email protected]] > Sent: Tuesday, May 26, 2015 5:40 PM > To: [email protected] > Subject: Re: DNN at Tencent > > That's why I am working on refactoring core module stuff : ) > > BSP is suitable for iterative applications and is for general purpose > parallelism. So as long as the application can be expressed as > iterative algorithm, ideally our framework (in terms of higher > abstraction level) should have no problem for such applications. > > > On 26 May 2015 at 16:08, Edward J. Yoon <[email protected]> wrote: >> Just FYI, http://www.vldb.org/pvldb/vol7/p1772-tencent.pdf >> >> According to them, they used BSP model (and additional Parameter >> server) like google's distbelief. Do you think we can do this? >> >> >> -- >> Best Regards, Edward J. Yoon > > -- Best Regards, Edward J. Yoon
