[ https://issues.apache.org/jira/browse/SYSTEMML-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
LI Guobao updated SYSTEMML-2420: -------------------------------- Description: It aims to implement the parameter exchange between ps and workers. We could leverage spark RPC to setup a ps endpoint in driver node which means that the ps service could be discovered by workers in the network. And then the workers could invoke the pull/push method via RPC using the registered endpoint of ps service. Hence, in details, this tasks consists of registering the ps endpoint in spark rpc framework and using rpc to invoke target method in worker side. We can learn that the spark rpc is implemented in Scala. But we could easily wrap them in java class to be reused. (was: It aims to implement the parameter exchange between ps and workers. We could leverage spark RPC to setup a ps endpoint in driver node which means that the ps service could be discovered by workers in the network. And then the workers could invoke the pull/push method via RPC using the registered endpoint of ps service. Hence, in details, this tasks consists of registering the ps endpoint in spark rpc framework and using rpc to invoke target method in worker side.) > Communication between ps and workers > ------------------------------------ > > Key: SYSTEMML-2420 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2420 > Project: SystemML > Issue Type: Sub-task > Reporter: LI Guobao > Assignee: LI Guobao > Priority: Major > > It aims to implement the parameter exchange between ps and workers. We could > leverage spark RPC to setup a ps endpoint in driver node which means that the > ps service could be discovered by workers in the network. And then the > workers could invoke the pull/push method via RPC using the registered > endpoint of ps service. Hence, in details, this tasks consists of registering > the ps endpoint in spark rpc framework and using rpc to invoke target method > in worker side. We can learn that the spark rpc is implemented in Scala. But > we could easily wrap them in java class to be reused. -- This message was sent by Atlassian JIRA (v7.6.3#76005)