Re: MXNET-1294: Priority-based parameter propagation for improved data parallel training throughput

2019-01-14 Thread Sheng Zha
Hi Anand,

Thanks for sharing the work and for offering to improve mxnet and ps-lite.

If you don't need to test the integration, you can
- fork dmlc/ps-lite
- make the changes
- send a pull request back to the repo, just as you would to mxnet

If you need to test the integration in mxnet first, you can
- fork both mxnet and ps-lite
- in your mxnet fork, switch the ps-lite to your own fork
- develop and test in your own forks
- once ready, send PR to ps-lite, followed by PR to mxnet.

Given that you intend to slice the gradients in to chunks and change the
order in which they are applied, I'm interested to see how you intend to
abstract them. If you have specific technical questions on either mxnet or
ps-lite, feel free to ask on github issues if that's easier for you.

-sz

On Mon, Jan 14, 2019 at 9:23 AM Anand J  wrote:

> Hi All,
>
> I'm planning to add some improvements to MXNet KVStore based on the
> ideas from my recent SysML'19 paper
> https://anandj.in/wp-content/uploads/sysml.pdf. I have created a JIRA
> ticket on this: https://issues.apache.org/jira/browse/MXNET-1294.
>
> The code changes I'm planning to do require changes in PS-Lite and
> KVStore. I'm new to MXNet developer community. It would be helpful If
> someone can give guidance on how should I be proceeding with this.
>
> Thanks,
> Anand
>


MXNET-1294: Priority-based parameter propagation for improved data parallel training throughput

2019-01-14 Thread Anand J

Hi All,

I'm planning to add some improvements to MXNet KVStore based on the 
ideas from my recent SysML'19 paper
https://anandj.in/wp-content/uploads/sysml.pdf. I have created a JIRA 
ticket on this: https://issues.apache.org/jira/browse/MXNET-1294.


The code changes I'm planning to do require changes in PS-Lite and 
KVStore. I'm new to MXNet developer community. It would be helpful If 
someone can give guidance on how should I be proceeding with this.


Thanks,
Anand