Thanks everyone for the feedback and input! I've rebased and resolved the conflict with master. I have also added some more unittests for SVRGModule especially on verifying SVRG gradients calculations and the test coverage is now: mxnet/contrib/svrg_optimization/svrg_module.py 175 21 88% mxnet/contrib/svrg_optimization/svrg_optimizer.py 38 1 97%
@sandeep-krishnamurthy I'm working on creating the MNIST example but I'm expecting it will take a few more days as I have other tasks in queue. Maybe I can include the example in my next PR on SVRG in distributed mode? @anirudhacharya SVRGOptimizer is designed to be used with SVRGModule alone. The actual SVRG optimization logic is implemented in the SVRGModule and the SVRGOptimizer is designed to assist the full gradients accumulation in the KVStore. I've made SVRGOptimizer a private class and added the reasonings behind the design in greater detail in the docs/api/python/contrib/svrg_optimization. Hopefully it clarifies things a bit. @aaronmarkham I've added pointer about the svrg_optimization in index.md and refined the docs in the example/svrg_module/README. [ Full content available at: https://github.com/apache/incubator-mxnet/pull/12376 ] This message was relayed via gitbox.apache.org for [email protected]
