Stephanie Yuan created MXNET-854:
------------------------------------

             Summary: SVRG Optimization in Python Module API
                 Key: MXNET-854
                 URL: https://issues.apache.org/jira/browse/MXNET-854
             Project: Apache MXNet
          Issue Type: New Feature
            Reporter: Stephanie Yuan


SVRG stands for Stochastic Variance Reduced Gradients, which is an optimization 
technique that employs explicit variance reduction. It has provable guarantees 
for strong smoothly convex functions and converges faster compared to SGD. 

The proposal is to create a SVRGModule that implements the SVRG optimization 
technique under the hood. 

SVRG Module should:
 * seamlessly supports both dense and sparse data, run on CPU and GPU instances 
on single machine and in distributed setting.
 * minimize the API differences with Python Module API.

Design Proposal can be found 
[here|https://cwiki.apache.org/confluence/display/MXNET/SVRG+Optimization+in+MXNet+Python+Module].
 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to