[ 
https://issues.apache.org/jira/browse/MXNET-854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stephanie Yuan updated MXNET-854:
---------------------------------
    Description: 
SVRG stands for Stochastic Variance Reduced Gradients, which is an optimization 
technique that employs explicit variance reduction. It has provable guarantees 
for strong smoothly convex functions and converges faster compared to SGD. 

The proposal is to create a SVRGModule that implements the SVRG optimization 
technique under the hood. 

SVRG Module should:
 * seamlessly supports both dense and sparse data, run on CPU and GPU instances 
on single machine and in distributed setting.
 * minimize the API differences with Python Module API.

Detailed descriptions of API changes and suggested usage can be found in the 
Design Proposal 
[here|https://cwiki.apache.org/confluence/display/MXNET/SVRG+Optimization+in+MXNet+Python+Module].
 

  was:
SVRG stands for Stochastic Variance Reduced Gradients, which is an optimization 
technique that employs explicit variance reduction. It has provable guarantees 
for strong smoothly convex functions and converges faster compared to SGD. 

The proposal is to create a SVRGModule that implements the SVRG optimization 
technique under the hood. 

SVRG Module should:
 * seamlessly supports both dense and sparse data, run on CPU and GPU instances 
on single machine and in distributed setting.
 * minimize the API differences with Python Module API.

Design Proposal can be found 
[here|https://cwiki.apache.org/confluence/display/MXNET/SVRG+Optimization+in+MXNet+Python+Module].
 


> SVRG Optimization in Python Module API
> --------------------------------------
>
>                 Key: MXNET-854
>                 URL: https://issues.apache.org/jira/browse/MXNET-854
>             Project: Apache MXNet
>          Issue Type: New Feature
>            Reporter: Stephanie Yuan
>            Priority: Major
>
> SVRG stands for Stochastic Variance Reduced Gradients, which is an 
> optimization technique that employs explicit variance reduction. It has 
> provable guarantees for strong smoothly convex functions and converges faster 
> compared to SGD. 
> The proposal is to create a SVRGModule that implements the SVRG optimization 
> technique under the hood. 
> SVRG Module should:
>  * seamlessly supports both dense and sparse data, run on CPU and GPU 
> instances on single machine and in distributed setting.
>  * minimize the API differences with Python Module API.
> Detailed descriptions of API changes and suggested usage can be found in the 
> Design Proposal 
> [here|https://cwiki.apache.org/confluence/display/MXNET/SVRG+Optimization+in+MXNet+Python+Module].
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to