@StephanieYuan Also the way `svrg_optimizer.py` is written it leaves a few questions open.
>From your design document I understand that SVRGOptimizer cannot be used with >Module API which is why you wrote the SVRGModule API which will use >SVRGOptimizer under the hood instead of the usual SGD or other optimizers. Can >you explain why in greater detail? Also if this is indeed the case, then the **SVRGOptimizer class needs to be declared private using a leading underscore in its name**, else it will be very misleading to the user who might try to use SVRGOptimizer with the plain vanilla Module API. Also it should be made clear in the doc string for the SRVGOptimizer that this optimizer is to be used with the SVRGModule alone. Also you might want to consider moving the SVRGOptimizer to the `optimizer.py` file and keep SVRGModule alone in the `contrib` folder. But I am not sure of this, you might want to get a second opinion on this. @piiswrong @eric-haibin-lin [ Full content available at: https://github.com/apache/incubator-mxnet/pull/12376 ] This message was relayed via gitbox.apache.org for [email protected]
