thomelane commented on issue #11508: 【Question】Are there any examples for gradients clipping in gluon? URL: https://github.com/apache/incubator-mxnet/issues/11508#issuecomment-404675656 @yukang2017 would you be able to open a separate issue for the performance issue? that way we can keep api usage separate from performance, and it can be tagged correctly for better community help. thanks! With regards to how to perform gradient clipping with Gluon, another option is to specify gradient clipping as part of the `Optimizer` given to the `Trainer`. Although this will simply clip by value: potentially changing the direction of the gradient tensor and effecting different gradient tensors in different ways. So overall, `gluon.utils.clip_global_norm` is the best for maintaining direction and relative magnitudes of gradient tensors. Still, this is an example of how value gradient clipping can be done: ``` mxnet.gluon.Trainer(net.collect_params(), optimizer='sgd', optimizer_params={'learning_rate': 0.1, 'clip_gradient':5}, kvstore='device') #for GPU ``` When elements of the gradient tensor are: * less than -5, they will be set to -5, * greater than +5, they will be set to +5.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
