leezu commented on a change in pull request #12042: Make check_isfinite,
check_scale optional in clip_global_norm
URL: https://github.com/apache/incubator-mxnet/pull/12042#discussion_r212812550
##########
File path: python/mxnet/gluon/utils.py
##########
@@ -115,8 +115,21 @@ def split_and_load(data, ctx_list, batch_axis=0,
even_split=True):
return [i.as_in_context(ctx) for i, ctx in zip(slices, ctx_list)]
-def clip_global_norm(arrays, max_norm):
+def clip_global_norm(arrays, max_norm, check_isfinite=True, check_scale=True):
"""Rescales NDArrays so that the sum of their 2-norm is smaller than
`max_norm`.
+
+ Parameters
+ ----------
+ arrays : list of NDArray
+ max_norm : float
+ check_isfinite : bool, default True
+ If True, check that the total_norm is finite (not nan or inf). This
+ requires a blocking .asscalar() call.
+ check_scale : bool, default True
+ If True, skip array rescaling if max_norm / total_norm >= 1. This
+ requires a blocking call. If False, rescale arrays with min(1,
Review comment:
`contrib.cond` would only help when working with the symbolic interface. For
ndarray, `contrib.cond` also uses a blocking `.asscalar()` call.
`clip_global_norm` always works with the ndarray API.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
