[
https://issues.apache.org/jira/browse/HIVEMALL-222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16661796#comment-16661796
] Makoto Yui edited comment on HIVEMALL-222 at 10/24/18 6:33 AM: --------------------------------------------------------------- Clip by value [https://github.com/tensorflow/tensorflow/blob/r1.11/tensorflow/python/ops/clip_ops.py#L37] [https://github.com/scikit-learn/scikit-learn/blob/0fc7ce6bb8bb6b5b98c66ad2c0a009753def945a/sklearn/linear_model/sgd_fast.pyx#L701] Clip by l2 norm [https://github.com/tensorflow/tensorflow/blob/r1.11/tensorflow/python/ops/clip_ops.py#L110] [http://www.cs.toronto.edu/~rgrosse/courses/csc321_2017/readings/L15%20Exploding%20and%20Vanishing%20Gradients.pdf] was (Author: myui): Clip by value [https://github.com/tensorflow/tensorflow/blob/r1.11/tensorflow/python/ops/clip_ops.py#L37 ] [https://github.com/scikit-learn/scikit-learn/blob/0fc7ce6bb8bb6b5b98c66ad2c0a009753def945a/sklearn/linear_model/sgd_fast.pyx#L701] Clip by l2 norm [https://github.com/tensorflow/tensorflow/blob/r1.11/tensorflow/python/ops/clip_ops.py#L110 ] [http://www.cs.toronto.edu/~rgrosse/courses/csc321_2017/readings/L15%20Exploding%20and%20Vanishing%20Gradients.pdf] > Introduce Gradient Clipping to avoid exploding gradient to General > Classifier/Regressor > --------------------------------------------------------------------------------------- > > Key: HIVEMALL-222 > URL: https://issues.apache.org/jira/browse/HIVEMALL-222 > Project: Hivemall > Issue Type: Improvement > Affects Versions: 0.5.0 > Reporter: Makoto Yui > Assignee: Makoto Yui > Priority: Minor > Fix For: 0.5.2 > > > Gradient Clipping is useful for avoiding exploding gradients > [https://github.com/scikit-learn/scikit-learn/blob/0fc7ce6bb8bb6b5b98c66ad2c0a009753def945a/sklearn/linear_model/sgd_fast.pyx#L701] > So, implement it for General Classifier/Regressor -- This message was sent by Atlassian JIRA (v7.6.3#76005)
