merrymercy commented on a change in pull request #7163:
URL: https://github.com/apache/tvm/pull/7163#discussion_r548783676
##########
File path: python/tvm/relay/op/nn/nn.py
##########
@@ -698,6 +698,29 @@ def softmax(data, axis=-1):
return _make.softmax(data, axis)
+def fast_softmax(data, axis=-1):
+ r"""Computes softmax.
+ Use approximation to compute exponent for faster speed.
+
+ .. math:: \text{softmax}(x)_i = \frac{exp(x_i)}{\sum_j exp(x_j)}
Review comment:
The math formula is still the same, but we use some approximation to
compute exp.
The approximation cannot be described as short formulas, so we only
mentioned this by text in the doc string at L703.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
