[
https://issues.apache.org/jira/browse/MXNET-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641222#comment-16641222
]
Jay Vercellone commented on MXNET-688:
--------------------------------------
[~oneraynyday] the referenced PR was merged back in July. Is this issue solved?
> Fix quantization divide by zero errors
> --------------------------------------
>
> Key: MXNET-688
> URL: https://issues.apache.org/jira/browse/MXNET-688
> Project: Apache MXNet
> Issue Type: Bug
> Reporter: Ray Zhang
> Priority: Critical
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> The current quantization strategy for `calib_mode='entropy'` is to calculate
> the KL divergence for different thresholds and choose the best threshold.
> This assumes that the random variable is nonzero for all reals and is a
> continuous random variable. Because we are discretizing the distribution, we
> smooth the distribution over the range `[-threshold, threshold]`. What we are
> not considering is that the entire sampled distribution may be not in the
> range `[-threshold, threshold]` and thus we end up with all zeros in the
> sampled candidate `p` distribution inside of `_get_optimal_threshold`.
> I have added a check that the distribution(possibly unnormalized) is proper
> before attempting to smooth or else we'll run into a divide by 0 error.
> In most cases, activation functions and layers for classification type
> problems output numbers symmetric around 0. This is not the case for a
> regressor's last layer, and there are various other examples where the
> activation distribution is not around 0, and this was a major blockage for
> airbnb's adoption into mxnet's quantization capabilities.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
