fhieber edited a comment on pull request #18531: URL: https://github.com/apache/incubator-mxnet/pull/18531#issuecomment-645859028
Just to follow up on this and provide some numbers (@eric-haibin-lin). I tried the label smoothing implementation by @sxjscience (thanks!) in Sockeye, making sure scaling and normalization is equivalent to how we currently use SoftmaxOutput. While this is much faster than a naive implementation of label smoothing with dense, smoothed labels, SoftmaxOutput is still 5-10% faster than the optimized implementation. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
