ptrendx commented on issue #15545: Softmax fwd optimization for GPU URL: https://github.com/apache/incubator-mxnet/pull/15545#issuecomment-513884981 @haojin2 For the perf improvement on end to end training - I did not measure yet, measured just the kernel speedup for now. BTW - how important do you think safe accumulation option is (as in, having an option to NOT do it instead of just always doing safe accumulation)? Personally I don't see value in not using it in softmax as it would most probably affect accuracy, and having 3 TYPE_SWITCH makes the compilation time quite big. You did not put the ability to skip it in `softmax_with_length`, should I remove it from regular softmax as well?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
