ptrendx commented on pull request #18905: URL: https://github.com/apache/incubator-mxnet/pull/18905#issuecomment-675726377
The strides change does not really change anything, you are right - this change is here just because cuDNN internally does tests only with the bias strides in the form of (C, 1, C, C), so with this change we are safer. As for performance impact - this change was prompted by tests of NVIDIA MLPerf SSD submissions, where we were comparing MXNet and pyTorch performance. The difference in bias gradient performance between those 2 implementations was 4x and this turned out to be the difference in order of wgrad and bgrad. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
