matteosal commented on issue #21111: URL: https://github.com/apache/incubator-mxnet/issues/21111#issuecomment-1206387032
@DickJC123 1) Training would obviously be the same, but there is a corner case when one might want to perform several forward passes in training mode without doing backwards. In this case the cudnn implementation would behave differently than the default GPU one and the CPU one as well 2) Yes beta and gamma are changed by the optimizer, not by the code in these examples. There is nothing wrong with them 3) Yes, it's supposed to be equivalent to [1, 6, 1]. So the fact that the cudnn variance is unbiased seems to explain the numerical error. I will make a few tests of this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@mxnet.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@mxnet.apache.org For additional commands, e-mail: issues-h...@mxnet.apache.org
