matteosal commented on issue #21111: URL: https://github.com/apache/incubator-mxnet/issues/21111#issuecomment-1235713824
So, to summarize the issues I found with the cuDNN implementation: 1) Moving arrays are normally updated only if a BACKWARD pass in training mode is performed, but on GPU + cudnn they are changed by the FORWARD 2) In training mode, all implementations compute the **biased** data variance during the forward but the cuDNN implementation uses the **unbiased** data variance to update the moving variance So the cuDNN implementation updates the moving variance using a different value (the unbiased one) and also at a different time (during the forward) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@mxnet.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@mxnet.apache.org For additional commands, e-mail: issues-h...@mxnet.apache.org
