anko-intel opened a new issue #18610:
URL: https://github.com/apache/incubator-mxnet/issues/18610


   ## Description
   During investigation of #14357 I have found that running_mean and 
running_var variables are sometimes not yet calculated when converted to NumPy.
   
   ### Error Message
   When runing the python test from 
https://github.com/apache/incubator-mxnet/issues/14357#issuecomment-644887485 
sometimes there is information that consecutive reads of the same variable 
differes, which means that backward pass is calculated after the first read:
   `var1 = layer2.running_var.data().asnumpy()`
   It shouldn't happen as asnumpy() function should force all needed 
calculations to get running_var values.
   I such case the test shows a  message similar to the one below:
   ```
   seed: 331752807
   cpu(0) - DIFF in running_var reads:
      0 : [3.9974644 3.9977627 3.997775 ] 
      2 : [3.9962945 4.0025434 3.9987571]
   cpu(0) [3.9962945 4.0025434 3.9987571] [10.001289  9.997483 10.002162]
   gpu(0) [3.9962947 4.0025434 3.9987576] [10.001292  9.997481 10.002162]
   ```
   Verified on master branch 959143696
   
   
   ## To Reproduce
   run python test from 
https://github.com/apache/incubator-mxnet/issues/14357#issuecomment-644887485
   
   ## What have you tried to solve it?
   
   I already have the fix. I will put it to github soon.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Reply via email to