indhub commented on issue #10437: [MXNET-171] Fix a bug that was causing 
training accuracy to be printed as nan sometimes
URL: https://github.com/apache/incubator-mxnet/pull/10437#issuecomment-379197228
 
 
   Output of [finetune 
notebook](https://github.com/dmlc/mxnet-notebooks/blob/master/python/how_to/finetune.ipynb)
 before fix:
   
   ```
   2018-04-03 23:40:21,316 Epoch[0] Batch [10]  Speed: 368.51 samples/sec       
accuracy=0.002131
   2018-04-03 23:40:24,773 Epoch[0] Batch [20]  Speed: 370.50 samples/sec       
accuracy=0.007812
   2018-04-03 23:40:28,226 Epoch[0] Batch [30]  Speed: 370.75 samples/sec       
accuracy=0.011719
   2018-04-03 23:40:31,684 Epoch[0] Batch [40]  Speed: 370.26 samples/sec       
accuracy=0.014063
   2018-04-03 23:40:35,161 Epoch[0] Batch [50]  Speed: 368.28 samples/sec       
accuracy=0.026562
   2018-04-03 23:40:38,624 Epoch[0] Batch [60]  Speed: 369.71 samples/sec       
accuracy=0.026562
   2018-04-03 23:40:42,108 Epoch[0] Batch [70]  Speed: 367.54 samples/sec       
accuracy=0.060937
   2018-04-03 23:40:45,583 Epoch[0] Batch [80]  Speed: 368.39 samples/sec       
accuracy=0.094531
   2018-04-03 23:40:49,061 Epoch[0] Batch [90]  Speed: 368.16 samples/sec       
accuracy=0.101562
   2018-04-03 23:40:52,542 Epoch[0] Batch [100] Speed: 367.77 samples/sec       
accuracy=0.135937
   2018-04-03 23:40:56,040 Epoch[0] Batch [110] Speed: 366.01 samples/sec       
accuracy=0.135937
   2018-04-03 23:40:59,541 Epoch[0] Batch [120] Speed: 365.68 samples/sec       
accuracy=0.171875
   2018-04-03 23:40:59,542 Epoch[0] Train-accuracy=nan
   2018-04-03 23:40:59,543 Epoch[0] Time cost=85.549
   2018-04-03 23:41:13,888 Epoch[0] Validation-accuracy=0.237001
   ```
   
   
   Output of [finetune 
notebook](https://github.com/dmlc/mxnet-notebooks/blob/master/python/how_to/finetune.ipynb)
 after fix:
   
   ```
   2018-04-06 08:45:23,057 Epoch[0] Batch [10]  Speed: 366.62 samples/sec       
accuracy=0.003551
   2018-04-06 08:45:26,538 Epoch[0] Batch [20]  Speed: 367.82 samples/sec       
accuracy=0.006250
   2018-04-06 08:45:30,026 Epoch[0] Batch [30]  Speed: 367.14 samples/sec       
accuracy=0.011719
   2018-04-06 08:45:33,512 Epoch[0] Batch [40]  Speed: 367.24 samples/sec       
accuracy=0.025781
   2018-04-06 08:45:36,997 Epoch[0] Batch [50]  Speed: 367.43 samples/sec       
accuracy=0.027344
   2018-04-06 08:45:40,489 Epoch[0] Batch [60]  Speed: 366.63 samples/sec       
accuracy=0.032031
   2018-04-06 08:45:43,973 Epoch[0] Batch [70]  Speed: 367.42 samples/sec       
accuracy=0.056250
   2018-04-06 08:45:47,469 Epoch[0] Batch [80]  Speed: 366.36 samples/sec       
accuracy=0.085156
   2018-04-06 08:45:50,969 Epoch[0] Batch [90]  Speed: 365.75 samples/sec       
accuracy=0.089844
   2018-04-06 08:45:54,482 Epoch[0] Batch [100] Speed: 364.56 samples/sec       
accuracy=0.123438
   2018-04-06 08:45:57,982 Epoch[0] Batch [110] Speed: 365.80 samples/sec       
accuracy=0.133594
   2018-04-06 08:46:01,484 Epoch[0] Batch [120] Speed: 365.58 samples/sec       
accuracy=0.184375
   2018-04-06 08:46:01,485 Epoch[0] Train-accuracy=0.184375
   2018-04-06 08:46:01,486 Epoch[0] Time cost=83.268
   2018-04-06 08:46:15,864 Epoch[0] Validation-accuracy=0.234112
   ```
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Reply via email to