wkcn edited a comment on pull request #17975: URL: https://github.com/apache/incubator-mxnet/pull/17975#issuecomment-692381230
A flaky test for batch norm https://jenkins.mxnet-ci.amazon-ml.com/blue/rest/organizations/jenkins/pipelines/mxnet-validation/pipelines/windows-gpu/branches/PR-17975/runs/13/nodes/109/steps/155/log/?start=0 ``` [2020-09-14T12:50:59.110Z] DEBUG:common:Setting test np/mx/python random seeds, use MXNET_TEST_SEED=476912465 to reproduce. [2020-09-14T12:50:59.110Z] [2020-09-14T12:50:59.110Z] [gw3] [ 59%] FAILED tests/python/unittest/test_numpy_op.py::test_npx_batch_norm[True-False-False-shape0] [2020-09-14T13:07:56.780Z] ================================== FAILURES =================================== [2020-09-14T13:07:56.780Z] ________________ test_npx_batch_norm[True-False-False-shape0] _________________ [2020-09-14T13:07:56.780Z] [gw3] win32 -- Python 3.7.3 C:\Python37\python.exe [2020-09-14T13:07:56.780Z] [2020-09-14T13:07:56.780Z] shape = (4, 2), fix_gamma = False, cudnn_off = False, output_mean_var = True [2020-09-14T13:07:56.781Z] > data_grad_req, gamma_grad_req, beta_grad_req) [2020-09-14T13:07:56.781Z] [2020-09-14T13:07:56.781Z] tests\python\unittest\test_numpy_op.py:1916: [2020-09-14T13:07:56.781Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [2020-09-14T13:07:56.781Z] tests\python\unittest\test_numpy_op.py:1892: in _test_batchnorm_impl [2020-09-14T13:07:56.781Z] ), running_var.asnumpy(), atol=atol, rtol=rtol) 2020-09-14T13:07:56.781Z] > raise AssertionError(msg) [2020-09-14T13:07:56.781Z] E AssertionError: [2020-09-14T13:07:56.781Z] E Items are not equal: [2020-09-14T13:07:56.781Z] E Error 1.046710 exceeds tolerance rtol=5.000000e-02, atol=5.000000e-02 (mismatch 25.000000%). [2020-09-14T13:07:56.781Z] E Location of maximum error: (1,), a=0.42751396, b=0.50634950 [2020-09-14T13:07:56.781Z] E ACTUAL: array([0.36993775, 0.42751396, 0.3712832 , 0.36561143], dtype=float32) [2020-09-14T13:07:56.781Z] E DESIRED: array([0.39119717, 0.5063495 , 0.3938881 , 0.3825443 ], dtype=float32) ``` ``` [2020-09-14T12:50:59.110Z] [gw3] [ 59%] FAILED tests/python/unittest/test_numpy_op.py::test_npx_batch_norm[True-False-False-shape0] [2020-09-14T12:50:42.201Z] [gw3] [ 56%] PASSED tests/python/unittest/test_numpy_op.py::test_npx_batch_norm[False-False-False-shape0] [2020-09-14T12:50:42.461Z] [gw3] [ 57%] PASSED tests/python/unittest/test_numpy_op.py::test_npx_batch_norm[False-False-False-shape1] [2020-09-14T12:50:42.461Z] [gw3] [ 57%] PASSED tests/python/unittest/test_numpy_op.py::test_npx_batch_norm[False-False-False-shape1] [2020-09-14T12:50:59.631Z] [gw3] [ 59%] PASSED tests/python/unittest/test_numpy_op.py::test_npx_batch_norm[True-False-False-shape1] ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
