larroy opened a new issue #11973: control flow operator tests failing on Linux and Windows URL: https://github.com/apache/incubator-mxnet/issues/11973 Note: Providing complete information in the most concise form is the best way to get help. This issue template serves as the checklist for essential information to most of the technical issues and bug reports. For non-technical issues and feature requests, feel free to present the information in what you believe is the best form. For Q & A and discussion, please start a discussion thread at https://discuss.mxnet.io ## Description nosetests test_contrib_control_flow.py is failing in at least py3 in linux and py2 in windows ## Environment info (Required) ``` ----------Python Info---------- Version : 3.5.2 Compiler : GCC 5.4.0 20160609 Build : ('default', 'Nov 23 2017 16:37:01') Arch : ('64bit', 'ELF') ------------Pip Info----------- Version : 18.0 Directory : /home/piotr/devel/mxnet/mxnet/mxnet_py3/lib/python3.5/site-packages/pip ----------MXNet Info----------- Version : 1.3.0 Directory : /home/piotr/devel/mxnet/mxnet/python/mxnet Hashtag not found. Not installed from pre-built package. ----------System Info---------- Platform : Linux-4.15.0-24-generic-x86_64-with-Ubuntu-16.04-xenial system : Linux node : oruga release : 4.15.0-24-generic version : #26~16.04.1-Ubuntu SMP Fri Jun 15 14:35:08 UTC 2018 ----------Hardware Info---------- machine : x86_64 processor : x86_64 Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 32 On-line CPU(s) list: 0-31 Thread(s) per core: 2 Core(s) per socket: 16 Socket(s): 1 NUMA node(s): 1 Vendor ID: AuthenticAMD CPU family: 23 Model: 1 Model name: AMD Ryzen Threadripper 1950X 16-Core Processor Stepping: 1 CPU MHz: 1865.541 CPU max MHz: 3400.0000 CPU min MHz: 2200.0000 BogoMIPS: 6799.26 Virtualization: AMD-V L1d cache: 32K L1i cache: 64K L2 cache: 512K L3 cache: 8192K NUMA node0 CPU(s): 0-31 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid amd_dcm aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb hw_pstate sme ssbd vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 xsaves clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif overflow_recov succor smca ----------Network Test---------- Setting timeout: 10 Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0267 sec, LOAD: 0.5596 sec. Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.0295 sec, LOAD: 0.2071 sec. Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0384 sec, LOAD: 0.8882 sec. Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0285 sec, LOAD: 1.1026 sec. Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.0265 sec, LOAD: 0.0963 sec. Timing for Gluon Tutorial(cn): https://zh.gluon.ai, DNS: 0.1624 sec, LOAD: 0.8599 sec. ``` Package used (Python/R/Scala/Julia): (I'm using Python) ## Build info (Required if built from source) Compiler (gcc/clang/mingw/visual studio): gcc and vs 14 MXNet commit hash: 394e5ccc7937de61892e79253e6a8c05aebe9d43 Build config: ``` cmake\ -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \ -DCMAKE_C_COMPILER_LAUNCHER=ccache \ -DUSE_CPP_PACKAGE=ON\ -DUSE_CUDA=OFF\ -DUSE_OPENMP=OFF\ -DUSE_OPENCV=ON\ -DCMAKE_BUILD_TYPE=Debug\ -GNinja .. ninja -v ``` ## Error Message: ``` .[INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=133245677 to reproduce. F..[INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1862402888 to reproduce. F[INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1371884833 to reproduce. F[INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1164686348 to reproduce. F.[INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=275941523 to reproduce. F ====================================================================== FAIL: test_contrib_control_flow.test_while_loop_for_foreach ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/piotr/devel/mxnet/mxnet/mxnet_py3/lib/python3.5/site-packages/nose/case.py", line 198, in runTest self.test(*self.arg) File "/home/piotr/devel/mxnet/mxnet/tests/python/unittest/common.py", line 172, in test_new orig_test(*args, **kwargs) File "/home/piotr/devel/mxnet/mxnet/tests/python/unittest/test_contrib_control_flow.py", line 618, in test_while_loop_for_foreach n_steps=5, File "/home/piotr/devel/mxnet/mxnet/tests/python/unittest/test_contrib_control_flow.py", line 322, in case_1 **params File "/home/piotr/devel/mxnet/mxnet/tests/python/unittest/test_contrib_control_flow.py", line 240, in _verify_while_loop assert_almost_equal(imp_out, sym_out, rtol=1e-4, atol=1e-4) File "/home/piotr/devel/mxnet/mxnet/python/mxnet/test_utils.py", line 493, in assert_almost_equal raise AssertionError(msg) AssertionError: Items are not equal: Error 9993.736328 exceeds tolerance rtol=0.000100, atol=0.000100. Location of maximum error:(3, 0), a=-2.064602, b=-4891.381348 a: array([[ 0.96158886], [ 0.6512103 ], [ 1.7375352 ],... b: array([[ 127.01133], [ -379.67126], [ 1419.797 ],... -------------------- >> begin captured logging << -------------------- common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=133245677 to reproduce. --------------------- >> end captured logging << --------------------- ====================================================================== FAIL: test_contrib_control_flow.test_cond ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/piotr/devel/mxnet/mxnet/mxnet_py3/lib/python3.5/site-packages/nose/case.py", line 198, in runTest self.test(*self.arg) File "/home/piotr/devel/mxnet/mxnet/tests/python/unittest/common.py", line 172, in test_new orig_test(*args, **kwargs) File "/home/piotr/devel/mxnet/mxnet/tests/python/unittest/test_contrib_control_flow.py", line 1129, in test_cond (2, 3), File "/home/piotr/devel/mxnet/mxnet/tests/python/unittest/test_contrib_control_flow.py", line 1087, in run_case **params File "/home/piotr/devel/mxnet/mxnet/tests/python/unittest/test_contrib_control_flow.py", line 1056, in _verify_cond assert_almost_equal(imp_out, sym_out, rtol=1e-5, atol=1e-5) File "/home/piotr/devel/mxnet/mxnet/python/mxnet/test_utils.py", line 493, in assert_almost_equal raise AssertionError(msg) AssertionError: Items are not equal: Error 30851.714844 exceeds tolerance rtol=0.000010, atol=0.000010. Location of maximum error:(0, 0), a=2.513998, b=4.081830 a: array([[ 2.513998 , -0.06135673, -0.42358735], [ 0.13876215, -0.403529 , 1.133575 ]], dtype=float32) b: array([[ 4.0818295 , -0.19492993, -0.59344125], [ 0.35262597, -0.72699374, 1.6560557 ]], dtype=float32) -------------------- >> begin captured logging << -------------------- common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1862402888 to reproduce. --------------------- >> end captured logging << --------------------- ====================================================================== FAIL: test_contrib_control_flow.test_contrib_rnn ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/piotr/devel/mxnet/mxnet/mxnet_py3/lib/python3.5/site-packages/nose/case.py", line 198, in runTest self.test(*self.arg) File "/home/piotr/devel/mxnet/mxnet/tests/python/unittest/common.py", line 172, in test_new orig_test(*args, **kwargs) File "/home/piotr/devel/mxnet/mxnet/tests/python/unittest/test_contrib_control_flow.py", line 1193, in test_contrib_rnn check_contrib_rnn(cell_type, num_states) File "/home/piotr/devel/mxnet/mxnet/tests/python/unittest/test_contrib_control_flow.py", line 1177, in check_contrib_rnn assert_almost_equal(res1.asnumpy(), res2.asnumpy(), rtol=0.001, atol=0.0001) File "/home/piotr/devel/mxnet/mxnet/python/mxnet/test_utils.py", line 493, in assert_almost_equal raise AssertionError(msg) AssertionError: Items are not equal: Error 5429.805664 exceeds tolerance rtol=0.001000, atol=0.000100. Location of maximum error:(4, 2, 29), a=-0.566625, b=-0.003677 a: array([[[ 0.33987436, -0.12906979, 0.33146507, ..., 0.4379619 , -0.40905568, 0.17931955], [-0.40162748, 0.07519079, 0.19347169, ..., -0.31540644,... b: array([[[ 1. , 1. , 0.9999986 , ..., 1. , -0.9180124 , 1. ], [-1. , -0.99999005, -0.999999 , ..., -1. ,... -------------------- >> begin captured logging << -------------------- common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1371884833 to reproduce. --------------------- >> end captured logging << --------------------- ====================================================================== FAIL: test_contrib_control_flow.test_foreach ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/piotr/devel/mxnet/mxnet/mxnet_py3/lib/python3.5/site-packages/nose/case.py", line 198, in runTest self.test(*self.arg) File "/home/piotr/devel/mxnet/mxnet/tests/python/unittest/common.py", line 172, in test_new orig_test(*args, **kwargs) File "/home/piotr/devel/mxnet/mxnet/tests/python/unittest/test_contrib_control_flow.py", line 1324, in test_foreach lambda frees : [frees[0] + frees[1]]) File "/home/piotr/devel/mxnet/mxnet/tests/python/unittest/test_contrib_control_flow.py", line 1297, in verify_foreach rtol=0.001, atol=0.0001) File "/home/piotr/devel/mxnet/mxnet/python/mxnet/test_utils.py", line 493, in assert_almost_equal raise AssertionError(msg) AssertionError: Items are not equal: Error 17112.298828 exceeds tolerance rtol=0.001000, atol=0.000100. Location of maximum error:(2, 1), a=1016.000000, b=56.000000 a: array([[ 2., 18.], [ 18., 90.], [ 198., 1016.]], dtype=float32) b: array([[ 2., 12.], [12., 30.], [30., 56.]], dtype=float32) -------------------- >> begin captured logging << -------------------- common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1164686348 to reproduce. --------------------- >> end captured logging << --------------------- ====================================================================== FAIL: test_contrib_control_flow.test_foreach_rnn ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/piotr/devel/mxnet/mxnet/mxnet_py3/lib/python3.5/site-packages/nose/case.py", line 198, in runTest self.test(*self.arg) File "/home/piotr/devel/mxnet/mxnet/tests/python/unittest/common.py", line 172, in test_new orig_test(*args, **kwargs) File "/home/piotr/devel/mxnet/mxnet/tests/python/unittest/test_contrib_control_flow.py", line 1664, in test_foreach_rnn check_foreach_rnn(cell_type, num_states) File "/home/piotr/devel/mxnet/mxnet/tests/python/unittest/test_contrib_control_flow.py", line 1652, in check_foreach_rnn rtol=0.001, atol=0.0001) File "/home/piotr/devel/mxnet/mxnet/python/mxnet/test_utils.py", line 493, in assert_almost_equal raise AssertionError(msg) AssertionError: Items are not equal: Error 482.427795 exceeds tolerance rtol=0.001000, atol=0.000100. Location of maximum error:(0, 0), a=14.919886, b=28.919886 a: array([[14.919886, 16.264572, 18.878408, 20.891766], [14.76185 , 8.976585, 15.503097, 27.326567]], dtype=float32) b: array([[28.919886, 18.264572, 19.878408, 23.891766], [18.76185 , 15.976585, 23.503098, 21.326567]], dtype=float32) -------------------- >> begin captured logging << -------------------- common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=275941523 to reproduce. --------------------- >> end captured logging << --------------------- ---------------------------------------------------------------------- Ran 9 tests in 7.259s FAILED (failures=5) ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
