leezu opened a new issue #19563: URL: https://github.com/apache/incubator-mxnet/issues/19563
It was run after test_psroipooling and custom op tests ``` [2020-11-18T04:20:50.688Z] [gw1] [ 34%] PASSED tests/python/gpu/test_operator_gpu.py::test_rcbrt_op [2020-11-18T04:20:50.942Z] tests/python/gpu/test_operator_gpu.py::test_custom_op [2020-11-18T04:20:52.011Z] [gw1] [ 35%] PASSED tests/python/gpu/test_operator_gpu.py::test_custom_op [2020-11-18T04:20:52.011Z] tests/python/gpu/test_operator_gpu.py::test_custom_op_exc [2020-11-18T04:20:52.011Z] [gw1] [ 35%] PASSED tests/python/gpu/test_operator_gpu.py::test_custom_op_exc [2020-11-18T04:20:52.011Z] tests/python/gpu/test_operator_gpu.py::test_psroipooling [2020-11-18T04:25:00.072Z] [gw1] [ 89%] PASSED tests/python/gpu/test_operator_gpu.py::test_psroipooling [2020-11-18T04:25:02.581Z] tests/python/gpu/test_operator_gpu.py::test_deformable_convolution [2020-11-18T04:45:13.758Z] Timeout (0:20:00)! [2020-11-18T04:45:13.758Z] Thread 0x00007f5cbcf54700 (most recent call first): [2020-11-18T04:45:13.758Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py", line 400 in read [2020-11-18T04:45:13.758Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py", line 432 in from_io [2020-11-18T04:45:13.758Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py", line 967 in _thread_receiver [2020-11-18T04:45:13.758Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py", line 220 in run [2020-11-18T04:45:13.758Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py", line 285 in _perform_spawn [2020-11-18T04:45:13.758Z] [2020-11-18T04:45:13.758Z] Thread 0x00007f5cbec25740 (most recent call first): [2020-11-18T04:45:13.758Z] File "/opt/rh/rh-python36/root/usr/lib64/python3.6/pathlib.py", line 649 in _parse_args [2020-11-18T04:45:13.758Z] File "/opt/rh/rh-python36/root/usr/lib64/python3.6/pathlib.py", line 662 in _from_parts [2020-11-18T04:45:13.758Z] File "/opt/rh/rh-python36/root/usr/lib64/python3.6/pathlib.py", line 624 in __new__ [2020-11-18T04:45:13.758Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/pathlib.py", line 411 in fnmatch_ex [2020-11-18T04:45:13.758Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/assertion/rewrite.py", line 210 in _early_rewrite_bailout [2020-11-18T04:45:13.758Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/assertion/rewrite.py", line 93 in find_spec [2020-11-18T04:45:13.758Z] File "<frozen importlib._bootstrap>", line 894 in _find_spec [2020-11-18T04:45:13.758Z] File "<frozen importlib._bootstrap>", line 951 in _find_and_load_unlocked [2020-11-18T04:45:13.758Z] File "<frozen importlib._bootstrap>", line 971 in _find_and_load [2020-11-18T04:45:13.758Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/py/_vendored_packages/apipkg.py", line 69 in importobj [2020-11-18T04:45:13.758Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/py/_vendored_packages/apipkg.py", line 148 in __makeattr [2020-11-18T04:45:13.758Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pytest_timeout.py", line 444 in write_title [2020-11-18T04:45:13.758Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pytest_timeout.py", line 432 in dump_stacks [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pytest_timeout.py", line 372 in timeout_sigalrm [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pytest_timeout.py", line 200 in handler [2020-11-18T04:45:13.759Z] File "/work/mxnet/python/mxnet/_ctypes/ndarray.py", line 181 in __call__ [2020-11-18T04:45:13.759Z] File "/work/mxnet/python/mxnet/executor.py", line 184 in forward [2020-11-18T04:45:13.759Z] File "/work/mxnet/python/mxnet/test_utils.py", line 1023 in numeric_grad [2020-11-18T04:45:13.759Z] File "/work/mxnet/python/mxnet/test_utils.py", line 1173 in check_numeric_gradient [2020-11-18T04:45:13.759Z] File "/work/mxnet/tests/python/unittest/test_operator.py", line 5708 in test_deformable_convolution [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/python.py", line 184 in pytest_pyfunc_call [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/callers.py", line 192 in _multicall [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda> [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__ [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/python.py", line 1627 in runtest [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", line 171 in pytest_runtest_call [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/callers.py", line 208 in _multicall [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda> [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__ [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", line 256 in <lambda> [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", line 326 in from_call [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", line 256 in call_runtest_hook [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/flaky/flaky_pytest_plugin.py", line 141 in call_and_report [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", line 127 in runtestprotocol [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", line 110 in pytest_runtest_protocol [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/flaky/flaky_pytest_plugin.py", line 94 in pytest_runtest_protocol [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda> [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__ [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/xdist/remote.py", line 87 in run_one_test [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/xdist/remote.py", line 70 in pytest_runtestloop [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda> [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__ [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/main.py", line 313 in _main [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/main.py", line 257 in wrap_session [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/main.py", line 306 in pytest_cmdline_main [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/callers.py", line 187 in _multicall [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 87 in <lambda> [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", line 93 in _hookexec [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/hooks.py", line 286 in __call__ [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/xdist/remote.py", line 237 in <module> [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py", line 1084 in executetask [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py", line 220 in run [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py", line 285 in _perform_spawn [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py", line 267 in integrate_as_primary_thread [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py", line 1060 in serve [2020-11-18T04:45:13.759Z] File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py", line 1554 in serve [2020-11-18T04:45:13.759Z] File "<string>", line 8 in <module> [2020-11-18T04:45:13.759Z] File "<string>", line 1 in <module> [2020-11-18T04:45:13.759Z] [gw1] [ 93%] FAILED tests/python/gpu/test_operator_gpu.py::test_deformable_convolution ``` https://jenkins.mxnet-ci.amazon-ml.com/blue/rest/organizations/jenkins/pipelines/mxnet-validation/pipelines/centos-gpu/branches/master/runs/2393/nodes/143/steps/215/log/?start=0 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
