leezu opened a new issue #19563:
URL: https://github.com/apache/incubator-mxnet/issues/19563


   It was run after test_psroipooling and custom op tests
   
   ```
   [2020-11-18T04:20:50.688Z] [gw1] [ 34%] PASSED 
tests/python/gpu/test_operator_gpu.py::test_rcbrt_op 
   [2020-11-18T04:20:50.942Z] 
tests/python/gpu/test_operator_gpu.py::test_custom_op 
   [2020-11-18T04:20:52.011Z] [gw1] [ 35%] PASSED 
tests/python/gpu/test_operator_gpu.py::test_custom_op 
   [2020-11-18T04:20:52.011Z] 
tests/python/gpu/test_operator_gpu.py::test_custom_op_exc 
   [2020-11-18T04:20:52.011Z] [gw1] [ 35%] PASSED 
tests/python/gpu/test_operator_gpu.py::test_custom_op_exc 
   [2020-11-18T04:20:52.011Z] 
tests/python/gpu/test_operator_gpu.py::test_psroipooling 
   [2020-11-18T04:25:00.072Z] [gw1] [ 89%] PASSED 
tests/python/gpu/test_operator_gpu.py::test_psroipooling 
   [2020-11-18T04:25:02.581Z] 
tests/python/gpu/test_operator_gpu.py::test_deformable_convolution 
   [2020-11-18T04:45:13.758Z] Timeout (0:20:00)!
   [2020-11-18T04:45:13.758Z] Thread 0x00007f5cbcf54700 (most recent call 
first):
   [2020-11-18T04:45:13.758Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py",
 line 400 in read
   [2020-11-18T04:45:13.758Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py",
 line 432 in from_io
   [2020-11-18T04:45:13.758Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py",
 line 967 in _thread_receiver
   [2020-11-18T04:45:13.758Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py",
 line 220 in run
   [2020-11-18T04:45:13.758Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py",
 line 285 in _perform_spawn
   [2020-11-18T04:45:13.758Z] 
   [2020-11-18T04:45:13.758Z] Thread 0x00007f5cbec25740 (most recent call 
first):
   [2020-11-18T04:45:13.758Z]   File 
"/opt/rh/rh-python36/root/usr/lib64/python3.6/pathlib.py", line 649 in 
_parse_args
   [2020-11-18T04:45:13.758Z]   File 
"/opt/rh/rh-python36/root/usr/lib64/python3.6/pathlib.py", line 662 in 
_from_parts
   [2020-11-18T04:45:13.758Z]   File 
"/opt/rh/rh-python36/root/usr/lib64/python3.6/pathlib.py", line 624 in __new__
   [2020-11-18T04:45:13.758Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/pathlib.py", 
line 411 in fnmatch_ex
   [2020-11-18T04:45:13.758Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/assertion/rewrite.py",
 line 210 in _early_rewrite_bailout
   [2020-11-18T04:45:13.758Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/assertion/rewrite.py",
 line 93 in find_spec
   [2020-11-18T04:45:13.758Z]   File "<frozen importlib._bootstrap>", line 894 
in _find_spec
   [2020-11-18T04:45:13.758Z]   File "<frozen importlib._bootstrap>", line 951 
in _find_and_load_unlocked
   [2020-11-18T04:45:13.758Z]   File "<frozen importlib._bootstrap>", line 971 
in _find_and_load
   [2020-11-18T04:45:13.758Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/py/_vendored_packages/apipkg.py",
 line 69 in importobj
   [2020-11-18T04:45:13.758Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/py/_vendored_packages/apipkg.py",
 line 148 in __makeattr
   [2020-11-18T04:45:13.758Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pytest_timeout.py", 
line 444 in write_title
   [2020-11-18T04:45:13.758Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pytest_timeout.py", 
line 432 in dump_stacks
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pytest_timeout.py", 
line 372 in timeout_sigalrm
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pytest_timeout.py", 
line 200 in handler
   [2020-11-18T04:45:13.759Z]   File 
"/work/mxnet/python/mxnet/_ctypes/ndarray.py", line 181 in __call__
   [2020-11-18T04:45:13.759Z]   File "/work/mxnet/python/mxnet/executor.py", 
line 184 in forward
   [2020-11-18T04:45:13.759Z]   File "/work/mxnet/python/mxnet/test_utils.py", 
line 1023 in numeric_grad
   [2020-11-18T04:45:13.759Z]   File "/work/mxnet/python/mxnet/test_utils.py", 
line 1173 in check_numeric_gradient
   [2020-11-18T04:45:13.759Z]   File 
"/work/mxnet/tests/python/unittest/test_operator.py", line 5708 in 
test_deformable_convolution
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/python.py", 
line 184 in pytest_pyfunc_call
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/callers.py", 
line 192 in _multicall
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", 
line 87 in <lambda>
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", 
line 93 in _hookexec
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/hooks.py", 
line 286 in __call__
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/python.py", 
line 1627 in runtest
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", 
line 171 in pytest_runtest_call
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/callers.py", 
line 208 in _multicall
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", 
line 87 in <lambda>
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", 
line 93 in _hookexec
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/hooks.py", 
line 286 in __call__
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", 
line 256 in <lambda>
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", 
line 326 in from_call
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", 
line 256 in call_runtest_hook
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/flaky/flaky_pytest_plugin.py",
 line 141 in call_and_report
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", 
line 127 in runtestprotocol
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/runner.py", 
line 110 in pytest_runtest_protocol
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/flaky/flaky_pytest_plugin.py",
 line 94 in pytest_runtest_protocol
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/callers.py", 
line 187 in _multicall
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", 
line 87 in <lambda>
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", 
line 93 in _hookexec
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/hooks.py", 
line 286 in __call__
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/xdist/remote.py", 
line 87 in run_one_test
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/xdist/remote.py", 
line 70 in pytest_runtestloop
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/callers.py", 
line 187 in _multicall
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", 
line 87 in <lambda>
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", 
line 93 in _hookexec
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/hooks.py", 
line 286 in __call__
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/main.py", 
line 313 in _main
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/main.py", 
line 257 in wrap_session
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/_pytest/main.py", 
line 306 in pytest_cmdline_main
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/callers.py", 
line 187 in _multicall
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", 
line 87 in <lambda>
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/manager.py", 
line 93 in _hookexec
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pluggy/hooks.py", 
line 286 in __call__
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/xdist/remote.py", 
line 237 in <module>
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py",
 line 1084 in executetask
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py",
 line 220 in run
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py",
 line 285 in _perform_spawn
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py",
 line 267 in integrate_as_primary_thread
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py",
 line 1060 in serve
   [2020-11-18T04:45:13.759Z]   File 
"/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/execnet/gateway_base.py",
 line 1554 in serve
   [2020-11-18T04:45:13.759Z]   File "<string>", line 8 in <module>
   [2020-11-18T04:45:13.759Z]   File "<string>", line 1 in <module>
   [2020-11-18T04:45:13.759Z] [gw1] [ 93%] FAILED 
tests/python/gpu/test_operator_gpu.py::test_deformable_convolution 
   ```
   
   
https://jenkins.mxnet-ci.amazon-ml.com/blue/rest/organizations/jenkins/pipelines/mxnet-validation/pipelines/centos-gpu/branches/master/runs/2393/nodes/143/steps/215/log/?start=0


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[email protected]



---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to