marcoabreu opened a new issue #11395: Check failed: e == cudaSuccess CUDA: unspecified launch failure URL: https://github.com/apache/incubator-mxnet/issues/11395 Sometimes, our slaves get corrupted and suddenly all test start to fail. This is unrelated to the tests directly. http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-11377/5/pipeline/ ``` ====================================================================== ERROR: test_operator_gpu.test_op_roi_align ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Anaconda3\envs\py2\lib\site-packages\nose\case.py", line 197, in runTest self.test(*self.arg) File "C:\Anaconda3\envs\py2\lib\site-packages\nose\util.py", line 620, in newfunc return func(*arg, **kw) File "C:\jenkins_slave\workspace\ut-python-gpu\tests\python\gpu\../unittest\common.py", line 157, in test_new orig_test(*args, **kwargs) File "C:\jenkins_slave\workspace\ut-python-gpu\tests\python\gpu\../unittest\test_operator.py", line 6269, in test_op_roi_align test_roi_align_value() File "C:\jenkins_slave\workspace\ut-python-gpu\tests\python\gpu\../unittest\test_operator.py", line 6230, in test_roi_align_value data = mx.nd.array(np.arange(N*C*W*H).reshape((N,C,H,W)), ctx=ctx, dtype = dtype) File "C:\jenkins_slave\workspace\ut-python-gpu\pkg_vc14_gpu\python\mxnet\ndarray\utils.py", line 146, in array return _array(source_array, ctx=ctx, dtype=dtype) File "C:\jenkins_slave\workspace\ut-python-gpu\pkg_vc14_gpu\python\mxnet\ndarray\ndarray.py", line 2357, in array arr[:] = source_array File "C:\jenkins_slave\workspace\ut-python-gpu\pkg_vc14_gpu\python\mxnet\ndarray\ndarray.py", line 444, in __setitem__ self._set_nd_basic_indexing(key, value) File "C:\jenkins_slave\workspace\ut-python-gpu\pkg_vc14_gpu\python\mxnet\ndarray\ndarray.py", line 710, in _set_nd_basic_indexing self._sync_copyfrom(value) File "C:\jenkins_slave\workspace\ut-python-gpu\pkg_vc14_gpu\python\mxnet\ndarray\ndarray.py", line 876, in _sync_copyfrom ctypes.c_size_t(source_array.size))) File "C:\jenkins_slave\workspace\ut-python-gpu\pkg_vc14_gpu\python\mxnet\base.py", line 210, in check_call raise MXNetError(py_str(_LIB.MXGetLastError())) MXNetError: [06:35:08] c:\jenkins_slave\workspace\build-gpu\3rdparty\mshadow\mshadow\./tensor_gpu-inl.h:69: Check failed: e == cudaSuccess CUDA: unspecified launch failure -------------------- >> begin captured logging << -------------------- common: INFO: Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1046236735 to reproduce. --------------------- >> end captured logging << --------------------- ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
