marcoabreu commented on issue #11395: Check failed: e == cudaSuccess CUDA: unspecified launch failure URL: https://github.com/apache/incubator-mxnet/issues/11395#issuecomment-400858626 This seems to be caused by test_conv: http://jenkins.mxnet-ci.amazon-ml.com/blue/rest/organizations/jenkins/pipelines/incubator-mxnet/branches/master/runs/1050/nodes/752/steps/1370/log/?start=0 ``` test_operator_gpu.test_ndarray_crop ... ok test_operator_gpu.test_cell_fill_shape ... ok test_operator_gpu.test_conv ... [00:06:32] c:\jenkins_slave\workspace\build-gpu\src\operator\nn\cudnn\./cudnn_algoreg-inl.h:107: Running performance tests to find the best convolution algorithm, this can take a while... (setting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable) [00:06:32] C:/jenkins_slave/workspace/build-gpu/src/operator/nn/convolution.cu:148: This convolution is not supported by cudnn, MXNET convolution is applied. [00:06:32] C:/jenkins_slave/workspace/build-gpu/src/operator/nn/convolution.cu:227: This convolution is not supported by cudnn, MXNET convolution is applied. [00:06:32] C:/jenkins_slave/workspace/build-gpu/src/operator/nn/convolution.cu:148: This convolution is not supported by cudnn, MXNET convolution is applied. [00:06:32] C:/jenkins_slave/workspace/build-gpu/src/operator/nn/convolution.cu:227: This convolution is not supported by cudnn, MXNET convolution is applied. [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=448545343 to reproduce. [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1330733257 to reproduce. ERROR test_operator_gpu.test_layer_fill_shape ... ERROR test_operator_gpu.test_normal_generator ... [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1592512793 to reproduce. ERROR test_operator_gpu.test_ndarray_concatenate ... [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=466881030 to reproduce. ERROR test_operator_gpu.test_sparse_nd_transpose ... [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=221598447 to reproduce. ERROR test_operator_gpu.test_sparse_nd_storage_fallback ... [00:06:32] c:\jenkins_slave\workspace\build-gpu\src\operator\../common/utils.h:417: Storage type fallback detected: operator = broadcast_add input storage types = [default, default, ] output storage types = [csr, ] params = {} context.dev_mask = gpu The operator with default storage type will be dispatched for execution. You're seeing this warning message because the operator above is unable to process the given ndarrays with specified storage types, context and parameter. Temporary dense ndarrays are generated in order to execute the operator. This does not affect the correctness of the programme. You can set environment variable MXNET_STORAGE_FALLBACK_LOG_VERBOSE to 0 to suppress this warning. [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1026960206 to reproduce. ERROR test_operator_gpu.test_clip ... [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1647052128 to reproduce. ERROR test_operator_gpu.test_convolution_with_type ... [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=1234 to reproduce. ERROR ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
