yiwuyao3863 opened a new issue #15583: forward error of mxnet-cu100 with version 1.4.1 URL: https://github.com/apache/incubator-mxnet/issues/15583 When I run the CNN model (resnet50) on GPU with mxnet-cu100, the error occurs as below: Traceback (most recent call last): File "test_score.py", line 84, in <module> test_imagenet1k(model=args.model, task_mode=args.task_mode, **kwargs) File "test_score.py", line 39, in test_imagenet1k score(model=m, rgb_mean=mean, rgb_std=std, scale=s, num_layer=nl, label_name=l, **kwargs) File "/app/score.py", line 225, in score predicts = mod.get_outputs()[0].asnumpy() File "/usr/local/lib/python2.7/dist-packages/mxnet/ndarray/ndarray.py", line 1980, in asnumpy ctypes.c_size_t(data.size))) File "/usr/local/lib/python2.7/dist-packages/mxnet/base.py", line 252, in check_call raise MXNetError(py_str(_LIB.MXGetLastError())) mxnet.base.MXNetError: [03:20:41] src/operator/nn/./cudnn/cudnn_convolution-inl.h:159: Check failed: e == CUDNN_STATUS_SUCCESS (7 vs. 0) cuDNN: CUDNN_STATUS_MAPPING_ERROR Stack trace returned 10 entries: [bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet/libmxnet.so(+0x4114ba) [0x7f5c93d054ba] [bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet/libmxnet.so(+0x411ad1) [0x7f5c93d05ad1] [bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet/libmxnet.so(+0x3765922) [0x7f5c97059922] [bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet/libmxnet.so(+0x3795cd9) [0x7f5c97089cd9] [bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet/libmxnet.so(+0x2ceb179) [0x7f5c965df179] [bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet/libmxnet.so(+0x2cf1ec6) [0x7f5c965e5ec6] [bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet/libmxnet.so(+0x2cd01c4) [0x7f5c965c41c4] [bt] (7) /usr/local/lib/python2.7/dist-packages/mxnet/libmxnet.so(+0x2cd44b3) [0x7f5c965c84b3] [bt] (8) /usr/local/lib/python2.7/dist-packages/mxnet/libmxnet.so(+0x2cd4706) [0x7f5c965c8706] [bt] (9) /usr/local/lib/python2.7/dist-packages/mxnet/libmxnet.so(+0x2cd08f4) [0x7f5c965c48f4] The setting of environment are: CUDA10.0 cuDNN7.5 or cuDNN7.6 mxnet-cu100 1.4.1 GPU driver 410.93 2080ti
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
