ajosh0504 commented on issue #10323: test_gluon_model_zoo_gpu.test_training out of memory URL: https://github.com/apache/incubator-mxnet/issues/10323#issuecomment-418810994 Not sure what's going on- I'm trying to train and test a CNN on MXNet: File "textcnn.py", line 179, in <module> curr_loss = nd.mean(loss_result).asscalar() File "/home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages/mxnet/ndarray/ndarray.py", line 1894, in asscalar return self.asnumpy()[0] File "/home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages/mxnet/ndarray/ndarray.py", line 1876, in asnumpy ctypes.c_size_t(data.size))) File "/home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages/mxnet/base.py", line 149, in check_call raise MXNetError(py_str(_LIB.MXGetLastError())) mxnet.base.MXNetError: [17:13:00] src/operator/nn/./../mxnet_op.h:576: Check failed: err == cudaSuccess (2 vs. 0) Name: mxnet_generic_kernel ErrStr:out of memory Stack trace returned 10 entries: [bt] (0) /home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x31f81a) [0x7f62d1b6d81a] [bt] (1) /home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x31fe41) [0x7f62d1b6de41] [bt] (2) /home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x2c59657) [0x7f62d44a7657] [bt] (3) /home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x2c61e5e) [0x7f62d44afe5e] [bt] (4) /home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x25090b0) [0x7f62d3d570b0] [bt] (5) /home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x2480ff9) [0x7f62d3cceff9] [bt] (6) /home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x248abf4) [0x7f62d3cd8bf4] [bt] (7) /home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x248ea93) [0x7f62d3cdca93] [bt] (8) /home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x248ece6) [0x7f62d3cdcce6] [bt] (9) /home/ubuntu/anaconda3/envs/python3/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x248b304) [0x7f62d3cd9304]
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
