samskalicky opened a new issue #15737: 1.5.0 MKLDNN error "Unknown MKLDNN format" URL: https://github.com/apache/incubator-mxnet/issues/15737 Resnext-50 model from MXNet model zoo fails in 1.5.0 build for "mxnet-mkl" pip wheel with MKLDNN error "Unknown MKLDNN format for 5 dimensions: 108". Works with: - "mxnet" pip wheel - "mxnet-mkl==1.4.1" ## Error message: ``` Traceback (most recent call last): File "resnext.py", line 43, in <module> results = mod.get_outputs()[0].asnumpy() File "/home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/ndarray/ndarray.py", line 1996, in asnumpy ctypes.c_size_t(data.size))) File "/home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/base.py", line 253, in check_call raise MXNetError(py_str(_LIB.MXGetLastError())) mxnet.base.MXNetError: [17:48:25] src/operator/nn/mkldnn/mkldnn_base.cc:398: Unknown MKLDNN format for 5 dimensions: 108 Stack trace: [bt] (0) /home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x25b2ab) [0x7fb152faf2ab] [bt] (1) /home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x267c95) [0x7fb152fbbc95] [bt] (2) /home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/libmxnet.so(mxnet::NDArray::GetMKLDNNData(mkldnn::memory::primitive_desc const&) const+0x1f4) [0x7fb15533c944] [bt] (3) /home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/libmxnet.so(mxnet::GetWeights(mxnet::NDArray const&, mkldnn::memory::primitive_desc const&, int)+0x21) [0x7fb152fc29c1] [bt] (4) /home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/libmxnet.so(mxnet::op::MKLDNNConvolutionForwardFullFeature(mxnet::op::MKLDNNConvFullParam const&, mxnet::OpContext const&, mxnet::op::MKLDNNConvForward*, std::vector<mxnet::NDArray, std::allocator<mxnet::NDArray> > const&, std::vector<mxnet::OpReqType, std::allocator<mxnet::OpReqType> > const&, std::vector<mxnet::NDArray, std::allocator<mxnet::NDArray> > const&)+0x4da) [0x7fb152fde26a] [bt] (5) /home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/libmxnet.so(mxnet::op::MKLDNNConvolutionForward(nnvm::NodeAttrs const&, mxnet::OpContext const&, std::vector<mxnet::NDArray, std::allocator<mxnet::NDArray> > const&, std::vector<mxnet::OpReqType, std::allocator<mxnet::OpReqType> > const&, std::vector<mxnet::NDArray, std::allocator<mxnet::NDArray> > const&)+0x434) [0x7fb152fdf9b4] [bt] (6) /home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x73b81a) [0x7fb15348f81a] [bt] (7) /home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x24298f7) [0x7fb15517d8f7] [bt] (8) /home/ubuntu/anaconda3/envs/mxnet_p27/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2431dbf) [0x7fb155185dbf] ``` ## Failing code: ``` import cv2 import numpy as np import mxnet as mx from collections import namedtuple # load_image # Description: load image for imagenet model testing # Returns formatted image as numpy array def load_image(): fname = mx.test_utils.download('https://github.com/dmlc/web-data/blob/master/mxnet/doc/tutorials/python/predict_image/cat.jpg?raw=true') img = cv2.imread(fname) img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) img = cv2.resize(img, (224, 224,)) img = np.swapaxes(img, 0, 2) img = np.swapaxes(img, 1, 2) img = img[np.newaxis, :] return img # download model mx.test_utils.download('http://data.mxnet.io/models/imagenet/resnext/50-layers/resnext-50-symbol.json') mx.test_utils.download('http://data.mxnet.io/models/imagenet/resnext/50-layers/resnext-50-0000.params') ctx = mx.cpu() # load model sym, arg_params, aux_params = mx.model.load_checkpoint('resnext-50', 0) mod = mx.mod.Module(symbol=sym, context=ctx) # bind model exe = mod.bind(for_training=False, data_shapes=[('data', (1,3,224,224))], label_shapes=mod._label_shapes) mod.set_params(arg_params, aux_params, allow_missing=True) # setup batch img = load_image() Batch = namedtuple('Batch', ['data']) data = Batch([mx.nd.array(img)]) # inference mod.forward(data, is_train=False) results = mod.get_outputs()[0].asnumpy() ``` @PatricZhao @ZhennanQin @TaoLv can you please help debug?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
