piyushghai opened a new issue #13913: pickle.save crashes when saving an NDArray on MXNet - MKL v1.3.1 taken from PyPi URL: https://github.com/apache/incubator-mxnet/issues/13913 Using pickle.save on an NDArray crashes with the exception : ``` Traceback (most recent call last): File "test.py", line 77, in <module> save_pickle_dump('op.pkl', output) File "test.py", line 58, in save_pickle_dump pickle.dump(data, f) File "/usr/lib/python2.7/pickle.py", line 1376, in dump Pickler(file, protocol).dump(obj) File "/usr/lib/python2.7/pickle.py", line 224, in dump self.save(obj) File "/usr/lib/python2.7/pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "/usr/lib/python2.7/pickle.py", line 606, in save_list self._batch_appends(iter(obj)) File "/usr/lib/python2.7/pickle.py", line 621, in _batch_appends save(x) File "/usr/lib/python2.7/pickle.py", line 306, in save rv = reduce(self.proto) File "/home/ubuntu/.local/lib/python2.7/site-packages/mxnet/ndarray/ndarray.py", line 194, in __reduce__ return NDArray, (None,), self.__getstate__() File "/home/ubuntu/.local/lib/python2.7/site-packages/mxnet/ndarray/ndarray.py", line 369, in __getstate__ ctypes.byref(cptr))) File "/home/ubuntu/.local/lib/python2.7/site-packages/mxnet/base.py", line 251, in check_call raise MXNetError(py_str(_LIB.MXGetLastError())) mxnet.base.MXNetError: [01:26:04] src/ndarray/ndarray.cc:706: Check failed: !IsMKLDNNData() We can't generate TBlob for MKLDNN data. Please use Reorder2Default() to generate a new NDArray first Stack trace returned 10 entries: [bt] (0) /home/ubuntu/.local/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x1b6f0d) [0x7f3aae810f0d] [bt] (1) /home/ubuntu/.local/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x1b72d8) [0x7f3aae8112d8] [bt] (2) /home/ubuntu/.local/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2e7f465) [0x7f3ab14d9465] [bt] (3) /home/ubuntu/.local/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x1c53f8) [0x7f3aae81f3f8] [bt] (4) /home/ubuntu/.local/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2ea21d0) [0x7f3ab14fc1d0] [bt] (5) /home/ubuntu/.local/lib/python2.7/site-packages/mxnet/libmxnet.so(MXNDArraySaveRawBytes+0x8f) [0x7f3ab12af47f] [bt] (6) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7f3ac86dae40] [bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7f3ac86da8ab] [bt] (8) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7f3ac88ea3df] [bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(+0x11d82) [0x7f3ac88eed82] ``` Note: Providing complete information in the most concise form is the best way to get help. This issue template serves as the checklist for essential information to most of the technical issues and bug reports. For non-technical issues and feature requests, feel free to present the information in what you believe is the best form. ## Environment info (Required) ``` ----------Python Info---------- ('Version :', '2.7.12') ('Compiler :', 'GCC 5.4.0 20160609') ('Build :', ('default', 'Nov 12 2018 14:36:49')) ('Arch :', ('64bit', 'ELF')) ------------Pip Info----------- ('Version :', '18.1') ('Directory :', '/usr/local/lib/python2.7/dist-packages/pip') ----------MXNet Info----------- ('Version :', '1.3.1') ('Directory :', '/home/ubuntu/.local/lib/python2.7/site-packages/mxnet') ('Commit Hash :', '19c501680183237d52a862e6ae1dc4ddc296305b') ----------System Info---------- ('Platform :', 'Linux-4.4.0-1074-aws-x86_64-with-Ubuntu-16.04-xenial') ('system :', 'Linux') ('node :', 'ip-172-31-11-28') ('release :', '4.4.0-1074-aws') ('version :', '#84-Ubuntu SMP Thu Dec 6 08:57:58 UTC 2018') ----------Hardware Info---------- ('machine :', 'x86_64') ('processor :', 'x86_64') Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 8 On-line CPU(s) list: 0-7 Thread(s) per core: 2 Core(s) per socket: 4 Socket(s): 1 NUMA node(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 85 Model name: Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz Stepping: 4 CPU MHz: 3000.000 BogoMIPS: 6000.00 Hypervisor vendor: KVM Virtualization type: full L1d cache: 32K L1i cache: 32K L2 cache: 1024K L3 cache: 25344K NUMA node0 CPU(s): 0-7 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single kaiser fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f rdseed adx smap clflushopt clwb avx512cd xsaveopt xsavec xgetbv1 ida arat pku ----------Network Test---------- Setting timeout: 10 Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0011 sec, LOAD: 0.5849 sec. Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0015 sec, LOAD: 0.0867 sec. Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0091 sec, LOAD: 0.5859 sec. Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.0123 sec, LOAD: 0.0283 sec. Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.0997 sec, LOAD: 0.1171 sec. Timing for Gluon Tutorial(cn): https://zh.gluon.ai, DNS: 0.2975 sec, LOAD: 0.5281 sec. ``` Package used (Python/R/Scala/Julia): Python 2.7.12 ## Build info (Required if built from source) Taken from PyPi using this command : ``` pip install mxnet-mkl``` Compiler (gcc/clang/mingw/visual studio): MXNet commit hash: (Paste the output of `git rev-parse HEAD` here.) Build config: (Paste the content of config.mk, or the build command.) ## Error Message: (Paste the complete error message, including stack trace.) As indicated above. ## Minimum reproducible example ``` import mxnet as mx import urllib import os.path from os.path import join from collections import namedtuple import pickle Batch = namedtuple('Batch', ['data']) print (mx.__version__) file_path = "/tmp/" def download(url, location): filename = url.split("/")[-1] filename = join(location, filename) if not os.path.exists(filename): urllib.urlretrieve(url, filename) def get_model(prefix, epoch, location): print ("Downloading Model --- " + prefix) download(prefix + '-symbol.json', location) download(prefix + '-%04d.params' % (epoch,), location) def _get_base_model(symbol, arg_params, aux_params, ctx, layer_name='flatten0'): """ symbol: the pretrained network symbol arg_params: the argument parameters of the pretrained model num_classes: the number of classes for the fine-tune datasets layer_name: the layer name before the last fully-connected layer """ all_layers = symbol.get_internals() net = all_layers[layer_name + '_output'] print (net) base_model = mx.mod.Module(symbol=net, label_names=None, context=ctx) base_model.bind(for_training=False, data_shapes=[('data', (1, 3, 224, 224))]) base_model.set_params(arg_params=arg_params, aux_params=aux_params) return base_model def get_image(url): # download and show the image. Remove query string from the file name. fname = mx.test_utils.download(url, fname=url.split('/')[-1].split('?')[0]) img = mx.image.imread(fname) if img is None: return None # convert into format (batch, RGB, width, height) img = mx.image.imresize(img, 224, 224) # resize img = img.transpose((2, 0, 1)) # Channel first img = img.expand_dims(axis=0) # batchify img = img.astype('float32') # for gpu context return img def save_pickle_dump(file_handle, data): with open(file_handle, 'wb') as f: pickle.dump(data, f) get_model('http://data.mxnet.io/models/imagenet/resnet/50-layers/resnet-50', 0, file_path) print ("Loading model checkpoint ---") sym, arg_params, aux_params = mx.model.load_checkpoint(join(file_path,'resnet-50'), 0) model = _get_base_model(sym, arg_params, aux_params, mx.cpu()) model.data_shapes img = get_image('https://github.com/dmlc/web-data/blob/master/mxnet/doc/tutorials/python/predict_image/cat.jpg?raw=true') train_data = mx.io.DataBatch(data=[img]) train_data.label = None model.forward(train_data) output = model.get_outputs() #print output #print output[0].asnumpy() #print output[0].shape save_pickle_dump('op.pkl', output) ``` ## Steps to reproduce (Paste the commands you ran that produced the error.) ```pip install mxnet-mkl``` ``` python test.py``` test.py is the name of the file whose contents are shared above. ## What have you tried to solve it? Downloaded non mkl package. i.e. ```pip install mxnet```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
