Justobe opened a new issue #17250: mxnet.base.MXNetError: Error in operator transpose176: [02:14:06] src/operator/tensor/./matrix_op-inl.h:354: Check failed: shp.ndim() == param.axes.ndim() (-1 vs. 4) URL: https://github.com/apache/incubator-mxnet/issues/17250 I get an exception when I load model using MXNET as keras's backend. **I can load the model correctly with Tensorflow , Theano and cntk as the backend, but I get the following error with MXNET:** > > Traceback (most recent call last): > File "/data/code/lemon-git/scripts/patch_mxnet/crash_checker.py", line 31, in <module> > model = keras.models.load_model(file_path,custom_objects=ModelUtils.custom_objects()) > File "/root/anaconda3/lib/python3.6/site-packages/keras/engine/saving.py", line 496, in load_model > model = deserialize_model(f, custom_objects, compile) > File "/root/anaconda3/lib/python3.6/site-packages/keras/engine/saving.py", line 302, in _deserialize_model > model = model_from_config(model_config, custom_objects=custom_objects) > File "/root/anaconda3/lib/python3.6/site-packages/keras/engine/saving.py", line 535, in model_from_config > return deserialize(config, custom_objects=custom_objects) > File "/root/anaconda3/lib/python3.6/site-packages/keras/layers/init.py", line 55, in deserialize > printable_module_name='layer') > File "/root/anaconda3/lib/python3.6/site-packages/keras/utils/generic_utils.py", line 145, in deserialize_keras_object > list(custom_objects.items()))) > File "/root/anaconda3/lib/python3.6/site-packages/keras/engine/sequential.py", line 301, in from_config > model.add(layer) > File "/root/anaconda3/lib/python3.6/site-packages/keras/engine/sequential.py", line 181, in add > output_tensor = layer(self.outputs[0]) > File "/root/anaconda3/lib/python3.6/site-packages/keras/engine/base_layer.py", line 470, in call > output = self.call(inputs, kwargs) > File "/root/anaconda3/lib/python3.6/site-packages/keras/layers/convolutional.py", line 175, in call > dilation_rate=self.dilation_rate) > File "/root/anaconda3/lib/python3.6/site-packages/keras/backend/mxnet_backend.py", line 3705, in conv2d > padding_mode=padding, data_format=data_format) > File "/root/anaconda3/lib/python3.6/site-packages/keras/backend/mxnet_backend.py", line 94, in func_wrapper > train_symbol = func(*args, kwargs) > File "/root/anaconda3/lib/python3.6/site-packages/keras/backend/mxnet_backend.py", line 5045, in _convnd > padding, is_slice, out_size = _preprocess_padding_mode(padding_mode, x.shape, > File "/root/anaconda3/lib/python3.6/site-packages/keras/backend/mxnet_backend.py", line 4395, in shape > return self.get_shape() > File "/root/anaconda3/lib/python3.6/site-packages/keras/backend/mxnet_backend.py", line 4404, in get_shape > _, out_shape, _ = self.symbol.infer_shape_partial() > File "/root/anaconda3/lib/python3.6/site-packages/mxnet/symbol/symbol.py", line 1152, in infer_shape_partial > return self.infer_shape_impl(True, *args, **kwargs) > File "/root/anaconda3/lib/python3.6/site-packages/mxnet/symbol/symbol.py", line 1210, in infer_shape_impl > ctypes.byref(complete))) > File "/root/anaconda3/lib/python3.6/site-packages/mxnet/base.py", line 253, in check_call > raise MXNetError(py_str(LIB.MXGetLastError())) > mxnet.base.MXNetError: Error in operator transpose176: [02:14:06] src/operator/tensor/./matrix_op-inl.h:354: Check failed: shp.ndim() == param.axes.ndim() (-1 vs. 4) : > Stack trace: > [bt] (0) /root/anaconda3/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x4b09db) [0x7fd6920089db] > [bt] (1) /root/anaconda3/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x235e45c) [0x7fd693eb645c] > [bt] (2) /root/anaconda3/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x2620d12) [0x7fd694178d12] > [bt] (3) /root/anaconda3/lib/python3.6/site-packages/mxnet/libmxnet.so(+0x26235fb) [0x7fd69417b5fb] > [bt] (4) /root/anaconda3/lib/python3.6/site-packages/mxnet/libmxnet.so(MXSymbolInferShapeEx+0x103e) [0x7fd6940dff7e] > [bt] (5) /root/anaconda3/lib/python3.6/site-packages/mxnet/libmxnet.so(MXSymbolInferShapePartialEx+0x82) [0x7fd6940e0672] > [bt] (6) /root/anaconda3/lib/python3.6/lib-dynload/../../libffi.so.6(ffi_call_unix64+0x4c) [0x7fd6c688bec0] > [bt] (7) /root/anaconda3/lib/python3.6/lib-dynload/../../libffi.so.6(ffi_call+0x22d) [0x7fd6c688b87d] > [bt] (8) /root/anaconda3/lib/python3.6/lib-dynload/ctypes.cpython-36m-x86_64-linux-gnu.so(ctypes_callproc+0x2ce) [0x7fd6c6d33ede] > The code is as follows: script.py: import os import sys import argparse """Parser of command args""" parse = argparse.ArgumentParser() parse.add_argument("--backend", type=str, help="name of backends") parse.add_argument("--crash_dir", type=str, help="path") flags, unparsed = parse.parse_known_args(sys.argv[1:]) bk = flags.backend os.environ['KERAS_BACKEND'] = bk from keras import backend as K import keras import traceback def custom_objects(): def no_activation(x): return x def leakyrelu(x): import keras.backend as K return K.relu(x, alpha=0.01) objects = {} objects['no_activation'] = no_activation objects['leakyrelu'] = leakyrelu return objects print("INFO:Using {} as backend for states extraction| {} is wanted".format(K.backend(), bk)) files = os.listdir(flags.crash_dir) pass_cnt = 0 for f in files: print(f) file_path = os.path.join(flags.crash_dir,f) try: print(f"Loading model. {pass_cnt+1} of {len(files)}") model = keras.models.load_model(file_path,custom_objects=custom_objects()) pass_cnt += 1 except: traceback.print_exc() print(f"Backend:{bk} Total: {len(files)} Pass:{pass_cnt}") This is a very simple code. Put the model in a folder, and then run the code to load the model with different backend. You can run like this:(change mxnet to other backend like tensorflow ) > python -u script.py --backend mxnet --crash_dir your/path/ My related library version is: > numpy 1.16.1 <pip> > tensorboard 1.14.0 py36hf484d3e_0 > tensorflow-estimator 1.14.0 <pip> > tensorflow-gpu 1.14.0 <pip> > Theano 1.0.4 > cntk-gpu 2.7 > mxnet-cu101 1.5.1.post0 > mkl 2018.0.3 1 > mkl-service 1.1.2 py36h17a0993_4 > mkl_fft 1.0.1 py36h3010b51_0 > mkl_random 1.0.1 py36h629b387_0 > Keras 2.2.4 <pip> > keras-applications 1.0.8 py_0 > keras-mxnet 2.2.4.2 <pip> > keras-preprocessing 1.1.0 py_1 > You can download the model from this link: https://send.firefox.com/download/8798078ff8dc36c7/#yJAfpS_y_69j5WbnJERoPw thanks in advance!
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
