[GitHub] spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export
spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r195590744
##
File path: python/mxnet/contrib/onnx/_export/op_translations.py
##
@@ -0,0 +1,1667 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Based on
+# https://github.com/NVIDIA/mxnet_to_onnx/blob/master/mx2onnx_converter/
+# mx2onnx_converter_functions.py
+# Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# * Redistributions of source code must retain the above copyright
+#notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+#notice, this list of conditions and the following disclaimer in the
+#documentation and/or other materials provided with the distribution.
+# * Neither the name of NVIDIA CORPORATION nor the names of its
+#contributors may be used to endorse or promote products derived
+#from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+# coding: utf-8
+# pylint: disable=too-many-locals,no-else-return,too-many-lines
+# pylint: disable=anomalous-backslash-in-string,eval-used
+"""
+Conversion Functions for common layers.
+Add new functions here with a decorator.
+"""
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+from __future__ import unicode_literals
+import re
+import numpy as np
+
+from onnx import helper, numpy_helper, mapping
+from .export_onnx import MXNetGraph as mx_op
+
+
+@mx_op.register("null")
+def convert_weights_and_inputs(node, **kwargs):
+"""Helper function to convert weights and inputs.
+"""
+name = node["name"]
+
+if kwargs["is_input"] is False:
+weights = kwargs["weights"]
+initializer = kwargs["initializer"]
+np_arr = weights[name]
+data_type = mapping.NP_TYPE_TO_TENSOR_TYPE[np_arr.dtype]
+dims = np.shape(np_arr)
+
+tensor_node = helper.make_tensor_value_info(name, data_type, dims)
+
+initializer.append(
+helper.make_tensor(
+name=name,
+data_type=data_type,
+dims=dims,
+vals=np_arr.flatten().tolist(),
+raw=False,
+)
+)
+
+return [tensor_node]
+else:
+tval_node = helper.make_tensor_value_info(name, kwargs["in_type"],
kwargs["in_shape"])
Review comment:
Dictionary is a good idea, but it might not work in a bunch of cases , like
Pooling maps to (maxpool, globalmaxpool, avgpool, globalmaxpool) or LeakyReLU
maps to (elu, leakyrelu, prelu) depending on one of the attributes. Also in few
cases , we use multiple ONNX ops to handle one mxnet operator, so will not be a
one-one match. This will make the info in the table only partial correct, which
might become cumbersome to maintain.
wdyt?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With
[GitHub] spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export
spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r195585825
##
File path: tests/python-pytest/onnx/export/mxnet_export_test.py
##
@@ -0,0 +1,186 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+Tests for individual operators
+This module contains operator tests which currently do not exist on
+ONNX backend test framework. Once we have PRs on the ONNX repo and get
+those PRs merged, this file will get EOL'ed.
+"""
+# pylint: disable=too-many-locals,wrong-import-position,import-error
+from __future__ import absolute_import
+import sys
+import os
+import logging
+import tarfile
+from collections import namedtuple
+import numpy as np
+import numpy.testing as npt
+from onnx import numpy_helper
+from onnx import TensorProto
+from mxnet.test_utils import download
+from mxnet.contrib import onnx as onnx_mxnet
+import mxnet as mx
+CURR_PATH = os.path.dirname(os.path.abspath(os.path.expanduser(__file__)))
+sys.path.insert(0, os.path.join(CURR_PATH, '../../python/unittest'))
+
+URLS = {
+'bvlc_googlenet':
+'https://s3.amazonaws.com/onnx-mxnet/model-zoo/bvlc_googlenet.tar.gz',
+'bvlc_reference_caffenet':
+
'https://s3.amazonaws.com/onnx-mxnet/model-zoo/bvlc_reference_caffenet.tar.gz',
+'bvlc_reference_rcnn_ilsvrc13':
+
'https://s3.amazonaws.com/onnx-mxnet/model-zoo/bvlc_reference_rcnn_ilsvrc13.tar.gz',
+'inception_v1':
+'https://s3.amazonaws.com/onnx-mxnet/model-zoo/inception_v1.tar.gz',
+'inception_v2':
+'https://s3.amazonaws.com/onnx-mxnet/model-zoo/inception_v2.tar.gz'
+}
+
+def get_test_files(name):
+"""Extract tar file and returns model path and input, output data"""
+tar_name = download(URLS.get(name), dirname=CURR_PATH.__str__())
+# extract tar file
+tar_path = os.path.join(CURR_PATH, tar_name)
+tar = tarfile.open(tar_path.__str__(), "r:*")
+tar.extractall(path=CURR_PATH.__str__())
+tar.close()
+data_dir = os.path.join(CURR_PATH, name)
+model_path = os.path.join(data_dir, 'model.onnx')
+
+inputs = []
+outputs = []
+# get test files
+for test_file in os.listdir(data_dir):
+case_dir = os.path.join(data_dir, test_file)
+# skip the non-dir files
+if not os.path.isdir(case_dir):
+continue
+input_file = os.path.join(case_dir, 'input_0.pb')
+input_tensor = TensorProto()
+with open(input_file, 'rb') as proto_file:
+input_tensor.ParseFromString(proto_file.read())
+inputs.append(numpy_helper.to_array(input_tensor))
+
+output_tensor = TensorProto()
+output_file = os.path.join(case_dir, 'output_0.pb')
+with open(output_file, 'rb') as proto_file:
+output_tensor.ParseFromString(proto_file.read())
+outputs.append(numpy_helper.to_array(output_tensor))
+
+return model_path, inputs, outputs
+
+
+def forward_pass(sym, arg, aux, data_names, input_data):
+""" Perform forward pass on given data"""
+# create module
+mod = mx.mod.Module(symbol=sym, data_names=data_names, context=mx.cpu(),
label_names=None)
+mod.bind(for_training=False, data_shapes=[(data_names[0],
input_data.shape)], label_shapes=None)
+mod.set_params(arg_params=arg, aux_params=aux,
+ allow_missing=True, allow_extra=True)
+# run inference
+batch = namedtuple('Batch', ['data'])
+mod.forward(batch([mx.nd.array(input_data)]), is_train=False)
+
+return mod.get_outputs()[0].asnumpy()
+
+
+def test_models(model_name, input_shape, output_shape):
+""" Tests Googlenet model for both onnx import and export"""
+model_path, inputs, outputs = get_test_files(model_name)
+logging.info("Translating model from ONNX model zoo to Mxnet")
+sym, arg_params, aux_params = onnx_mxnet.import_model(model_path)
+params = {}
+params.update(arg_params)
+params.update(aux_params)
+
+onnx_file = model_path.rsplit('/', 1)[0] + "/exported_"+model_name+".onnx"
+
+logging.info("Translating converted model from mxnet to ONNX")
+converted_model_path = onnx_mxnet.export_model(sym, params,
[GitHub] spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export
spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r195585681
##
File path: python/mxnet/contrib/onnx/_export/export_onnx.py
##
@@ -0,0 +1,270 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Based on
+#
https://github.com/NVIDIA/mxnet_to_onnx/blob/master/mx2onnx_converter/mx2onnx_converter.py#
+# Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# * Redistributions of source code must retain the above copyright
+#notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+#notice, this list of conditions and the following disclaimer in the
+#documentation and/or other materials provided with the distribution.
+# * Neither the name of NVIDIA CORPORATION nor the names of its
+#contributors may be used to endorse or promote products derived
+#from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+# coding: utf-8
+# pylint: disable=invalid-name,too-many-locals,no-self-use,too-many-arguments,
+# pylint: disable=maybe-no-member,too-many-nested-blocks
+"""MXNet to ONNX graph converter functions"""
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+from __future__ import unicode_literals
+
+import json
+import numpy as np
+
+from import context
+from import ndarray as nd
+from import io
+from import module as mod
+
+
+class MXNetGraph(object):
+"""Class to convert MXNet to ONNX graph"""
+registry_ = {}
+input_output_maps_ = {}
+
+def __init__(self):
+# topologically sorted nodes
+self.nodes = []
+self.input_tensors = []
+self.output_tensors = []
+
+@staticmethod
+def register(op_name):
Review comment:
usually , detailed info is only added for public api's.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export
spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r195585456 ## File path: python/mxnet/contrib/onnx/_export/export_model.py ## @@ -0,0 +1,89 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# coding: utf-8 +#pylint: disable-msg=too-many-arguments + +"""export function""" +from __future__ import absolute_import +from __future__ import division +from __future__ import print_function +from __future__ import unicode_literals +import logging +import numpy as np + +from base import string_types +from .export_onnx import MXNetGraph +from .export_helper import load_module + + +def export_model(model, weights, input_shape, input_type=np.float32, Review comment: done This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export
spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r195585421 ## File path: python/mxnet/contrib/onnx/__init__.py ## @@ -18,3 +18,4 @@ from ._import.import_model import import_model, get_model_metadata Review comment: folder name changed to be public , _import --> onnx2mx , _export-->mx2onnx . also changed the files in the folder as per their usage This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export
spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r195505069
##
File path: tests/python-pytest/onnx/export/backend.py
##
@@ -0,0 +1,98 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# coding: utf-8
+"""backend wrapper for onnx test infrastructure"""
+import mxnet as mx
+import numpy as np
+from mxnet.contrib.onnx._import.import_onnx import GraphProto
+from mxnet.contrib.onnx._export.export_onnx import MXNetGraph
+try:
+from onnx import helper, TensorProto, mapping
+from onnx.backend.base import Backend
+except ImportError:
+raise ImportError("Onnx and protobuf need to be installed")
+from backend_rep import MXNetBackendRep
+
+# Using these functions for onnx test infrastructure.
+# Implemented by following onnx docs guide:
+#
https://github.com/onnx/onnx/blob/master/docs/Implementing%20an%20ONNX%20backend.md
+# MXNetBackend class will take an ONNX model with inputs, perform a
computation,
+# and then return the output.
+
+class MXNetBackend(Backend):
+"""MXNet backend for ONNX"""
+
+@staticmethod
+def perform_import_export(graph_proto, input_shape):
+""" Import ONNX model to mxnet model and then export to ONNX model
+and then import it back to mxnet for verifying the result"""
+graph = GraphProto()
+
+sym, arg_params, aux_params = graph.from_onnx(graph_proto)
+
+params = {}
+params.update(arg_params)
+params.update(aux_params)
+# exporting to onnx graph proto format
+converter = MXNetGraph()
+graph_proto = converter.create_onnx_graph_proto(sym, params,
in_shape=input_shape,
in_type=mapping.NP_TYPE_TO_TENSOR_TYPE[np.dtype('float32')])
+
+# importing back to MXNET for verifying result.
+sym, arg_params, aux_params = graph.from_onnx(graph_proto)
+
+return sym, arg_params, aux_params
+
+
+@classmethod
+def prepare(cls, model, device='CPU', **kwargs):
Review comment:
this method is declared by ONNX. The backends using ONNX test framework
derives from "backend" class and implements these functions.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export
spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r195502414
##
File path: python/mxnet/contrib/onnx/_export/export_onnx.py
##
@@ -0,0 +1,270 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Based on
+#
https://github.com/NVIDIA/mxnet_to_onnx/blob/master/mx2onnx_converter/mx2onnx_converter.py#
+# Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# * Redistributions of source code must retain the above copyright
+#notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+#notice, this list of conditions and the following disclaimer in the
+#documentation and/or other materials provided with the distribution.
+# * Neither the name of NVIDIA CORPORATION nor the names of its
+#contributors may be used to endorse or promote products derived
+#from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+# coding: utf-8
+# pylint: disable=invalid-name,too-many-locals,no-self-use,too-many-arguments,
+# pylint: disable=maybe-no-member,too-many-nested-blocks
+"""MXNet to ONNX graph converter functions"""
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+from __future__ import unicode_literals
+
+import json
+import numpy as np
+
+from import context
+from import ndarray as nd
+from import io
+from import module as mod
+
+
+class MXNetGraph(object):
+"""Class to convert MXNet to ONNX graph"""
+registry_ = {}
+input_output_maps_ = {}
+
+def __init__(self):
+# topologically sorted nodes
+self.nodes = []
+self.input_tensors = []
+self.output_tensors = []
+
+@staticmethod
+def register(op_name):
+"""Register operator"""
+def wrapper(func):
+"""Helper function to map functions"""
+MXNetGraph.registry_[op_name] = func
+return func
+
+return wrapper
+
+@staticmethod
+def convert_layer(node, **kwargs):
+"""Convert MXNet layer to ONNX"""
+op = str(node["op"])
+if op not in MXNetGraph.registry_:
+raise AttributeError("No conversion function registered for op
type %s yet." % op)
+convert_fun = MXNetGraph.registry_[op]
+return convert_fun(node, **kwargs)
+
+@staticmethod
+def forward_pass(inputs, sym, arg_params, aux_params):
+""" Do a forward pass based on the sym and params"""
+data_names = [graph_input for graph_input in sym.list_inputs()
+ if graph_input not in arg_params and graph_input not in
aux_params
+ and graph_input != 'softmax_label']
+
+data_shapes = []
+# Adding extra dimension of batch_size 1 if the batch_size is
different for multiple inputs.
+for idx, input_name in enumerate(data_names):
+data_shapes.append((input_name, inputs[idx].shape))
+
+# create module, passing cpu context
+ctx = context.cpu()
+test_mod = mod.Module(symbol=sym, data_names=data_names, context=ctx,
label_names=None)
+
[GitHub] spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export
spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r195501798
##
File path: python/mxnet/contrib/onnx/_export/export_onnx.py
##
@@ -0,0 +1,270 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Based on
+#
https://github.com/NVIDIA/mxnet_to_onnx/blob/master/mx2onnx_converter/mx2onnx_converter.py#
+# Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# * Redistributions of source code must retain the above copyright
+#notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+#notice, this list of conditions and the following disclaimer in the
+#documentation and/or other materials provided with the distribution.
+# * Neither the name of NVIDIA CORPORATION nor the names of its
+#contributors may be used to endorse or promote products derived
+#from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+# coding: utf-8
+# pylint: disable=invalid-name,too-many-locals,no-self-use,too-many-arguments,
+# pylint: disable=maybe-no-member,too-many-nested-blocks
+"""MXNet to ONNX graph converter functions"""
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+from __future__ import unicode_literals
+
+import json
+import numpy as np
+
+from import context
+from import ndarray as nd
+from import io
+from import module as mod
+
+
+class MXNetGraph(object):
+"""Class to convert MXNet to ONNX graph"""
+registry_ = {}
+input_output_maps_ = {}
+
+def __init__(self):
+# topologically sorted nodes
+self.nodes = []
+self.input_tensors = []
+self.output_tensors = []
+
+@staticmethod
+def register(op_name):
+"""Register operator"""
+def wrapper(func):
+"""Helper function to map functions"""
+MXNetGraph.registry_[op_name] = func
+return func
+
+return wrapper
+
+@staticmethod
+def convert_layer(node, **kwargs):
+"""Convert MXNet layer to ONNX"""
+op = str(node["op"])
+if op not in MXNetGraph.registry_:
+raise AttributeError("No conversion function registered for op
type %s yet." % op)
+convert_fun = MXNetGraph.registry_[op]
+return convert_fun(node, **kwargs)
+
+@staticmethod
+def forward_pass(inputs, sym, arg_params, aux_params):
+""" Do a forward pass based on the sym and params"""
+data_names = [graph_input for graph_input in sym.list_inputs()
+ if graph_input not in arg_params and graph_input not in
aux_params
+ and graph_input != 'softmax_label']
+
+data_shapes = []
+# Adding extra dimension of batch_size 1 if the batch_size is
different for multiple inputs.
+for idx, input_name in enumerate(data_names):
+data_shapes.append((input_name, inputs[idx].shape))
+
+# create module, passing cpu context
+ctx = context.cpu()
+test_mod = mod.Module(symbol=sym, data_names=data_names, context=ctx,
label_names=None)
+
[GitHub] spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export
spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r195501638 ## File path: python/mxnet/contrib/onnx/_export/export_model.py ## @@ -0,0 +1,89 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# coding: utf-8 +#pylint: disable-msg=too-many-arguments + +"""export function""" +from __future__ import absolute_import +from __future__ import division +from __future__ import print_function +from __future__ import unicode_literals +import logging +import numpy as np + +from base import string_types +from .export_onnx import MXNetGraph +from .export_helper import load_module + + +def export_model(model, weights, input_shape, input_type=np.float32, + onnx_file_path='model.onnx', log=False): +"""Exports the MXNet model file, passed as a parameter, into ONNX model. +Accepts both symbol,parameter objects as well as json and params filepaths as input. +Operator support and coverage - https://cwiki.apache.org/confluence/display/MXNET/ONNX + +Parameters +-- +model : str or symbol object +Path to the json file or Symbol object +weights : str or symbol object +Path to the params file or Params object. (Including both arg_params and aux_params) Review comment: it can be both , changed the desc to be more explicit. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export
spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r195501399 ## File path: python/mxnet/contrib/onnx/_export/export_model.py ## @@ -0,0 +1,89 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# coding: utf-8 +#pylint: disable-msg=too-many-arguments + +"""export function""" +from __future__ import absolute_import +from __future__ import division +from __future__ import print_function +from __future__ import unicode_literals +import logging +import numpy as np + +from base import string_types +from .export_onnx import MXNetGraph +from .export_helper import load_module + + +def export_model(model, weights, input_shape, input_type=np.float32, Review comment: changed. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export
spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194908833
##
File path: python/mxnet/contrib/onnx/_export/export_helper.py
##
@@ -0,0 +1,64 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""export helper functions"""
+# coding: utf-8
+import os
+import logging
+import mxnet as mx
+
+def load_module(json_path, params_path, input_shape):
+"""Loads the MXNet model file, retrieves symbol and parameters and returns.
+
+Parameters
+--
+json_path : str
+Path to the json file
+params_path : str
+Path to the params file
+input_shape :
+Input shape of the model e.g (1,3,224,224)
+
+Returns
+---
+sym : MXNet symbol
+Model symbol object
+
+params : params object
+Model weights including both arg and aux params.
+"""
+if not (os.path.isfile(json_path) and os.path.isfile(params_path)):
+raise ValueError("Provide valid path to the json and params file")
+else:
+try:
+model_name = json_path.rsplit('.', 1)[0].rsplit('-', 1)[0]
+num_epochs = int(params_path.rsplit('.', 1)[0].rsplit('-', 1)[1])
+except IndexError:
+logging.info("Model and params name should be in format: "
+ "prefix-symbol.json, prefix-epoch.params")
+raise
+
+sym, arg_params, aux_params = mx.model.load_checkpoint(model_name,
num_epochs)
+trained_model = mx.mod.Module(symbol=sym, label_names=None)
Review comment:
done. extracting from sym and params
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export
spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194908752
##
File path: python/mxnet/contrib/onnx/_export/export_helper.py
##
@@ -0,0 +1,64 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""export helper functions"""
+# coding: utf-8
+import os
+import logging
+import mxnet as mx
+
+def load_module(json_path, params_path, input_shape):
+"""Loads the MXNet model file, retrieves symbol and parameters and returns.
+
+Parameters
+--
+json_path : str
+Path to the json file
+params_path : str
+Path to the params file
+input_shape :
+Input shape of the model e.g (1,3,224,224)
+
+Returns
+---
+sym : MXNet symbol
+Model symbol object
+
+params : params object
+Model weights including both arg and aux params.
+"""
+if not (os.path.isfile(json_path) and os.path.isfile(params_path)):
+raise ValueError("Provide valid path to the json and params file")
+else:
+try:
+model_name = json_path.rsplit('.', 1)[0].rsplit('-', 1)[0]
+num_epochs = int(params_path.rsplit('.', 1)[0].rsplit('-', 1)[1])
+except IndexError:
+logging.info("Model and params name should be in format: "
+ "prefix-symbol.json, prefix-epoch.params")
+raise
+
+sym, arg_params, aux_params = mx.model.load_checkpoint(model_name,
num_epochs)
+trained_model = mx.mod.Module(symbol=sym, label_names=None)
Review comment:
ONNX models are context agnostic. For running an ONNX model you will import
ONNX model to ur framework model format i.e sym and params for MXNet. Now you
use mxnet api to set your context and either retrain this model(i.e ignore the
params file ) or use it for inference.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export
spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194901801
##
File path: python/mxnet/contrib/onnx/_export/op_translations.py
##
@@ -0,0 +1,1667 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Based on
+# https://github.com/NVIDIA/mxnet_to_onnx/blob/master/mx2onnx_converter/
+# mx2onnx_converter_functions.py
+# Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# * Redistributions of source code must retain the above copyright
+#notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+#notice, this list of conditions and the following disclaimer in the
+#documentation and/or other materials provided with the distribution.
+# * Neither the name of NVIDIA CORPORATION nor the names of its
+#contributors may be used to endorse or promote products derived
+#from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+# coding: utf-8
+# pylint: disable=too-many-locals,no-else-return,too-many-lines
+# pylint: disable=anomalous-backslash-in-string,eval-used
+"""
+Conversion Functions for common layers.
+Add new functions here with a decorator.
+"""
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+from __future__ import unicode_literals
+import re
+import numpy as np
+
+from onnx import helper, numpy_helper, mapping
+from .export_onnx import MXNetGraph as mx_op
+
+
+@mx_op.register("null")
+def convert_weights_and_inputs(node, **kwargs):
+"""Helper function to convert weights and inputs.
+"""
+name = node["name"]
+
+if kwargs["is_input"] is False:
+weights = kwargs["weights"]
+initializer = kwargs["initializer"]
+np_arr = weights[name]
+data_type = mapping.NP_TYPE_TO_TENSOR_TYPE[np_arr.dtype]
+dims = np.shape(np_arr)
+
+tensor_node = helper.make_tensor_value_info(name, data_type, dims)
+
+initializer.append(
+helper.make_tensor(
+name=name,
+data_type=data_type,
+dims=dims,
+vals=np_arr.flatten().tolist(),
+raw=False,
+)
+)
+
+return [tensor_node]
+else:
+tval_node = helper.make_tensor_value_info(name, kwargs["in_type"],
kwargs["in_shape"])
+return [tval_node]
+
+
+def parse_helper(attrs, attrs_name, alt_value=None):
+"""Helper function to parse operator attributes in required format."""
+tuple_re = re.compile('\([0-9L|,| ]+\)')
+if attrs is None:
+return alt_value
+attrs_str = str(attrs.get(attrs_name))
+if attrs_str is None:
+return alt_value
+attrs_match = tuple_re.search(attrs_str)
+if attrs_match is not None:
+if attrs_match.span() == (0, len(attrs_str)):
+dims = eval(attrs_str)
+return dims
+else:
+raise AttributeError("Malformed %s dimensions: %s" % (attrs_name,
str(attrs_str)))
+return alt_value
+
+
+@mx_op.register("Convolution")
+def convert_convolution(node, **kwargs):
+"""Map MXNet's convolution operator
[GitHub] spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export
spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194892098
##
File path: python/mxnet/contrib/onnx/_export/op_translations.py
##
@@ -0,0 +1,1667 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Based on
+# https://github.com/NVIDIA/mxnet_to_onnx/blob/master/mx2onnx_converter/
+# mx2onnx_converter_functions.py
+# Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# * Redistributions of source code must retain the above copyright
+#notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+#notice, this list of conditions and the following disclaimer in the
+#documentation and/or other materials provided with the distribution.
+# * Neither the name of NVIDIA CORPORATION nor the names of its
+#contributors may be used to endorse or promote products derived
+#from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+# coding: utf-8
+# pylint: disable=too-many-locals,no-else-return,too-many-lines
+# pylint: disable=anomalous-backslash-in-string,eval-used
+"""
+Conversion Functions for common layers.
+Add new functions here with a decorator.
+"""
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+from __future__ import unicode_literals
+import re
+import numpy as np
+
+from onnx import helper, numpy_helper, mapping
+from .export_onnx import MXNetGraph as mx_op
+
+
+@mx_op.register("null")
+def convert_weights_and_inputs(node, **kwargs):
+"""Helper function to convert weights and inputs.
+"""
+name = node["name"]
+
+if kwargs["is_input"] is False:
+weights = kwargs["weights"]
+initializer = kwargs["initializer"]
+np_arr = weights[name]
+data_type = mapping.NP_TYPE_TO_TENSOR_TYPE[np_arr.dtype]
+dims = np.shape(np_arr)
+
+tensor_node = helper.make_tensor_value_info(name, data_type, dims)
+
+initializer.append(
+helper.make_tensor(
+name=name,
+data_type=data_type,
+dims=dims,
+vals=np_arr.flatten().tolist(),
+raw=False,
+)
+)
+
+return [tensor_node]
+else:
+tval_node = helper.make_tensor_value_info(name, kwargs["in_type"],
kwargs["in_shape"])
+return [tval_node]
+
+
+def parse_helper(attrs, attrs_name, alt_value=None):
+"""Helper function to parse operator attributes in required format."""
+tuple_re = re.compile('\([0-9L|,| ]+\)')
+if attrs is None:
+return alt_value
+attrs_str = str(attrs.get(attrs_name))
+if attrs_str is None:
+return alt_value
+attrs_match = tuple_re.search(attrs_str)
+if attrs_match is not None:
+if attrs_match.span() == (0, len(attrs_str)):
+dims = eval(attrs_str)
+return dims
+else:
+raise AttributeError("Malformed %s dimensions: %s" % (attrs_name,
str(attrs_str)))
+return alt_value
+
+
+@mx_op.register("Convolution")
+def convert_convolution(node, **kwargs):
+"""Map MXNet's convolution operator
[GitHub] spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export
spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194869698
##
File path: python/mxnet/contrib/onnx/_export/op_translations.py
##
@@ -0,0 +1,1667 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Based on
+# https://github.com/NVIDIA/mxnet_to_onnx/blob/master/mx2onnx_converter/
+# mx2onnx_converter_functions.py
+# Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# * Redistributions of source code must retain the above copyright
+#notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+#notice, this list of conditions and the following disclaimer in the
+#documentation and/or other materials provided with the distribution.
+# * Neither the name of NVIDIA CORPORATION nor the names of its
+#contributors may be used to endorse or promote products derived
+#from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+# coding: utf-8
+# pylint: disable=too-many-locals,no-else-return,too-many-lines
+# pylint: disable=anomalous-backslash-in-string,eval-used
+"""
+Conversion Functions for common layers.
+Add new functions here with a decorator.
+"""
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+from __future__ import unicode_literals
+import re
+import numpy as np
+
+from onnx import helper, numpy_helper, mapping
+from .export_onnx import MXNetGraph as mx_op
+
+
+@mx_op.register("null")
+def convert_weights_and_inputs(node, **kwargs):
+"""Helper function to convert weights and inputs.
+"""
+name = node["name"]
+
+if kwargs["is_input"] is False:
+weights = kwargs["weights"]
+initializer = kwargs["initializer"]
+np_arr = weights[name]
+data_type = mapping.NP_TYPE_TO_TENSOR_TYPE[np_arr.dtype]
+dims = np.shape(np_arr)
+
+tensor_node = helper.make_tensor_value_info(name, data_type, dims)
+
+initializer.append(
+helper.make_tensor(
+name=name,
+data_type=data_type,
+dims=dims,
+vals=np_arr.flatten().tolist(),
+raw=False,
+)
+)
+
+return [tensor_node]
+else:
+tval_node = helper.make_tensor_value_info(name, kwargs["in_type"],
kwargs["in_shape"])
+return [tval_node]
+
+
+def parse_helper(attrs, attrs_name, alt_value=None):
+"""Helper function to parse operator attributes in required format."""
+tuple_re = re.compile('\([0-9L|,| ]+\)')
+if attrs is None:
+return alt_value
+attrs_str = str(attrs.get(attrs_name))
+if attrs_str is None:
+return alt_value
+attrs_match = tuple_re.search(attrs_str)
+if attrs_match is not None:
+if attrs_match.span() == (0, len(attrs_str)):
+dims = eval(attrs_str)
+return dims
+else:
+raise AttributeError("Malformed %s dimensions: %s" % (attrs_name,
str(attrs_str)))
+return alt_value
+
+
+@mx_op.register("Convolution")
+def convert_convolution(node, **kwargs):
+"""Map MXNet's convolution operator
[GitHub] spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export
spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX
export
URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194868703
##
File path: python/mxnet/contrib/onnx/_export/op_translations.py
##
@@ -0,0 +1,1667 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Based on
+# https://github.com/NVIDIA/mxnet_to_onnx/blob/master/mx2onnx_converter/
+# mx2onnx_converter_functions.py
+# Copyright (c) 2017, NVIDIA CORPORATION. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# * Redistributions of source code must retain the above copyright
+#notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+#notice, this list of conditions and the following disclaimer in the
+#documentation and/or other materials provided with the distribution.
+# * Neither the name of NVIDIA CORPORATION nor the names of its
+#contributors may be used to endorse or promote products derived
+#from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+# coding: utf-8
+# pylint: disable=too-many-locals,no-else-return,too-many-lines
+# pylint: disable=anomalous-backslash-in-string,eval-used
+"""
+Conversion Functions for common layers.
+Add new functions here with a decorator.
+"""
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+from __future__ import unicode_literals
+import re
+import numpy as np
+
+from onnx import helper, numpy_helper, mapping
+from .export_onnx import MXNetGraph as mx_op
+
+
+@mx_op.register("null")
+def convert_weights_and_inputs(node, **kwargs):
+"""Helper function to convert weights and inputs.
+"""
+name = node["name"]
+
+if kwargs["is_input"] is False:
+weights = kwargs["weights"]
+initializer = kwargs["initializer"]
+np_arr = weights[name]
+data_type = mapping.NP_TYPE_TO_TENSOR_TYPE[np_arr.dtype]
+dims = np.shape(np_arr)
+
+tensor_node = helper.make_tensor_value_info(name, data_type, dims)
+
+initializer.append(
+helper.make_tensor(
+name=name,
+data_type=data_type,
+dims=dims,
+vals=np_arr.flatten().tolist(),
+raw=False,
+)
+)
+
+return [tensor_node]
+else:
+tval_node = helper.make_tensor_value_info(name, kwargs["in_type"],
kwargs["in_shape"])
+return [tval_node]
+
+
+def parse_helper(attrs, attrs_name, alt_value=None):
+"""Helper function to parse operator attributes in required format."""
+tuple_re = re.compile('\([0-9L|,| ]+\)')
+if attrs is None:
+return alt_value
+attrs_str = str(attrs.get(attrs_name))
+if attrs_str is None:
+return alt_value
+attrs_match = tuple_re.search(attrs_str)
+if attrs_match is not None:
+if attrs_match.span() == (0, len(attrs_str)):
+dims = eval(attrs_str)
+return dims
+else:
+raise AttributeError("Malformed %s dimensions: %s" % (attrs_name,
str(attrs_str)))
+return alt_value
+
+
+@mx_op.register("Convolution")
+def convert_convolution(node, **kwargs):
+"""Map MXNet's convolution operator
[GitHub] spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export
spidyDev commented on a change in pull request #11213: [MXNET-533] MXNet-ONNX export URL: https://github.com/apache/incubator-mxnet/pull/11213#discussion_r194859380 ## File path: tests/python-pytest/onnx/export/backend.py ## @@ -0,0 +1,98 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review comment: mxnet unittests uses nosetests but onnx backend test framework uses pytest, so keep them separate we created another folder for pytests. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
