Hi Hagay, As mentioned on my previous thread, " MXNet has lots of users already using the Symbolic API which hopefully mean that is a mature API that is not likely to have breaking changes or major issues." actually indicate NNVM is stable. Because MXNet uses NNVM's symbolic.h directly, see https://github.com/dmlc/nnvm/blob/master/include/nnvm/symbolic.h
But I do agree that NNVM/TOP is newer than MXNet/Operator. But NNVM/TOP is designed according to the lessons we learned in the past two years, and we are pushing it to be the standard for the mxnet ecosystem. I didn't see the value to implement the convert based on the older MXNet/Operator interface. It does not align with the strategic roadmap of TVM. It could waste our engineers' value time because we need to re-do it later. Also, it disobeys the LP "Insist on the Highest Standards" given the benefits of NNVM/TOP Tianqi explained. One concern is developers may be less familiar with NNVM/TOP, so it could take more days to work on the NNVM/TOP approach. But the community can help on it. On Wed, Oct 18, 2017 at 4:57 PM, Lupesko, Hagay <lupe...@gmail.com> wrote: > Roshani – this is an exciting initiative, ONNX support on MXNet will > enable more users to ramp up on MXNet, which is great. > > Tianqi – a few questions and thoughts about your note: > - “More hardware backends to mxnet” – MXNet users get the same benefit of > HW support implementing ONNX import on top of MXNet symbolic, right? > - “NNVM Compiler now received contributions from AWS, UW and many other > folks in MXNet community.” – agreed it is ramping up, but when you look at > the data, it is clear that it is very early on for NNVM. Looking at the > repo, it has overall 223 commits, 0 releases. Compare it to MXNet with 6136 > commits and 32 releases. It seems to be still early on for NNVM, and for a > more reliable initial implementation building the import on top of MXNet is > easier, faster and safer. MXNet has lots of users already using the > Symbolic API which hopefully mean that is a mature API that is not likely > to have breaking changes or major issues. > > I’m supportive option 1 proposed by Roshani (building serde on top of > MXNet symbolic), but to do it as an encapsulated implementation detail, so > the implementation can be migrated to NNVM or another implementation in the > future, if at that point it seems like the right thing to do. > > Interested in hearing other opinions though… > > Hagay > > On 10/18/17, 14:13, "Tianqi Chen" <workc...@gmail.com on behalf of > tqc...@cs.washington.edu> wrote: > > I am strongly recommending going through the nnvm/top. One major > reason in > here is that the support of nnvm/top layer NOT ONLY mean compatibility > of > model format with onnx. These are the major benefits: > > > - More hardware backends to mxnet, including opencl, metal, Raspberry > Pi, > web browser. These things are automatically enabled by going through > this > layer. In general, we design nnvm/tvm stack to resolve the challenge of > current mxnet's weakness in terms deploying to more hardware backends. > > - More frontend capabilities, nnvm's gluon style IR ingests now from > CoreML, ONNX and in future keras. Supporting those will reduce the > amount > of engineering effort needed. > > - Future compatibility. We all agree that the future being migrated to > gluon's API. NNVM/top tries to look ahead by directly adopting the > symbolic > API to be gluon. > > > I would also like to correct some of the mentioned facts with regard to > nnvm/tvm stack > > 1. Nascent project with few contributors > > NNVM Compiler now received contributions from AWS, UW and many other > folks > in MXNet community. NNVM itself is already being used by MXNet. > MXNet's internal IR is migrating toward gluon, and its final form being > nnvm/top > > 3. Does not support all operators that exist in MXNet Symbolic API > > Neither NNVM/top or onnx support all operators that exist in mxnet > symbolic > API. The end goal here is mainly to make nnvm/top onnx compatible, > which is > a more reasonable goal. > > 4. No CI Pipeline and testcases > > NNVM already contains a compiler contains unittests and ci tested with > integration https://github.com/dmlc/nnvm, with a CI pipline that is > well > tested on CPU and GPU cases for front-ends. > > Tianqi > > > On Wed, Oct 18, 2017 at 1:41 PM, Roshani Nagmote < > roshaninagmo...@gmail.com> > wrote: > > > Hi guys, > > > > > > I am working on supporting ONNX <https://github.com/onnx/onnx> > pre-trained > > models in Apache MXNet and would like to seek your opinion on the > choice of > > implementation. I also have created a GitHub issue > > <https://github.com/apache/incubator-mxnet/issues/8319>. Supporting > ONNX > > in > > MXNet will enable users to move between frameworks with their > models, this > > will also enable MXNet project to be a part of the ONNX open > standard and > > steer the direction of ONNX. > > > > > > For those who don’t know ONNX, ONNX is an open source format for AI > models > > which enables models to be transferred between frameworks. Refer to > > https://github.com/onnx/onnx for more details. > > > > > > To implement the import/export functionality in MXNet, I propose to > expose > > a MXNet python module “serde”(name taken from Apache Hive project) > with the > > following methods supporting different formats: > > > > sym, params = mxnet.serde.import(other_format_file, > other_format=‘onnx’) > > > > other_format_file = mxnet.serde.export(mxnet_sym, mxnet_params, > ‘onnx’) > > > > > > The implementation under the hood can be done in two ways: > > > > > > 1) Implement at the MXNet layer by parsing the ONNX model(in protobuf > > format) and turn into MXNet Symbolic operators and build MXNet model > > directly. Similarly, I can convert the MXNet model to ONNX format at > this > > layer. > > > > > > 2) The DMLC community has released the nnvm/tvm complier and an > > intermediate representation of the models, refer: > > http://www.tvmlang.org/2017/10/06/nnvm/tvm-compiler- > announcement.html > > <http://www.tvmlang.org/2017/10/06/nnvm-compiler-announcement.html> > > > > Based on the conversation on the GitHub issue > > <https://github.com/apache/incubator-mxnet/issues/8319> I opened, Mu > > mentioned that MXNet would use nnvm/tvm as the backend in the future. > > > > > > We could hook into this layer to implement the import/export > functionality. > > nnvm/tvm has ONNX 0.1 version import implemented. > > > > For import, > > > > 1. > > > > I will need to enhance nnvm/tvm’s importer to support ONNX 0.2 > > 2. > > > > Implement nnvm/tvm->mxnet symbolic operators. > > > > For export: > > > > > > 1. > > > > mxnet->nnvm/tvm ( nnvm/tvm provides this implementation already) > > 2. > > > > I will need to Implement nnvm/tvm>onnx. > > > > > > These are the pros and cons I see in the above approaches: > > > > 1. > > > > Import/export at mxnet layer > > > > Pros: > > > > 1. > > > > Stable APIs currently used by users. > > 2. > > > > Larger Apache MXNet community of contributors. > > 3. > > > > CI pipeline to catch bugs. > > 4. > > > > Comparatively less time to implement and put it in the hands of > the > > users. > > > > Cons: > > > > 1. > > > > In the future we may have to reimplement at the nnvm/tvm layer, > in case > > MXNet moves to the nnvm/tvm backend(assuming it will move). > > > > > > > > 1. > > > > Import/export at nnvm/tvm layer > > > > Pros: > > > > 1. > > > > Less engineering work in case mxnet moves to nnvm/tvm > > 2. > > > > nnvm/tvm would become a hub to convert to different formats. > > 3. > > > > nnvm operators are more in parity with mxnet’s gluon APIs this > could be > > useful in case Gluon becomes the only standard that MXNet will > support. > > > > Cons: > > > > 1. > > > > Nascent project with few contributors > > 2. > > > > Does not support all operators that exist in MXNet Symbolic API > > 3. > > > > No CI Pipeline > > 4. > > > > Current Apache MXNet project does not use nnvm/tvm backend > > 5. > > > > mxnet->nnvm/tvm backend needs more testing and user feedback. > > > > > > Any suggestions on both of these approaches? From user's > perspective, this > > will be an implementation detail that is not exposed. > > > > Thanks, > > > > Roshani > > > > > >
