I think "We" refer to "Tianqi and me" on this thread. On Thu, Oct 19, 2017 at 12:53 PM, Suneel Marthi <smar...@apache.org> wrote:
> I guess the whole discussion here is about - Who is 'We' in your email ? > > 'We thought there is a better way of doing this' > > It may just be misinterpretation or misunderstanding amongst folks here due > to language barrier. > > > On Thu, Oct 19, 2017 at 3:48 PM, Tianqi Chen <tqc...@cs.washington.edu> > wrote: > > > We thought there is a better way of doing this, by proposing nnvm as part > > of apache deep learning stack along with Mxnet. This is the reason why we > > did not simply move the repo over now > > > > Tianqi > > On Thu, Oct 19, 2017 at 12:43 PM Chris Olivier <cjolivie...@gmail.com> > > wrote: > > > > > Why don't we just move all of these dmlc modules into the Apache > > repository > > > right now and have the correct discussions on dev? What's the argument > > > against this? IMHO, I thought that's what was going to be done > > originally. > > > > > > On Thu, Oct 19, 2017 at 12:14 PM, Tianqi Chen < > tqc...@cs.washington.edu> > > > wrote: > > > > > > > Hi Hen: > > > > > > > > It is sad to think DMLC adversarially in this matter. DMLC projects > > > adopt > > > > apache way of doing things and we are planning moving more modules > into > > > > Apache. > > > > > > > > All the discussion so far happens under the Apache manner and I do > > think > > > > that healthy discussion on critical design issues is important. It is > > > > unfair to say something is rotten just when there is a debate going > on > > in > > > > terms of technical issues. > > > > > > > > They are merely based on our technical assessment of what is better > for > > > the > > > > project in general, instead of being political or chanting the > detailed > > > > credits or ownership of the code. > > > > > > > > > > > > Tianqi > > > > > > > > On Thu, Oct 19, 2017 at 12:03 PM, Hen <bay...@apache.org> wrote: > > > > > > > > > What I think I'm seeing here is that: > > > > > > > > > > * MXNet moved to Apache. > > > > > * Some of the code it relied on (50% per the last release thread, > but > > > > that > > > > > may have been bombastic) remained at DMLC. > > > > > * The MXNet community thinks one thing. > > > > > * The DMLC community (which is a subset of the MXNet community that > > > runs > > > > > under different community rules) thinks another. > > > > > > > > > > Something is rotten. > > > > > > > > > > One solution: The MXNet community forks the DMLC code it relies on > > into > > > > the > > > > > MXNet codebase and moves on without being tied down by the > decisions > > > of a > > > > > non-compatible community. > > > > > > > > > > Hen > > > > > > > > > > > > > > > > > > > > On Thu, Oct 19, 2017 at 11:59 AM, Tianqi Chen < > > > tqc...@cs.washington.edu> > > > > > wrote: > > > > > > > > > > > Here are the detailed points(sorry for resenting it over again) > > > > > > > > > > > > Technical Reasoning: > > > > > > > > > > > > - Model exchange format like CoreML and ONNX are not lossless > and > > > > > > complete. They are designed to an contain a core set of the > > > > > > minimum operators to support necessary inference tasks like > ResNet, > > > > etc. > > > > > > So you cannot rely on a bi-directional serialization with this > > format > > > > for > > > > > > all MXNet models. As a simple example, broadcast add/mul is > simply > > > not > > > > > > supported in onnx. > > > > > > > > > > > > - Same problem goes for compilation and in-memory IR, a core set > of > > > > most > > > > > > interesting primitives are effectively supported. > > > > > > > > > > > > - Either in the case of supporting exchange format, or in-memory > > IR, > > > we > > > > > > need to make the decision on what core set of operators are we > > > > interested > > > > > > in support. We cannot simply say let us support everything from > > the > > > > > > beginning due to the limitations of the exchange format. > > > > > > > > > > > > - It is crucial for us articulate what is the core set of > operators > > > we > > > > > care > > > > > > about in MXNet. Either in terms of providing guidelines to the > > > > community, > > > > > > or influence the design of model exchange format them-selfs to > move > > > in > > > > > > favor of MXNet. > > > > > > > > > > > > - nnvm/top is that initial core set of operators for both > compiler > > > > > support > > > > > > and exchange purposes. It is modeled under numpy and gluon, under > > the > > > > > > supervision of Eric, Me and Mu. It can be bi-directionally > > exchanged > > > > > with > > > > > > a current mxnet operator without loss of information. > > > > > > > > > > > > The Effort of Engineering: > > > > > > > > > > > > - Because nnvm/top is modeled with numpy and gluon, mxnet<-> > > nnvm/top > > > > is > > > > > > quite easy, and we already have one direction done. I would be > very > > > > happy > > > > > > to answer any questions on another. No information loss will > happen > > > > with > > > > > > this path. > > > > > > > > > > > > - mxnet/symbol or nnvm/symbol(they are essentially the same thing > > > with > > > > a > > > > > > bit different op defs) <- onnx is harder. There has been already > > > enough > > > > > > effort to support onnx 0.1 as Roshani mentioned. Which is > > contributed > > > > by > > > > > > Zhi Zhang, another Apache MXNet committer. Zhi already provided > > code > > > to > > > > > > alleviate this process. Built code on the existing effort would > > > > actually > > > > > > make the problem easier. > > > > > > > > > > > > On Thu, Oct 19, 2017 at 11:55 AM, Tianqi Chen < > > > > tqc...@cs.washington.edu> > > > > > > wrote: > > > > > > > > > > > > > As for where the code should sit, we have seen onnx's support > for > > > > > caffe2 > > > > > > > sitting on a separate repo. My suggestion would be put code > > under > > > > > > nnvm/top > > > > > > > and migrate into mxnet eventually when the top components get > > into > > > > > MXNet, > > > > > > > hopefully by end of next month. > > > > > > > > > > > > > > I have elaborated my point in the last email thread. This > (going > > > > > through > > > > > > > nnvm/top) is an important design decision both technically > > > > > (compilation, > > > > > > > more hardware) and strategically (articulate our core set of > > > > operators > > > > > > and > > > > > > > influence the model exchange format). > > > > > > > > > > > > > > I am glad to see the discussion happening and surely there is > > > doubt, > > > > as > > > > > > > with every big step of changes. But with the rapidly changing > > pace > > > > of > > > > > > deep > > > > > > > learning systems, this is the direction that we thought is most > > > > > > promising. > > > > > > > We can call for a vote if necessary among the committers for > the > > > > design > > > > > > > decision if there is still debate on this issue. Or we can keep > > the > > > > > > > discussion open and start some effort around nnvm/top to see > how > > it > > > > > goes > > > > > > > > > > > > > > Tianqi > > > > > > > > > > > > > > On Thu, Oct 19, 2017 at 11:15 AM, Lupesko, Hagay < > > > lupe...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > >> Mu, > > > > > > >> > > > > > > >> You’re mentioning plans for a new model format and compiler, > > but I > > > > > don’t > > > > > > >> recall seeing it shared/discussed on the dev list. Can you > share > > > > > these, > > > > > > so > > > > > > >> it is more accessible to folks to understand the plan and > > vision? > > > > > > >> > > > > > > >> Personally, I think it will be a shame to add ONNX support to > > > MXNet, > > > > > and > > > > > > >> have it implemented outside of MXNet. At the end of the day, > it > > > > makes > > > > > > >> things difficult for MXNet users. > > > > > > >> > > > > > > >> Hagay > > > > > > >> > > > > > > >> On 10/19/17, 10:01, "Mu Li" <limu...@gmail.com on behalf of > > > > > > >> muli....@gmail.com> wrote: > > > > > > >> > > > > > > >> I'm speaking under my "MXNet contributor" hat. > > > > > > >> > > > > > > >> It will be sad that our new model format and compiler is > not > > > > > > >> supported by > > > > > > >> our own contributors. It puts us in a bad position to > reach > > > out > > > > to > > > > > > >> outside > > > > > > >> to ask for support. > > > > > > >> > > > > > > >> If you really what to do it with the onnx <-> mxnet way, I > > > > suggest > > > > > > >> putting > > > > > > >> the codes under https://github.com/aws. > > > > > > >> > > > > > > >> Best > > > > > > >> Mu > > > > > > >> > > > > > > >> On Thu, Oct 19, 2017 at 9:51 AM, Lupesko, Hagay < > > > > > lupe...@gmail.com> > > > > > > >> wrote: > > > > > > >> > > > > > > >> > Since there seems to be a difficulty to reach a > consensus > > > > here, > > > > > > and > > > > > > >> this > > > > > > >> > is a new area, maybe a good compromise would be to > > > contribute > > > > > this > > > > > > >> under > > > > > > >> > /contrib as experimental, with whatever way Roshani > thinks > > > > makes > > > > > > >> sense. > > > > > > >> > Once there is code in place, and MXNet users and > > > contributors > > > > > are > > > > > > >> able to > > > > > > >> > check it out, we can consider future steps. > > > > > > >> > > > > > > > >> > Does this proposal make sense to folks? > > > > > > >> > > > > > > > >> > On 10/18/17, 23:01, "Tianqi Chen" <workc...@gmail.com > on > > > > behalf > > > > > > of > > > > > > >> > tqc...@cs.washington.edu> wrote: > > > > > > >> > > > > > > > >> > I want to offer one last thing in terms of technical > > > > > details. > > > > > > I > > > > > > >> > mentioned > > > > > > >> > two trends in the deep learning systems. There is > one > > > last > > > > > > >> thing that > > > > > > >> > is > > > > > > >> > omitted. How should we build a good deploy end for > > deep > > > > > > learning > > > > > > >> > models. > > > > > > >> > > > > > > > >> > There is always a paradox to this problem: > > > > > > >> > > > > > > > >> > - On one hand, the deployment end needs to be > > > lightweight > > > > > and > > > > > > >> portable. > > > > > > >> > - We want a lot of optimizations (memory layout > > compute) > > > > and > > > > > > >> feature > > > > > > >> > support, this makes the project big. > > > > > > >> > > > > > > > >> > All the existing systems suffer from this problem. > The > > > > > > solution > > > > > > >> is > > > > > > >> > simple, > > > > > > >> > separates the optimization part from the actual > > runtime > > > > and > > > > > > >> compiles > > > > > > >> > the > > > > > > >> > things down to a bare metal module. And this is the > > > > solution > > > > > > >> nnvm/top > > > > > > >> > compiler pipeline offer, which I believe will > become a > > > > > > standard > > > > > > >> > practice of > > > > > > >> > deployment and where all systems go to > > > > > > >> > > > > > > > >> > Tianqi > > > > > > >> > > > > > > > >> > On Wed, Oct 18, 2017 at 10:03 PM, Tianqi Chen < > > > > > > >> > tqc...@cs.washington.edu> > > > > > > >> > wrote: > > > > > > >> > > > > > > > >> > > OK, there is some miscommunication in here I > guess. > > > We > > > > > only > > > > > > >> need to > > > > > > >> > do a > > > > > > >> > > "canonization" step in python API that goes a > symbol > > > to > > > > > > symbol > > > > > > >> > translation > > > > > > >> > > layer. It can be done in purely in python, and > there > > > is > > > > no > > > > > > >> need for > > > > > > >> > going > > > > > > >> > > "down" into c++ to do this. > > > > > > >> > > > > > > > > >> > > For example, the current nnvm.from_mxnet API takes > > > > Module > > > > > or > > > > > > >> Gluon > > > > > > >> > module > > > > > > >> > > and get you back nnvm/top graph in python. > > > > > > >> > > > > > > > > >> > > All we are asking for is to decomposing it into > > > > > > >> > > > > > > > > >> > > def mxnet_to_onnx(module): > > > > > > >> > > nnvm_graph, params = nnvm_from_mxnet(module) > > > > > > >> > > onnx = nnvm_to_onnx(nnvm_graph, params) > > > > > > >> > > return onnx > > > > > > >> > > > > > > > > >> > > This allows nnvm_from_mxnet to be reused for other > > > > > purposes, > > > > > > >> like > > > > > > >> > > compiling API to deployable modules > > > > > > >> > > > > > > > > >> > > Tianqi > > > > > > >> > > > > > > > > >> > > On Wed, Oct 18, 2017 at 9:55 PM, Lupesko, Hagay < > > > > > > >> lupe...@gmail.com> > > > > > > >> > wrote: > > > > > > >> > > > > > > > > >> > >> Tianqi: > > > > > > >> > >> Thanks for detailing the trends. I fully agree > that > > > > ONNX > > > > > is > > > > > > >> just a > > > > > > >> > graph > > > > > > >> > >> serialization format – nothing more, nothing > less. > > I > > > > also > > > > > > >> think we > > > > > > >> > all > > > > > > >> > >> agree that this simple mechanism holds lots of > > value > > > to > > > > > DL > > > > > > >> users > > > > > > >> > since it > > > > > > >> > >> allows them to move between frameworks easily > (e.g. > > > > train > > > > > > >> with > > > > > > >> > MXNet, > > > > > > >> > >> deploy on a mobile device with Caffe2, or the > other > > > way > > > > > > >> around). > > > > > > >> > >> As you said, In Memory IR is different than > > > > serialization > > > > > > >> formats > > > > > > >> > such as > > > > > > >> > >> ONNX. They are designed to make the runtime > > execution > > > > as > > > > > > >> efficient > > > > > > >> > as > > > > > > >> > >> possible, leveraging software and hardware > > > > optimizations. > > > > > > >> They are > > > > > > >> > indeed > > > > > > >> > >> complex, and where the “meat” is. > > > > > > >> > >> (BTW ONNX regards itself as an “IR” format, but > not > > > in > > > > > the > > > > > > >> same > > > > > > >> > sense as > > > > > > >> > >> NNVM). > > > > > > >> > >> > > > > > > >> > >> At the end of the day, Roshani is aiming to > > deliver a > > > > > > simple > > > > > > >> > >> functionality to MXNet users: (1) take an ONNX > > file, > > > > and > > > > > > >> load it > > > > > > >> > into MXNet > > > > > > >> > >> so you get a graph+weights you can work with (2) > > > Given > > > > a > > > > > > >> trained > > > > > > >> > model, > > > > > > >> > >> save it as an ONNX file. Since MXNet users do not > > > > > interact > > > > > > >> with NNVM > > > > > > >> > >> directly, but rather interact with MXNet API > (MXNet > > > > > > Module), > > > > > > >> isn’t > > > > > > >> > the > > > > > > >> > >> simplest thing to do is just to construct the > > Module > > > > “on > > > > > > the > > > > > > >> fly” > > > > > > >> > using > > > > > > >> > >> MXNet API? Taking the other approach, we will go > > from > > > > the > > > > > > >> top level > > > > > > >> > MXNet > > > > > > >> > >> “load” API, go “down” to NNVM to construct the > > graph, > > > > go > > > > > > >> back up to > > > > > > >> > MXNet > > > > > > >> > >> to expose it as a Module. This seems to complex > and > > > > does > > > > > > not > > > > > > >> add any > > > > > > >> > >> benefit. In whatever way we construct the MXNet > > > Module > > > > > > >> object, NNVM > > > > > > >> > will > > > > > > >> > >> always be the underlying in memory IR that is > being > > > > > > >> executed, so > > > > > > >> > why not > > > > > > >> > >> take the simpler route? > > > > > > >> > >> > > > > > > >> > >> Hagay > > > > > > >> > >> > > > > > > >> > >> On 10/18/17, 19:42, "Tianqi Chen" < > > > workc...@gmail.com > > > > on > > > > > > >> behalf of > > > > > > >> > >> tqc...@cs.washington.edu> wrote: > > > > > > >> > >> > > > > > > >> > >> Hi Chris: > > > > > > >> > >> > > > > > > >> > >> There is no intention to move things away > from > > > > mxnet. > > > > > > The > > > > > > >> > reduction of > > > > > > >> > >> lines of code by having a better design in > > > general, > > > > > and > > > > > > >> > usually, you > > > > > > >> > >> write > > > > > > >> > >> less redundant code by benefiting from better > > > > design. > > > > > > As > > > > > > >> I may > > > > > > >> > quote: > > > > > > >> > >> "the > > > > > > >> > >> best design is not achieved not when there is > > > > nothing > > > > > > to > > > > > > >> add, > > > > > > >> > but when > > > > > > >> > >> there is nothing to be taken away." > > > > > > >> > >> > > > > > > >> > >> MXNet has always benefited from this > philosophy > > > and > > > > > > >> improves > > > > > > >> > with the > > > > > > >> > >> new > > > > > > >> > >> designs and proper modularization. For > example, > > > we > > > > > see > > > > > > >> such > > > > > > >> > reduction > > > > > > >> > >> and > > > > > > >> > >> convenience happening when migrating from > > MXNet's > > > > > > legacy > > > > > > >> op to > > > > > > >> > the > > > > > > >> > >> NNVM's mechanism. The new mechanism now > enables > > > > > things > > > > > > >> like > > > > > > >> > sparse > > > > > > >> > >> aware > > > > > > >> > >> support and other stuff which would be much > > > harder > > > > to > > > > > > >> support. > > > > > > >> > >> > > > > > > >> > >> The nnvm/tvm stack comes brings the same > > > benefit(if > > > > > not > > > > > > >> more) > > > > > > >> > and it > > > > > > >> > >> will > > > > > > >> > >> only add more features to MXNet itself. > > Offering > > > > more > > > > > > >> hardware > > > > > > >> > >> backends and > > > > > > >> > >> optimization, allowing us to write less code > > and > > > > > spent > > > > > > >> less > > > > > > >> > time to > > > > > > >> > >> optimize for each backend by going through > TVM > > > > > > >> > >> > > > > > > >> > >> Tianqi > > > > > > >> > >> > > > > > > >> > >> On Wed, Oct 18, 2017 at 7:15 PM, Chris > Olivier > > < > > > > > > >> > cjolivie...@gmail.com > > > > > > >> > >> > > > > > > > >> > >> wrote: > > > > > > >> > >> > > > > > > >> > >> > Reduce code base of mxnet? By increasing > > scope > > > of > > > > > the > > > > > > >> dmlc > > > > > > >> > modules? > > > > > > >> > >> Is the > > > > > > >> > >> > intent to make mxnet a thin language > wrapper > > > > > around a > > > > > > >> group > > > > > > >> > of dmlc > > > > > > >> > >> > modules? > > > > > > >> > >> > > > > > > > >> > >> > > > > > > > >> > >> > On Wed, Oct 18, 2017 at 6:58 PM Tianqi > Chen < > > > > > > >> > >> tqc...@cs.washington.edu> > > > > > > >> > >> > wrote: > > > > > > >> > >> > > > > > > > >> > >> > > To better answer Hagay's question, I > would > > > like > > > > > to > > > > > > >> dive > > > > > > >> > down a > > > > > > >> > >> bit deeper > > > > > > >> > >> > > on the relation between MXNet, NNVM and > > model > > > > > > >> exchange > > > > > > >> > format > > > > > > >> > >> like ONNX. > > > > > > >> > >> > > > > > > > > >> > >> > > There are two major trends in deep > learning > > > > > systems > > > > > > >> now: > > > > > > >> > >> > > > > > > > > >> > >> > > - Common serializable formats, like ONNX > > and > > > > > > CoreML, > > > > > > >> that > > > > > > >> > defines > > > > > > >> > >> the > > > > > > >> > >> > model > > > > > > >> > >> > > exchange format. > > > > > > >> > >> > > - The in-memory graph IR for quick > > > optimization > > > > > and > > > > > > >> JIT. > > > > > > >> > NNVM, > > > > > > >> > >> > Tensorflow's > > > > > > >> > >> > > XLA falls into this category. > > > > > > >> > >> > > > > > > > > >> > >> > > The exchange formats are great, it only > > > poses a > > > > > > >> layer of > > > > > > >> > >> conversion, > > > > > > >> > >> > which > > > > > > >> > >> > > is good for exchange. The real meat still > > > comes > > > > > > from > > > > > > >> the > > > > > > >> > >> compilation and > > > > > > >> > >> > > JIT pipeline you have to offer. For that, > > we > > > > will > > > > > > >> need an > > > > > > >> > >> in-memory IR, > > > > > > >> > >> > > because of the cost of constructing, > > > serialize > > > > > > could > > > > > > >> be > > > > > > >> > high for > > > > > > >> > >> the > > > > > > >> > >> > > exchange formats like protobuf. And > > usually, > > > > the > > > > > > >> exchange > > > > > > >> > >> formats are > > > > > > >> > >> > > designed in a minimalistic fashion, > making > > it > > > > > less > > > > > > >> easy to > > > > > > >> > extend > > > > > > >> > >> more > > > > > > >> > >> > > information to support in-depth > > optimization > > > > like > > > > > > >> automatic > > > > > > >> > >> quantization, > > > > > > >> > >> > > accelerator support. > > > > > > >> > >> > > > > > > > > >> > >> > > The current MXNet relies on NNVM for > > > in-memory > > > > IR > > > > > > >> > manipulation > > > > > > >> > >> but does > > > > > > >> > >> > not > > > > > > >> > >> > > contain a compilation component that > > compiles > > > > to > > > > > > the > > > > > > >> > hardware > > > > > > >> > >> backends. > > > > > > >> > >> > > Doing export to an exchange format and > then > > > > back > > > > > > >> into NNVM > > > > > > >> > run the > > > > > > >> > >> > > compilation poses too much burden that > JIT > > > > > compiler > > > > > > >> could > > > > > > >> > pay. > > > > > > >> > >> Using the > > > > > > >> > >> > > same in-memory graph IR as the > compilation > > > > stack > > > > > > >> give much > > > > > > >> > more > > > > > > >> > >> advantage > > > > > > >> > >> > > in terms of this. > > > > > > >> > >> > > > > > > > > >> > >> > > The newly introduces nnvm/top and > compiler > > > > offers > > > > > > >> in-memory > > > > > > >> > graph > > > > > > >> > >> > > optimization and compilation and offers > > more > > > > > > hardware > > > > > > >> > backend > > > > > > >> > >> directly > > > > > > >> > >> > via > > > > > > >> > >> > > TVM. We already see promising results in > > edge > > > > > > >> deployments > > > > > > >> > with a > > > > > > >> > >> much > > > > > > >> > >> > lower > > > > > > >> > >> > > overhead of runtime. We will further > > benefit > > > > > > quickly > > > > > > >> from > > > > > > >> > more > > > > > > >> > >> graph > > > > > > >> > >> > > optimizations that it has to offer. > > > > > > >> > >> > > > > > > > > >> > >> > > Building support around this new paradigm > > > > offers > > > > > us > > > > > > >> > advantage of > > > > > > >> > >> being > > > > > > >> > >> > > future compatible and takes full benefit > of > > > the > > > > > > >> points I > > > > > > >> > >> mentioned above > > > > > > >> > >> > > > > > > > > >> > >> > > Tianqi > > > > > > >> > >> > > > > > > > > >> > >> > > > > > > > > >> > >> > > > > > > > > >> > >> > > On Wed, Oct 18, 2017 at 4:57 PM, Lupesko, > > > > Hagay < > > > > > > >> > >> lupe...@gmail.com> > > > > > > >> > >> > wrote: > > > > > > >> > >> > > > > > > > > >> > >> > > > Roshani – this is an exciting > initiative, > > > > ONNX > > > > > > >> support on > > > > > > >> > MXNet > > > > > > >> > >> will > > > > > > >> > >> > > > enable more users to ramp up on MXNet, > > > which > > > > is > > > > > > >> great. > > > > > > >> > >> > > > > > > > > > >> > >> > > > Tianqi – a few questions and thoughts > > about > > > > > your > > > > > > >> note: > > > > > > >> > >> > > > - “More hardware backends to mxnet” – > > MXNet > > > > > users > > > > > > >> get the > > > > > > >> > same > > > > > > >> > >> benefit > > > > > > >> > >> > of > > > > > > >> > >> > > > HW support implementing ONNX import on > > top > > > of > > > > > > MXNet > > > > > > >> > symbolic, > > > > > > >> > >> right? > > > > > > >> > >> > > > - “NNVM Compiler now received > > contributions > > > > > from > > > > > > >> AWS, UW > > > > > > >> > and > > > > > > >> > >> many other > > > > > > >> > >> > > > folks in MXNet community.” – agreed it > is > > > > > ramping > > > > > > >> up, but > > > > > > >> > when > > > > > > >> > >> you look > > > > > > >> > >> > > at > > > > > > >> > >> > > > the data, it is clear that it is very > > early > > > > on > > > > > > for > > > > > > >> NNVM. > > > > > > >> > >> Looking at the > > > > > > >> > >> > > > repo, it has overall 223 commits, 0 > > > releases. > > > > > > >> Compare it > > > > > > >> > to > > > > > > >> > >> MXNet with > > > > > > >> > >> > > 6136 > > > > > > >> > >> > > > commits and 32 releases. It seems to be > > > still > > > > > > >> early on for > > > > > > >> > >> NNVM, and > > > > > > >> > >> > for > > > > > > >> > >> > > a > > > > > > >> > >> > > > more reliable initial implementation > > > building > > > > > the > > > > > > >> import > > > > > > >> > on top > > > > > > >> > >> of > > > > > > >> > >> > MXNet > > > > > > >> > >> > > is > > > > > > >> > >> > > > easier, faster and safer. MXNet has > lots > > of > > > > > users > > > > > > >> already > > > > > > >> > using > > > > > > >> > >> the > > > > > > >> > >> > > > Symbolic API which hopefully mean that > > is a > > > > > > mature > > > > > > >> API > > > > > > >> > that is > > > > > > >> > >> not > > > > > > >> > >> > likely > > > > > > >> > >> > > > to have breaking changes or major > issues. > > > > > > >> > >> > > > > > > > > > >> > >> > > > I’m supportive option 1 proposed by > > Roshani > > > > > > >> (building > > > > > > >> > serde on > > > > > > >> > >> top of > > > > > > >> > >> > > > MXNet symbolic), but to do it as an > > > > > encapsulated > > > > > > >> > implementation > > > > > > >> > >> detail, > > > > > > >> > >> > > so > > > > > > >> > >> > > > the implementation can be migrated to > > NNVM > > > or > > > > > > >> another > > > > > > >> > >> implementation in > > > > > > >> > >> > > the > > > > > > >> > >> > > > future, if at that point it seems like > > the > > > > > right > > > > > > >> thing to > > > > > > >> > do. > > > > > > >> > >> > > > > > > > > > >> > >> > > > Interested in hearing other opinions > > > though… > > > > > > >> > >> > > > > > > > > > >> > >> > > > Hagay > > > > > > >> > >> > > > > > > > > > >> > >> > > > On 10/18/17, 14:13, "Tianqi Chen" < > > > > > > >> workc...@gmail.com on > > > > > > >> > >> behalf of > > > > > > >> > >> > > > tqc...@cs.washington.edu> wrote: > > > > > > >> > >> > > > > > > > > > >> > >> > > > I am strongly recommending going > > > through > > > > > the > > > > > > >> > nnvm/top. One > > > > > > >> > >> major > > > > > > >> > >> > > > reason in > > > > > > >> > >> > > > here is that the support of > nnvm/top > > > > layer > > > > > > NOT > > > > > > >> ONLY > > > > > > >> > mean > > > > > > >> > >> > > compatibility > > > > > > >> > >> > > > of > > > > > > >> > >> > > > model format with onnx. These are > the > > > > major > > > > > > >> benefits: > > > > > > >> > >> > > > > > > > > > >> > >> > > > > > > > > > >> > >> > > > - More hardware backends to mxnet, > > > > > including > > > > > > >> opencl, > > > > > > >> > metal, > > > > > > >> > >> > Raspberry > > > > > > >> > >> > > > Pi, > > > > > > >> > >> > > > web browser. These things are > > > > automatically > > > > > > >> enabled > > > > > > >> > by going > > > > > > >> > >> > through > > > > > > >> > >> > > > this > > > > > > >> > >> > > > layer. In general, we design > nnvm/tvm > > > > stack > > > > > > to > > > > > > >> > resolve the > > > > > > >> > >> > challenge > > > > > > >> > >> > > of > > > > > > >> > >> > > > current mxnet's weakness in terms > > > > deploying > > > > > > to > > > > > > >> more > > > > > > >> > hardware > > > > > > >> > >> > > backends. > > > > > > >> > >> > > > > > > > > > >> > >> > > > - More frontend capabilities, > nnvm's > > > > gluon > > > > > > >> style IR > > > > > > >> > ingests > > > > > > >> > >> now > > > > > > >> > >> > from > > > > > > >> > >> > > > CoreML, ONNX and in future keras. > > > > > Supporting > > > > > > >> those > > > > > > >> > will > > > > > > >> > >> reduce the > > > > > > >> > >> > > > amount > > > > > > >> > >> > > > of engineering effort needed. > > > > > > >> > >> > > > > > > > > > >> > >> > > > - Future compatibility. We all > agree > > > that > > > > > the > > > > > > >> future > > > > > > >> > being > > > > > > >> > >> migrated > > > > > > >> > >> > > to > > > > > > >> > >> > > > gluon's API. NNVM/top tries to look > > > ahead > > > > > by > > > > > > >> directly > > > > > > >> > >> adopting the > > > > > > >> > >> > > > symbolic > > > > > > >> > >> > > > API to be gluon. > > > > > > >> > >> > > > > > > > > > >> > >> > > > > > > > > > >> > >> > > > I would also like to correct some > of > > > the > > > > > > >> mentioned > > > > > > >> > facts > > > > > > >> > >> with > > > > > > >> > >> > regard > > > > > > >> > >> > > to > > > > > > >> > >> > > > nnvm/tvm stack > > > > > > >> > >> > > > > > > > > > >> > >> > > > 1. Nascent project with few > > > > contributors > > > > > > >> > >> > > > > > > > > > >> > >> > > > NNVM Compiler now received > > > contributions > > > > > from > > > > > > >> AWS, UW > > > > > > >> > and > > > > > > >> > >> many > > > > > > >> > >> > other > > > > > > >> > >> > > > folks > > > > > > >> > >> > > > in MXNet community. NNVM itself is > > > > already > > > > > > >> being used > > > > > > >> > by > > > > > > >> > >> MXNet. > > > > > > >> > >> > > > MXNet's internal IR is migrating > > toward > > > > > > gluon, > > > > > > >> and its > > > > > > >> > >> final form > > > > > > >> > >> > > being > > > > > > >> > >> > > > nnvm/top > > > > > > >> > >> > > > > > > > > > >> > >> > > > 3. Does not support all operators > > > that > > > > > > exist > > > > > > >> in > > > > > > >> > MXNet > > > > > > >> > >> Symbolic > > > > > > >> > >> > API > > > > > > >> > >> > > > > > > > > > >> > >> > > > Neither NNVM/top or onnx support > all > > > > > > operators > > > > > > >> that > > > > > > >> > exist > > > > > > >> > >> in mxnet > > > > > > >> > >> > > > symbolic > > > > > > >> > >> > > > API. The end goal here is mainly to > > > make > > > > > > >> nnvm/top onnx > > > > > > >> > >> compatible, > > > > > > >> > >> > > > which is > > > > > > >> > >> > > > a more reasonable goal. > > > > > > >> > >> > > > > > > > > > >> > >> > > > 4. No CI Pipeline and testcases > > > > > > >> > >> > > > > > > > > > >> > >> > > > NNVM already contains a compiler > > > contains > > > > > > >> unittests > > > > > > >> > and ci > > > > > > >> > >> tested > > > > > > >> > >> > > with > > > > > > >> > >> > > > integration > > > > https://github.com/dmlc/nnvm, > > > > > > >> with a CI > > > > > > >> > >> pipline that > > > > > > >> > >> > is > > > > > > >> > >> > > > well > > > > > > >> > >> > > > tested on CPU and GPU cases for > > > > front-ends. > > > > > > >> > >> > > > > > > > > > >> > >> > > > Tianqi > > > > > > >> > >> > > > > > > > > > >> > >> > > > > > > > > > >> > >> > > > On Wed, Oct 18, 2017 at 1:41 PM, > > > Roshani > > > > > > >> Nagmote < > > > > > > >> > >> > > > roshaninagmo...@gmail.com> > > > > > > >> > >> > > > wrote: > > > > > > >> > >> > > > > > > > > > >> > >> > > > > Hi guys, > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > I am working on supporting ONNX < > > > > > > >> > >> https://github.com/onnx/onnx> > > > > > > >> > >> > > > pre-trained > > > > > > >> > >> > > > > models in Apache MXNet and would > > like > > > > to > > > > > > >> seek your > > > > > > >> > >> opinion on the > > > > > > >> > >> > > > choice of > > > > > > >> > >> > > > > implementation. I also have > > created a > > > > > > GitHub > > > > > > >> issue > > > > > > >> > >> > > > > <https://github.com/apache/ > > > > > > >> > incubator-mxnet/issues/8319>. > > > > > > >> > >> > > Supporting > > > > > > >> > >> > > > ONNX > > > > > > >> > >> > > > > in > > > > > > >> > >> > > > > MXNet will enable users to move > > > between > > > > > > >> frameworks > > > > > > >> > with > > > > > > >> > >> their > > > > > > >> > >> > > > models, this > > > > > > >> > >> > > > > will also enable MXNet project to > > be > > > a > > > > > part > > > > > > >> of the > > > > > > >> > ONNX > > > > > > >> > >> open > > > > > > >> > >> > > > standard and > > > > > > >> > >> > > > > steer the direction of ONNX. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > For those who don’t know ONNX, > ONNX > > > is > > > > an > > > > > > >> open > > > > > > >> > source > > > > > > >> > >> format for > > > > > > >> > >> > AI > > > > > > >> > >> > > > models > > > > > > >> > >> > > > > which enables models to be > > > transferred > > > > > > >> between > > > > > > >> > >> frameworks. Refer > > > > > > >> > >> > to > > > > > > >> > >> > > > > https://github.com/onnx/onnx for > > > more > > > > > > >> details. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > To implement the import/export > > > > > > functionality > > > > > > >> in > > > > > > >> > MXNet, I > > > > > > >> > >> propose > > > > > > >> > >> > to > > > > > > >> > >> > > > expose > > > > > > >> > >> > > > > a MXNet python module > “serde”(name > > > > taken > > > > > > from > > > > > > >> > Apache Hive > > > > > > >> > >> > project) > > > > > > >> > >> > > > with the > > > > > > >> > >> > > > > following methods supporting > > > different > > > > > > >> formats: > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > sym, params = > > > > > > mxnet.serde.import(other_forma > > > > > > >> t_file, > > > > > > >> > >> > > > other_format=‘onnx’) > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > other_format_file = > > > > > > >> mxnet.serde.export(mxnet_sym, > > > > > > >> > >> mxnet_params, > > > > > > >> > >> > > > ‘onnx’) > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > The implementation under the hood > > can > > > > be > > > > > > >> done in > > > > > > >> > two ways: > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > 1) Implement at the MXNet layer > by > > > > > parsing > > > > > > >> the ONNX > > > > > > >> > >> model(in > > > > > > >> > >> > > protobuf > > > > > > >> > >> > > > > format) and turn into MXNet > > Symbolic > > > > > > >> operators and > > > > > > >> > build > > > > > > >> > >> MXNet > > > > > > >> > >> > > model > > > > > > >> > >> > > > > directly. Similarly, I can > convert > > > the > > > > > > MXNet > > > > > > >> model > > > > > > >> > to > > > > > > >> > >> ONNX format > > > > > > >> > >> > > at > > > > > > >> > >> > > > this > > > > > > >> > >> > > > > layer. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > 2) The DMLC community has > released > > > the > > > > > > >> nnvm/tvm > > > > > > >> > complier > > > > > > >> > >> and an > > > > > > >> > >> > > > > intermediate representation of > the > > > > > models, > > > > > > >> refer: > > > > > > >> > >> > > > > http://www.tvmlang.org/2017/ > > > > > > >> > 10/06/nnvm/tvm-compiler- > > > > > > >> > >> > > > announcement.html > > > > > > >> > >> > > > > <http://www.tvmlang.org/2017/1 > > > > > > >> 0/06/nnvm-compiler- > > > > > > >> > >> > announcement.html > > > > > > >> > >> > > > > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > Based on the conversation on the > > > GitHub > > > > > > issue > > > > > > >> > >> > > > > <https://github.com/apache/ > > > > > > >> > incubator-mxnet/issues/8319> I > > > > > > >> > >> > opened, > > > > > > >> > >> > > Mu > > > > > > >> > >> > > > > mentioned that MXNet would use > > > nnvm/tvm > > > > > as > > > > > > >> the > > > > > > >> > backend in > > > > > > >> > >> the > > > > > > >> > >> > > future. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > We could hook into this layer to > > > > > implement > > > > > > >> the > > > > > > >> > >> import/export > > > > > > >> > >> > > > functionality. > > > > > > >> > >> > > > > nnvm/tvm has ONNX 0.1 version > > import > > > > > > >> implemented. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > For import, > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > 1. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > I will need to enhance > > nnvm/tvm’s > > > > > > >> importer to > > > > > > >> > support > > > > > > >> > >> ONNX 0.2 > > > > > > >> > >> > > > > 2. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > Implement nnvm/tvm->mxnet > > symbolic > > > > > > >> operators. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > For export: > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > 1. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > mxnet->nnvm/tvm ( nnvm/tvm > > > provides > > > > > this > > > > > > >> > implementation > > > > > > >> > >> > already) > > > > > > >> > >> > > > > 2. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > I will need to Implement > > > > > nnvm/tvm>onnx. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > These are the pros and cons I see > > in > > > > the > > > > > > >> above > > > > > > >> > approaches: > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > 1. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > Import/export at mxnet layer > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > Pros: > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > 1. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > Stable APIs currently used by > > > users. > > > > > > >> > >> > > > > 2. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > Larger Apache MXNet community > of > > > > > > >> contributors. > > > > > > >> > >> > > > > 3. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > CI pipeline to catch bugs. > > > > > > >> > >> > > > > 4. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > Comparatively less time to > > > implement > > > > > and > > > > > > >> put it > > > > > > >> > in the > > > > > > >> > >> hands > > > > > > >> > >> > of > > > > > > >> > >> > > > the > > > > > > >> > >> > > > > users. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > Cons: > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > 1. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > In the future we may have to > > > > > reimplement > > > > > > >> at the > > > > > > >> > >> nnvm/tvm > > > > > > >> > >> > layer, > > > > > > >> > >> > > > in case > > > > > > >> > >> > > > > MXNet moves to the nnvm/tvm > > > > > > >> backend(assuming it > > > > > > >> > will > > > > > > >> > >> move). > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > 1. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > Import/export at nnvm/tvm > layer > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > Pros: > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > 1. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > Less engineering work in case > > > mxnet > > > > > > moves > > > > > > >> to > > > > > > >> > nnvm/tvm > > > > > > >> > >> > > > > 2. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > nnvm/tvm would become a hub to > > > > convert > > > > > > to > > > > > > >> > different > > > > > > >> > >> formats. > > > > > > >> > >> > > > > 3. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > nnvm operators are more in > > parity > > > > with > > > > > > >> mxnet’s > > > > > > >> > gluon > > > > > > >> > >> APIs this > > > > > > >> > >> > > > could be > > > > > > >> > >> > > > > useful in case Gluon becomes > the > > > > only > > > > > > >> standard > > > > > > >> > that > > > > > > >> > >> MXNet will > > > > > > >> > >> > > > support. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > Cons: > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > 1. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > Nascent project with few > > > > contributors > > > > > > >> > >> > > > > 2. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > Does not support all operators > > > that > > > > > > exist > > > > > > >> in > > > > > > >> > MXNet > > > > > > >> > >> Symbolic > > > > > > >> > >> > API > > > > > > >> > >> > > > > 3. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > No CI Pipeline > > > > > > >> > >> > > > > 4. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > Current Apache MXNet project > > does > > > > not > > > > > > use > > > > > > >> > nnvm/tvm > > > > > > >> > >> backend > > > > > > >> > >> > > > > 5. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > mxnet->nnvm/tvm backend needs > > more > > > > > > >> testing and > > > > > > >> > user > > > > > > >> > >> feedback. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > Any suggestions on both of these > > > > > > approaches? > > > > > > >> From > > > > > > >> > user's > > > > > > >> > >> > > > perspective, this > > > > > > >> > >> > > > > will be an implementation detail > > that > > > > is > > > > > > not > > > > > > >> > exposed. > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > Thanks, > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > Roshani > > > > > > >> > >> > > > > > > > > > > >> > >> > > > > > > > > > >> > >> > > > > > > > > > >> > >> > > > > > > > > > >> > >> > > > > > > > > > >> > >> > > > > > > > > >> > >> > > > > > > > >> > >> > > > > > > >> > >> > > > > > > >> > >> > > > > > > >> > >> > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > >
