Great!!!!!! Yes, please share! On Fri, Mar 9, 2018 at 7:29 PM Can Balioglu <[email protected]> wrote:
> We found a rather "hacky" way to dynamically inject our operators and data > iterators during runtime. I can share the details if you are interested. We > also plan to send out a design proposal for a proper extensibility > framework similar to TensorFlow's plugin API. > > -Can > > On Wed, Mar 7, 2018, at 00:28, Eric Xie wrote: > > One potential problem is with libstdc++. Specifically with vectors. Our > > operator interface uses vectors, which breaks when core lib and > > extension are compiled with different compiler version > > > > On 2018/03/06 22:45:16, Chris Olivier <[email protected]> wrote: > > > The static init of your module should register your operator just as it > > > does for the operators in mxnet (NNVM_REGISTER_OP). While I haven't > done > > > it personally, I see no reason why it wouldn't work like any other > operator > > > at that point. > > > > > > On Tue, Mar 6, 2018 at 1:28 PM, Barber, Christopher < > > > [email protected]> wrote: > > > > > > > There are two separate issues: how to compile against the MXNet > source and > > > > how to dynamically load external dynamic libraries. While it would > be nice > > > > to have all necessary headers in the same place, it doesn't really > matter > > > > that much if people building extensions have to have access to the > entire > > > > MXNet source tree. The real issue is whether you support the ability > to > > > > dynamically load such extensions. > > > > > > > > - Christopher > > > > > > > > On 3/6/18, 4:12 PM, "Chris Olivier" <[email protected]> > wrote: > > > > > > > > This was discussed in the past and so far, it seems possible for > Unix > > > > builds, although it’s going to be messy since the compile would > > > > generally > > > > need access to all a large portion of the headers in the source > tree, > > > > since > > > > the “things needed to create your own op†aren’t > necessarily all > > > > contained > > > > in the include/mxnet directory. > > > > > > > > On Tue, Mar 6, 2018 at 11:40 AM kellen sunderland < > > > > [email protected]> wrote: > > > > > > > > > Could we actually just define a mechanism so the libs could > register > > > > their > > > > > ops at runtime? No linking required? > > > > > > > > > > On Tue, Mar 6, 2018, 8:36 PM Pedro Larroy < > > > > [email protected]> > > > > > wrote: > > > > > > > > > > > This is a good point. What additional blockers would there > be for > > > > linking > > > > > > against a user provided library with custom operators? > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Mar 6, 2018 at 5:16 PM, Barber, Christopher < > > > > > > [email protected]> wrote: > > > > > > > > > > > > > To avoid this kind of problem, you really need to support > > > > features that > > > > > > > allow MXNet to be extended without having to resort to > forking. > > > > There > > > > > is > > > > > > > currently no way to add C++ custom operators without > forking, > > > > and no > > > > > way > > > > > > to > > > > > > > share such operators across projects. This creates a > perverse > > > > incentive > > > > > > to > > > > > > > try to get changes that may not belong into the main > product. > > > > > > > > > > > > > > On 3/5/18, 6:26 PM, "Marco de Abreu" < > > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > we recently had a few cases in which it has been > attempted > > > > to add > > > > > new > > > > > > > functionality to old branches because a customer of > somebodys > > > > > > $DAY_JOB > > > > > > > requested it and was unable to switch to the latest > release > > > > or that > > > > > > > certain > > > > > > > feature did not make it into the release. This lead to > quite > > > > a lot > > > > > of > > > > > > > discussions and there was no clear standing within the > > > > community. > > > > > > > > > > > > > > Just to cite semantic versioning: > > > > > > > > > > > > > > 1. MAJOR version when you make incompatible API > changes, > > > > > > > 2. MINOR version when you add functionality in a > > > > > > > backwards-compatible > > > > > > > manner, and > > > > > > > 3. PATCH version when you make backwards-compatible > bug > > > > fixes. > > > > > > > > > > > > > > > > > > > > > We as a community agreed on following this system and I > > > > think we > > > > > > should > > > > > > > either stick to it or drop it entirely - exceptions to > > > > SemVer are > > > > > > > usually > > > > > > > discouraged. While I see that adding functionality > might be > > > > a minor > > > > > > > thing, > > > > > > > I don't think that we should educate our users into > > > > expecting us to > > > > > > > backport new features. The development happening on the > > > > Apache > > > > > MXNet > > > > > > > repository should not be influenced by something like > this; > > > > > > especially > > > > > > > considering that whoever supports that customer on > their > > > > $DAY_JOB > > > > > can > > > > > > > assist them at creating a fork and cherrypicking that > > > > feature. But > > > > > I > > > > > > > don't > > > > > > > see much reason why we're running against best > pracitices. > > > > One > > > > > > > important > > > > > > > thing to note here is that we're not maintaining CI on > old > > > > branches > > > > > > and > > > > > > > neither are we making patch releases - so what do these > > > > users do? > > > > > > > Compile > > > > > > > off a stale development branch with unvalidated > changes? > > > > > > > > > > > > > > In my opinion this whole topic is just causing trouble > and > > > > > > > fragementation > > > > > > > on our end. If a features doesn't make it into the > release > > > > (for > > > > > > > whatever > > > > > > > reason), so be it. But I think we should discuss this > topic > > > > and > > > > > > > emphasize a > > > > > > > no-exceptions-rule to SemVer. > > > > > > > > > > > > > > Best regards, > > > > > > > Marco > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
