Sorry, the namespace should have been 'org.apache.mxnet' with the artifact as 'mxnet-incubating'.
On Sun, Mar 11, 2018 at 4:44 PM, kellen sunderland < kellen.sunderl...@gmail.com> wrote: > YiZhi, In general I agree that your points and examples are the ideal > case, but in the MXNet situation there are some trade-offs we have to > make. Let me try to specifically answer your points: > > "Do you mean we have different version for 'ml.dmlc' namespace and > 'org.apache' namespace?" > No I am not trying to saying that. I believe Marco, Naveen and I are all > proposing we use a single org.apache.incubating.mxnet namespace moving > forward, which would require a major version change to our product API > under our current versioning scheme. Marco and I are proposing we apply > this MV change _only_ to the scala package's API. > > "How to tell which Scala API version works with which MXNet core version? > By document?" > Yes users will be able to tell via the website, release docs, maven > package information, pom file, etc. > > "How many users will read the whole document and carefully pair the > version id before they run into a strange error and give up?" > They won't get a strange error, assuming we're talking about Scala users > who are upgrading from a package with the same namespace they will rely on > the package manager to give them an update which should be painless. > > Secondly software developers understand that packages, not products, have > versions. They know that these versions are used to communicate when APIs > are broken. There's examples of Apache packages doing this for packages > that include multiple interfaces, for example first-party modules packaged > with the HTTP server, or log4j's language bindings (arguably quite similar > to what Naveen is doing). > > While we can debate the right way to version packages, I think there's a > clear community decision here to get Naveen unblocked: > > (1) We continue semantically versioning across all APIs, meaning that this > change would get released with MXNet 2.*. > (2) You version package interfaces semantically and have a compatible > version mapping. > (3) Status quo, we continue to release a Scala package as-is, breaking > apache guidelines for artifact generation. > (4) We rely on the namespace change itself to communicate a change in the > interface. We don't consider this a major change. > > My (non-binding) preference would be for option 2. > > -Kellen > > On Sun, Mar 11, 2018 at 12:44 PM, Marco de Abreu < > marco.g.ab...@googlemail.com> wrote: > >> Changing namespaces is one example of a required major version change, but >> there are more reasons like general refactoring or some deprecated APIs >> just being hard to maintain. Things like these happen quite frequently and >> it's a problem every software project has to face and find a solution for. >> >> Regarding ' How to tell which Scala API version works with which MXNet >> core >> version?': We could just bundle MXNet with the released API package as we >> do right now, but we would give each interface it's own version and >> publish >> them on their distribution platforms accordingly. Just an example: >> >Scala-Package -> MXNet-Version >> >> 1.0 -> 1.0 >> >> 1.1 -> 1.1 >> >> 2.0 -> 1.2 >> >> 2.1 -> 1.3 >> >> 3.0 -> 2.0 >> >> > R-Package -> MXNet-Version >> >> 1.0 -> 1.0 >> >> 2.0 -> 1.1 >> >> 2.1 -> 1.2 >> >> 2.2 -> 1.3 >> >> 3.0 -> 2.0 >> >> This is always an N-to-1 mapping, while N being the versions of our APIs >> and 1 the MXNet Core version. From MXNets versioning perspective, this >> would then looking the following: >> > MXNet-Version -> APIs >> >> 1.0 -> Scala_1.0; R_1.0 >> >> 1.1 -> Scala_1.1; R_2.0 >> >> 1.2 -> Scala_2.0; R_2.1 >> >> 1.3 -> Scala_2.1; R_2.2 >> >> 2.0 -> Scala_3.0; R_3.0 >> >> This would give us the liberty to develop MXNet without restricting us too >> much - of course, major version increments will still have to be >> considered >> carefully. I don't think that this would harm transparency too much and >> there's no need to write big documentation. >> >> -Marco >> >> >> On Sun, Mar 11, 2018 at 12:16 PM, YiZhi Liu <liuyi...@apache.org> wrote: >> >> > I have no idea how separating Scala API version can solve the >> > 'compatibility' problem. Do you mean we have different version for >> > 'ml.dmlc' namespace and 'org.apache' namespace? Do these two versions >> > have same behavior? How to tell which Scala API version works with >> > which MXNet core version? By document? How many users will read the >> > whole document and carefully pair the version id before they run into >> > a strange error and give up? >> > >> > Moreover, changing namespace is an issue that is really rare and >> > hardly happens. For other 'compatibility' problem, for example, the >> > class/function definitions, should handle the compatibility itself. >> > You'll never expect a project to have a different version for changing >> > 'calculate(int)' to 'calculate(float)', it should just add a new >> > function 'calculate(float)'. >> > >> > Regarding 'In this case the Scala interface is clearly a separate >> > entity from the C API.'. Everything can be seen as a separate entity, >> > the mxnet engine, the graph description, operators, python API, gluon >> > API, etc. We should think carefully what we want to provide, and what >> > our users need. >> > >> > As an example, Apache Spark, still has SparkR (R API), PySpark (Python >> > API), MLLib, GraphX ... as part of its release, and have the same >> > version as Spark core as well as its Scala/Java API. >> > >> > 2018-03-10 23:58 GMT-08:00 kellen sunderland < >> kellen.sunderl...@gmail.com >> > >: >> > > +1 (non-binding) to what Marco is describing. +1 (non-binding) to >> > getting the Scala bindings with the namespace change into Maven. >> > > >> > > The general best practice for SemVer, which is used by most projects >> > that employ SemVer, is to apply SemVer to the public APIs of packages >> that >> > ship with your project. If you have several independent APIs this could >> > mean that they are versioned separately from each other, and from the >> > overall project versioning mechanism. >> > > >> > > For example, the .NET Core library ships with a number of binaries, >> each >> > with their own SemVerioned APIs. They also have a high-level, easy to >> > understand version for the package as a whole: >> > https://docs.microsoft.com/en-us/dotnet/core/versions/. >> > > >> > > Nodesource has a good description of this: >> http://nodesource.com/blog/ >> > semver-a-primer/ >> > > “Semver is a scheme for interface versioning for the benefit of >> > interface consumers, thus if a tool has multiple interfaces, e.g. an API >> > and a CLI, these interfaces may evolve independent versioning.” >> > > >> > > SemVer at its core is a communication mechanism to inform developers >> of >> > incompatibilities. In this case the Scala interface is clearly a >> separate >> > entity from the C API. I.e. changing the Scala namespace isn’t going to >> > break C API users. It does not communicate anything useful to these >> users >> > if we up their major version in response to a Scala change, it simply >> > breaks compatibility. If we group all interfaces together, and >> increment >> > whenever any of them has a breaking change we’ll soon be at MXNet >> version >> > 587. We’ll be forcing our users to check compatibility and update their >> > dependency tracking constantly. The end result is that our users will >> stop >> > pulling in new versions of the library. >> > > >> > > What I would propose is that (1) we have a high-level SemVer system >> that >> > tracks our C_API. This is the ‘MXNet’ version that we generally refer >> to >> > and emphasize for our public releases. For each API we have an >> independent >> > versioning system that if we can, we fix to the MXNet version. When it >> > makes sense we version these APIs independently. So for example we >> could >> > have a MXNet 1.2 release that ships with a 2.0 Scala API / R API. >> > > >> > > In terms of Apache process I think shipping artifacts with a >> non-Apache >> > namespace is a bigger issue than whatever versioning conventions we >> decide >> > to use. >> > > >> > > -Kellen >> > > >> > > From: Carin Meier >> > > Sent: Saturday, March 10, 2018 1:41 PM >> > > To: dev@mxnet.incubator.apache.org >> > > Cc: d...@mxnet.apache.org >> > > Subject: Re: Publishing Scala Package/namespace change >> > > >> > > +1 as well. I'm actively developing a Clojure package for MXNet that >> uses >> > > the jars from the Scala package. >> > > >> > > - Carin >> > > >> > > On Fri, Mar 9, 2018 at 4:44 PM, YiZhi Liu <eazhi....@gmail.com> >> wrote: >> > > >> > >> +1 for changing the namespace asap. for the maven deploy, we can have >> > >> it build along with pip deployment. >> > >> >> > >> >> > >> 2018-03-09 10:15 GMT-08:00 Naveen Swamy <mnnav...@gmail.com>: >> > >> > Hi Guys, >> > >> > >> > >> > I am working on MXNet Scala Inference APIs >> > >> > <https://issues.apache.org/jira/browse/MXNET-50> along with >> another >> > >> > contributor Roshani. A while back I noticed that we haven't been >> > >> publishing >> > >> > the scala package to Maven for a while now(last one being v0.11.1a >> > under >> > >> > the dmlc namespace). >> > >> > Currently users have to build the package manually and then use it, >> > this >> > >> > hinders adoption and also is painful to build everything from >> source. >> > >> > >> > >> > I also see that we haven't changed the namespace to org.apache and >> > >> instead >> > >> > are still ml.dmlc namespace. >> > >> > >> > >> > I wanted to seek your opinion about changing the MXNet-Scala >> package >> > >> > namespace to org.apache for the Scala package and publish to Maven >> in >> > the >> > >> > upcoming release. I understand that this probably breaks the Semver >> > >> > semantics that is agreed upon, However I would like to point out >> that >> > the >> > >> > Scala package has never been published to maven as 1.0 under >> > org.apache. >> > >> > >> > >> > Open to suggestions. >> > >> > >> > >> > Thanks, Naveen >> > >> >> > >> >> > >> >> > >> -- >> > >> Yizhi Liu >> > >> DMLC member >> > >> Amazon Web Services >> > >> Vancouver, Canada >> > >> >> > > >> > >> > >> > >> > -- >> > Yizhi Liu >> > DMLC member >> > Amazon Web Services >> > Vancouver, Canada >> > >> > >
