Re: Publishing Scala Package/namespace change

[YiZhi Liu]

t; >> On Sun, Mar 11, 2018 at 12:16 PM, YiZhi Liu <
> liuyi...@apache.org
> > >
> > >> > > wrote:
> > >> > > >>
> > >> > > >> > I have no idea how separating Scala API version can solve the
> > >> > > >> > 'compatibility' problem. Do you mean we have different
> version
> > >> for
> > >> > > >> > 'ml.dmlc' namespace and 'org.apache' namespace? Do these two
> > >> > versions
> > >> > > >> > have same behavior? How to tell which Scala API version works
> > >> with
> > >> > > >> > which MXNet core version? By document? How many users will
> read
> > >> the
> > >> > > >> > whole document and carefully pair the version id before they
> > run
> > >> > into
> > >> > > >> > a strange error and give up?
> > >> > > >> >
> > >> > > >> > Moreover, changing namespace is an issue that is really rare
> > and
> > >> > > >> > hardly happens. For other 'compatibility' problem, for
> example,
> > >> the
> > >> > > >> > class/function definitions, should handle the compatibility
> > >> itself.
> > >> > > >> > You'll never expect a project to have a different version for
> > >> > changing
> > >> > > >> > 'calculate(int)' to 'calculate(float)', it should just add a
> > new
> > >> > > >> > function 'calculate(float)'.
> > >> > > >> >
> > >> > > >> > Regarding 'In this case the Scala interface is clearly a
> > separate
> > >> > > >> > entity from the C API.'. Everything can be seen as a separate
> > >> > entity,
> > >> > > >> > the mxnet engine, the graph description, operators, python
> API,
> > >> > gluon
> > >> > > >> > API, etc. We should think carefully what we want to provide,
> > and
> > >> > what
> > >> > > >> > our users need.
> > >> > > >> >
> > >> > > >> > As an example, Apache Spark, still has SparkR (R API),
> PySpark
> > >> > (Python
> > >> > > >> > API), MLLib, GraphX ... as part of its release, and have the
> > same
> > >> > > >> > version as Spark core as well as its Scala/Java API.
> > >> > > >> >
> > >> > > >> > 2018-03-10 23:58 GMT-08:00 kellen sunderland <
> > >> > > >> kellen.sunderl...@gmail.com
> > >> > > >> > >:
> > >> > > >> > > +1 (non-binding) to what Marco is describing.  +1
> > >> (non-binding) to
> > >> > > >> > getting the Scala bindings with the namespace change into
> > Maven.
> > >> > > >> > >
> > >> > > >> > > The general best practice for SemVer, which is used by most
> > >> > projects
> > >> > > >> > that employ SemVer, is to apply SemVer to the public APIs of
> > >> > packages
> > >> > > >> that
> > >> > > >> > ship with your project.  If you have several independent APIs
> > >> this
> > >> > > could
> > >> > > >> > mean that they are versioned separately from each other, and
> > from
> > >> > the
> > >> > > >> > overall project versioning mechanism.
> > >> > > >> > >
> > >> > > >> > > For example, the .NET Core library ships with a number of
> > >> > binaries,
> > >> > > >> each
> > >> > > >> > with their own SemVerioned APIs.  They also have a
> high-level,
> > >> easy
> > >> > to
> > >> > > >> > understand version for the package as a whole:
> > >> > > >> > https://docs.microsoft.com/en-us/dotnet/core/versions/.
> > >> > > >> > >
> > >> > > >> > > Nodesource has a good description of this:
> > >> > > >> http://nodesource.com/blog/
> > >> > > >> > semver-a-primer/
> > >> > > >> > > “Semver is a scheme for interface versioning for the
> benefit
> > of
> > >> > > >> > interface consumers, thus if a tool has multiple interfaces,
> > >> e.g. an
> > >> > > API
> > >> > > >> > and a CLI, these interfaces may evolve independent
> versioning.”
> > >> > > >> > >
> > >> > > >> > > SemVer at its core is a communication mechanism to inform
> > >> > developers
> > >> > > >> of
> > >> > > >> > incompatibilities. In this case the Scala interface is
> clearly
> > a
> > >> > > >> separate
> > >> > > >> > entity from the C API.  I.e. changing the Scala namespace
> isn’t
> > >> > going
> > >> > > to
> > >> > > >> > break C API users.  It does not communicate anything useful
> to
> > >> these
> > >> > > >> users
> > >> > > >> > if we up their major version in response to a Scala change,
> it
> > >> > simply
> > >> > > >> > breaks compatibility.  If we group all interfaces together,
> and
> > >> > > >> increment
> > >> > > >> > whenever any of them has a breaking change we’ll soon be at
> > MXNet
> > >> > > >> version
> > >> > > >> > 587.  We’ll be forcing our users to check compatibility and
> > >> update
> > >> > > their
> > >> > > >> > dependency tracking constantly.  The end result is that our
> > users
> > >> > will
> > >> > > >> stop
> > >> > > >> > pulling in new versions of the library.
> > >> > > >> > >
> > >> > > >> > > What I would propose is that (1) we have a high-level
> SemVer
> > >> > system
> > >> > > >> that
> > >> > > >> > tracks our C_API.  This is the ‘MXNet’ version that we
> > generally
> > >> > refer
> > >> > > >> to
> > >> > > >> > and emphasize for our public releases.  For each API we have
> an
> > >> > > >> independent
> > >> > > >> > versioning system that if we can, we fix to the MXNet
> version.
> > >> When
> > >> > > it
> > >> > > >> > makes sense we version these APIs independently.  So for
> > example
> > >> we
> > >> > > >> could
> > >> > > >> > have a MXNet 1.2 release that ships with a 2.0 Scala API / R
> > API.
> > >> > > >> > >
> > >> > > >> > > In terms of Apache process I think shipping artifacts with
> a
> > >> > > >> non-Apache
> > >> > > >> > namespace is a bigger issue than whatever versioning
> > conventions
> > >> we
> > >> > > >> decide
> > >> > > >> > to use.
> > >> > > >> > >
> > >> > > >> > > -Kellen
> > >> > > >> > >
> > >> > > >> > > From: Carin Meier
> > >> > > >> > > Sent: Saturday, March 10, 2018 1:41 PM
> > >> > > >> > > To: dev@mxnet.incubator.apache.org
> > >> > > >> > > Cc: d...@mxnet.apache.org
> > >> > > >> > > Subject: Re: Publishing Scala Package/namespace change
> > >> > > >> > >
> > >> > > >> > > +1 as well. I'm actively developing a Clojure package for
> > MXNet
> > >> > that
> > >> > > >> uses
> > >> > > >> > > the jars from the Scala package.
> > >> > > >> > >
> > >> > > >> > > - Carin
> > >> > > >> > >
> > >> > > >> > > On Fri, Mar 9, 2018 at 4:44 PM, YiZhi Liu <
> > eazhi@gmail.com
> > >> >
> > >> > > >> wrote:
> > >> > > >> > >
> > >> > > >> > >> +1 for changing the namespace asap. for the maven deploy,
> we
> > >> can
> > >> > > have
> > >> > > >> > >> it build along with pip deployment.
> > >> > > >> > >>
> > >> > > >> > >>
> > >> > > >> > >> 2018-03-09 10:15 GMT-08:00 Naveen Swamy <
> mnnav...@gmail.com
> > >:
> > >> > > >> > >> > Hi Guys,
> > >> > > >> > >> >
> > >> > > >> > >> > I am working on MXNet Scala Inference APIs
> > >> > > >> > >> > <https://issues.apache.org/jira/browse/MXNET-50> along
> > with
> > >> > > >> another
> > >> > > >> > >> > contributor Roshani. A while back I noticed that we
> > haven't
> > >> > been
> > >> > > >> > >> publishing
> > >> > > >> > >> > the scala package to Maven for a while now(last one
> being
> > >> > > v0.11.1a
> > >> > > >> > under
> > >> > > >> > >> > the dmlc namespace).
> > >> > > >> > >> > Currently users have to build the package manually and
> > then
> > >> use
> > >> > > it,
> > >> > > >> > this
> > >> > > >> > >> > hinders adoption and also is painful to build everything
> > >> from
> > >> > > >> source.
> > >> > > >> > >> >
> > >> > > >> > >> > I also see that we haven't changed the namespace to
> > >> org.apache
> > >> > > and
> > >> > > >> > >> instead
> > >> > > >> > >> > are still ml.dmlc namespace.
> > >> > > >> > >> >
> > >> > > >> > >> > I wanted to seek your opinion about changing the
> > MXNet-Scala
> > >> > > >> package
> > >> > > >> > >> > namespace to org.apache for the Scala package and
> publish
> > to
> > >> > > Maven
> > >> > > >> in
> > >> > > >> > the
> > >> > > >> > >> > upcoming release. I understand that this probably breaks
> > the
> > >> > > Semver
> > >> > > >> > >> > semantics that is agreed upon, However I would like to
> > point
> > >> > out
> > >> > > >> that
> > >> > > >> > the
> > >> > > >> > >> > Scala package has never been published to maven as 1.0
> > under
> > >> > > >> > org.apache.
> > >> > > >> > >> >
> > >> > > >> > >> > Open to suggestions.
> > >> > > >> > >> >
> > >> > > >> > >> > Thanks, Naveen
> > >> > > >> > >>
> > >> > > >> > >>
> > >> > > >> > >>
> > >> > > >> > >> --
> > >> > > >> > >> Yizhi Liu
> > >> > > >> > >> DMLC member
> > >> > > >> > >> Amazon Web Services
> > >> > > >> > >> Vancouver, Canada
> > >> > > >> > >>
> > >> > > >> > >
> > >> > > >> >
> > >> > > >> >
> > >> > > >> >
> > >> > > >> > --
> > >> > > >> > Yizhi Liu
> > >> > > >> > DMLC member
> > >> > > >> > Amazon Web Services
> > >> > > >> > Vancouver, Canada
> > >> > > >> >
> > >> > > >>
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Yizhi Liu
> > >> DMLC member
> > >> Amazon Web Services
> > >> Vancouver, Canada
> > >>
> > >
> > >
> >
>



-- 
Yizhi Liu
DMLC member
Amazon Web Services
Vancouver, Canada

Re: Publishing Scala Package/namespace change

[YiZhi Liu]

pace is an issue that is really rare
> and
> >> > > >> > hardly happens. For other 'compatibility' problem, for example,
> >> the
> >> > > >> > class/function definitions, should handle the compatibility
> >> itself.
> >> > > >> > You'll never expect a project to have a different version for
> >> > changing
> >> > > >> > 'calculate(int)' to 'calculate(float)', it should just add a
> new
> >> > > >> > function 'calculate(float)'.
> >> > > >> >
> >> > > >> > Regarding 'In this case the Scala interface is clearly a
> separate
> >> > > >> > entity from the C API.'. Everything can be seen as a separate
> >> > entity,
> >> > > >> > the mxnet engine, the graph description, operators, python API,
> >> > gluon
> >> > > >> > API, etc. We should think carefully what we want to provide,
> and
> >> > what
> >> > > >> > our users need.
> >> > > >> >
> >> > > >> > As an example, Apache Spark, still has SparkR (R API), PySpark
> >> > (Python
> >> > > >> > API), MLLib, GraphX ... as part of its release, and have the
> same
> >> > > >> > version as Spark core as well as its Scala/Java API.
> >> > > >> >
> >> > > >> > 2018-03-10 23:58 GMT-08:00 kellen sunderland <
> >> > > >> kellen.sunderl...@gmail.com
> >> > > >> > >:
> >> > > >> > > +1 (non-binding) to what Marco is describing.  +1
> >> (non-binding) to
> >> > > >> > getting the Scala bindings with the namespace change into
> Maven.
> >> > > >> > >
> >> > > >> > > The general best practice for SemVer, which is used by most
> >> > projects
> >> > > >> > that employ SemVer, is to apply SemVer to the public APIs of
> >> > packages
> >> > > >> that
> >> > > >> > ship with your project.  If you have several independent APIs
> >> this
> >> > > could
> >> > > >> > mean that they are versioned separately from each other, and
> from
> >> > the
> >> > > >> > overall project versioning mechanism.
> >> > > >> > >
> >> > > >> > > For example, the .NET Core library ships with a number of
> >> > binaries,
> >> > > >> each
> >> > > >> > with their own SemVerioned APIs.  They also have a high-level,
> >> easy
> >> > to
> >> > > >> > understand version for the package as a whole:
> >> > > >> > https://docs.microsoft.com/en-us/dotnet/core/versions/.
> >> > > >> > >
> >> > > >> > > Nodesource has a good description of this:
> >> > > >> http://nodesource.com/blog/
> >> > > >> > semver-a-primer/
> >> > > >> > > “Semver is a scheme for interface versioning for the benefit
> of
> >> > > >> > interface consumers, thus if a tool has multiple interfaces,
> >> e.g. an
> >> > > API
> >> > > >> > and a CLI, these interfaces may evolve independent versioning.”
> >> > > >> > >
> >> > > >> > > SemVer at its core is a communication mechanism to inform
> >> > developers
> >> > > >> of
> >> > > >> > incompatibilities. In this case the Scala interface is clearly
> a
> >> > > >> separate
> >> > > >> > entity from the C API.  I.e. changing the Scala namespace isn’t
> >> > going
> >> > > to
> >> > > >> > break C API users.  It does not communicate anything useful to
> >> these
> >> > > >> users
> >> > > >> > if we up their major version in response to a Scala change, it
> >> > simply
> >> > > >> > breaks compatibility.  If we group all interfaces together, and
> >> > > >> increment
> >> > > >> > whenever any of them has a breaking change we’ll soon be at
> MXNet
> >> > > >> version
> >> > > >> > 587.  We’ll be forcing our users to check compatibility and
> >> update
> >> > > their
> >> > > >> > dependency tracking constantly.  The end result is that our
> users
> >> > will
> >> > > >> stop
> >> > > >> > pulling in new versions of the library.
> >> > > >> > >
> >> > > >> > > What I would propose is that (1) we have a high-level SemVer
> >> > system
> >> > > >> that
> >> > > >> > tracks our C_API.  This is the ‘MXNet’ version that we
> generally
> >> > refer
> >> > > >> to
> >> > > >> > and emphasize for our public releases.  For each API we have an
> >> > > >> independent
> >> > > >> > versioning system that if we can, we fix to the MXNet version.
> >> When
> >> > > it
> >> > > >> > makes sense we version these APIs independently.  So for
> example
> >> we
> >> > > >> could
> >> > > >> > have a MXNet 1.2 release that ships with a 2.0 Scala API / R
> API.
> >> > > >> > >
> >> > > >> > > In terms of Apache process I think shipping artifacts with a
> >> > > >> non-Apache
> >> > > >> > namespace is a bigger issue than whatever versioning
> conventions
> >> we
> >> > > >> decide
> >> > > >> > to use.
> >> > > >> > >
> >> > > >> > > -Kellen
> >> > > >> > >
> >> > > >> > > From: Carin Meier
> >> > > >> > > Sent: Saturday, March 10, 2018 1:41 PM
> >> > > >> > > To: dev@mxnet.incubator.apache.org
> >> > > >> > > Cc: d...@mxnet.apache.org
> >> > > >> > > Subject: Re: Publishing Scala Package/namespace change
> >> > > >> > >
> >> > > >> > > +1 as well. I'm actively developing a Clojure package for
> MXNet
> >> > that
> >> > > >> uses
> >> > > >> > > the jars from the Scala package.
> >> > > >> > >
> >> > > >> > > - Carin
> >> > > >> > >
> >> > > >> > > On Fri, Mar 9, 2018 at 4:44 PM, YiZhi Liu <
> eazhi@gmail.com
> >> >
> >> > > >> wrote:
> >> > > >> > >
> >> > > >> > >> +1 for changing the namespace asap. for the maven deploy, we
> >> can
> >> > > have
> >> > > >> > >> it build along with pip deployment.
> >> > > >> > >>
> >> > > >> > >>
> >> > > >> > >> 2018-03-09 10:15 GMT-08:00 Naveen Swamy <mnnav...@gmail.com
> >:
> >> > > >> > >> > Hi Guys,
> >> > > >> > >> >
> >> > > >> > >> > I am working on MXNet Scala Inference APIs
> >> > > >> > >> > <https://issues.apache.org/jira/browse/MXNET-50> along
> with
> >> > > >> another
> >> > > >> > >> > contributor Roshani. A while back I noticed that we
> haven't
> >> > been
> >> > > >> > >> publishing
> >> > > >> > >> > the scala package to Maven for a while now(last one being
> >> > > v0.11.1a
> >> > > >> > under
> >> > > >> > >> > the dmlc namespace).
> >> > > >> > >> > Currently users have to build the package manually and
> then
> >> use
> >> > > it,
> >> > > >> > this
> >> > > >> > >> > hinders adoption and also is painful to build everything
> >> from
> >> > > >> source.
> >> > > >> > >> >
> >> > > >> > >> > I also see that we haven't changed the namespace to
> >> org.apache
> >> > > and
> >> > > >> > >> instead
> >> > > >> > >> > are still ml.dmlc namespace.
> >> > > >> > >> >
> >> > > >> > >> > I wanted to seek your opinion about changing the
> MXNet-Scala
> >> > > >> package
> >> > > >> > >> > namespace to org.apache for the Scala package and publish
> to
> >> > > Maven
> >> > > >> in
> >> > > >> > the
> >> > > >> > >> > upcoming release. I understand that this probably breaks
> the
> >> > > Semver
> >> > > >> > >> > semantics that is agreed upon, However I would like to
> point
> >> > out
> >> > > >> that
> >> > > >> > the
> >> > > >> > >> > Scala package has never been published to maven as 1.0
> under
> >> > > >> > org.apache.
> >> > > >> > >> >
> >> > > >> > >> > Open to suggestions.
> >> > > >> > >> >
> >> > > >> > >> > Thanks, Naveen
> >> > > >> > >>
> >> > > >> > >>
> >> > > >> > >>
> >> > > >> > >> --
> >> > > >> > >> Yizhi Liu
> >> > > >> > >> DMLC member
> >> > > >> > >> Amazon Web Services
> >> > > >> > >> Vancouver, Canada
> >> > > >> > >>
> >> > > >> > >
> >> > > >> >
> >> > > >> >
> >> > > >> >
> >> > > >> > --
> >> > > >> > Yizhi Liu
> >> > > >> > DMLC member
> >> > > >> > Amazon Web Services
> >> > > >> > Vancouver, Canada
> >> > > >> >
> >> > > >>
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >> Yizhi Liu
> >> DMLC member
> >> Amazon Web Services
> >> Vancouver, Canada
> >>
> >
> >
>



-- 
Yizhi Liu
DMLC member
Amazon Web Services
Vancouver, Canada

Re: Publishing Scala Package/namespace change

[Chris Olivier]

gt; >> > > >> > entity from the C API.'. Everything can be seen as a separate
> >> > entity,
> >> > > >> > the mxnet engine, the graph description, operators, python API,
> >> > gluon
> >> > > >> > API, etc. We should think carefully what we want to provide,
> and
> >> > what
> >> > > >> > our users need.
> >> > > >> >
> >> > > >> > As an example, Apache Spark, still has SparkR (R API), PySpark
> >> > (Python
> >> > > >> > API), MLLib, GraphX ... as part of its release, and have the
> same
> >> > > >> > version as Spark core as well as its Scala/Java API.
> >> > > >> >
> >> > > >> > 2018-03-10 23:58 GMT-08:00 kellen sunderland <
> >> > > >> kellen.sunderl...@gmail.com
> >> > > >> > >:
> >> > > >> > > +1 (non-binding) to what Marco is describing.  +1
> >> (non-binding) to
> >> > > >> > getting the Scala bindings with the namespace change into
> Maven.
> >> > > >> > >
> >> > > >> > > The general best practice for SemVer, which is used by most
> >> > projects
> >> > > >> > that employ SemVer, is to apply SemVer to the public APIs of
> >> > packages
> >> > > >> that
> >> > > >> > ship with your project.  If you have several independent APIs
> >> this
> >> > > could
> >> > > >> > mean that they are versioned separately from each other, and
> from
> >> > the
> >> > > >> > overall project versioning mechanism.
> >> > > >> > >
> >> > > >> > > For example, the .NET Core library ships with a number of
> >> > binaries,
> >> > > >> each
> >> > > >> > with their own SemVerioned APIs.  They also have a high-level,
> >> easy
> >> > to
> >> > > >> > understand version for the package as a whole:
> >> > > >> > https://docs.microsoft.com/en-us/dotnet/core/versions/.
> >> > > >> > >
> >> > > >> > > Nodesource has a good description of this:
> >> > > >> http://nodesource.com/blog/
> >> > > >> > semver-a-primer/
> >> > > >> > > “Semver is a scheme for interface versioning for the benefit
> of
> >> > > >> > interface consumers, thus if a tool has multiple interfaces,
> >> e.g. an
> >> > > API
> >> > > >> > and a CLI, these interfaces may evolve independent versioning.”
> >> > > >> > >
> >> > > >> > > SemVer at its core is a communication mechanism to inform
> >> > developers
> >> > > >> of
> >> > > >> > incompatibilities. In this case the Scala interface is clearly
> a
> >> > > >> separate
> >> > > >> > entity from the C API.  I.e. changing the Scala namespace isn’t
> >> > going
> >> > > to
> >> > > >> > break C API users.  It does not communicate anything useful to
> >> these
> >> > > >> users
> >> > > >> > if we up their major version in response to a Scala change, it
> >> > simply
> >> > > >> > breaks compatibility.  If we group all interfaces together, and
> >> > > >> increment
> >> > > >> > whenever any of them has a breaking change we’ll soon be at
> MXNet
> >> > > >> version
> >> > > >> > 587.  We’ll be forcing our users to check compatibility and
> >> update
> >> > > their
> >> > > >> > dependency tracking constantly.  The end result is that our
> users
> >> > will
> >> > > >> stop
> >> > > >> > pulling in new versions of the library.
> >> > > >> > >
> >> > > >> > > What I would propose is that (1) we have a high-level SemVer
> >> > system
> >> > > >> that
> >> > > >> > tracks our C_API.  This is the ‘MXNet’ version that we
> generally
> >> > refer
> >> > > >> to
> >> > > >> > and emphasize for our public releases.  For each API we have an
> >> > > >> independent
> >> > > >> > versioning system that if we can, we fix to the MXNet version.
> >> When
> >> > > it
> >> > > >> > makes sense we version these APIs independently.  So for
> example
> >> we
> >> > > >> could
> >> > > >> > have a MXNet 1.2 release that ships with a 2.0 Scala API / R
> API.
> >> > > >> > >
> >> > > >> > > In terms of Apache process I think shipping artifacts with a
> >> > > >> non-Apache
> >> > > >> > namespace is a bigger issue than whatever versioning
> conventions
> >> we
> >> > > >> decide
> >> > > >> > to use.
> >> > > >> > >
> >> > > >> > > -Kellen
> >> > > >> > >
> >> > > >> > > From: Carin Meier
> >> > > >> > > Sent: Saturday, March 10, 2018 1:41 PM
> >> > > >> > > To: dev@mxnet.incubator.apache.org
> >> > > >> > > Cc: d...@mxnet.apache.org
> >> > > >> > > Subject: Re: Publishing Scala Package/namespace change
> >> > > >> > >
> >> > > >> > > +1 as well. I'm actively developing a Clojure package for
> MXNet
> >> > that
> >> > > >> uses
> >> > > >> > > the jars from the Scala package.
> >> > > >> > >
> >> > > >> > > - Carin
> >> > > >> > >
> >> > > >> > > On Fri, Mar 9, 2018 at 4:44 PM, YiZhi Liu <
> eazhi@gmail.com
> >> >
> >> > > >> wrote:
> >> > > >> > >
> >> > > >> > >> +1 for changing the namespace asap. for the maven deploy, we
> >> can
> >> > > have
> >> > > >> > >> it build along with pip deployment.
> >> > > >> > >>
> >> > > >> > >>
> >> > > >> > >> 2018-03-09 10:15 GMT-08:00 Naveen Swamy <mnnav...@gmail.com
> >:
> >> > > >> > >> > Hi Guys,
> >> > > >> > >> >
> >> > > >> > >> > I am working on MXNet Scala Inference APIs
> >> > > >> > >> > <https://issues.apache.org/jira/browse/MXNET-50> along
> with
> >> > > >> another
> >> > > >> > >> > contributor Roshani. A while back I noticed that we
> haven't
> >> > been
> >> > > >> > >> publishing
> >> > > >> > >> > the scala package to Maven for a while now(last one being
> >> > > v0.11.1a
> >> > > >> > under
> >> > > >> > >> > the dmlc namespace).
> >> > > >> > >> > Currently users have to build the package manually and
> then
> >> use
> >> > > it,
> >> > > >> > this
> >> > > >> > >> > hinders adoption and also is painful to build everything
> >> from
> >> > > >> source.
> >> > > >> > >> >
> >> > > >> > >> > I also see that we haven't changed the namespace to
> >> org.apache
> >> > > and
> >> > > >> > >> instead
> >> > > >> > >> > are still ml.dmlc namespace.
> >> > > >> > >> >
> >> > > >> > >> > I wanted to seek your opinion about changing the
> MXNet-Scala
> >> > > >> package
> >> > > >> > >> > namespace to org.apache for the Scala package and publish
> to
> >> > > Maven
> >> > > >> in
> >> > > >> > the
> >> > > >> > >> > upcoming release. I understand that this probably breaks
> the
> >> > > Semver
> >> > > >> > >> > semantics that is agreed upon, However I would like to
> point
> >> > out
> >> > > >> that
> >> > > >> > the
> >> > > >> > >> > Scala package has never been published to maven as 1.0
> under
> >> > > >> > org.apache.
> >> > > >> > >> >
> >> > > >> > >> > Open to suggestions.
> >> > > >> > >> >
> >> > > >> > >> > Thanks, Naveen
> >> > > >> > >>
> >> > > >> > >>
> >> > > >> > >>
> >> > > >> > >> --
> >> > > >> > >> Yizhi Liu
> >> > > >> > >> DMLC member
> >> > > >> > >> Amazon Web Services
> >> > > >> > >> Vancouver, Canada
> >> > > >> > >>
> >> > > >> > >
> >> > > >> >
> >> > > >> >
> >> > > >> >
> >> > > >> > --
> >> > > >> > Yizhi Liu
> >> > > >> > DMLC member
> >> > > >> > Amazon Web Services
> >> > > >> > Vancouver, Canada
> >> > > >> >
> >> > > >>
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >> Yizhi Liu
> >> DMLC member
> >> Amazon Web Services
> >> Vancouver, Canada
> >>
> >
> >
>

Re: Publishing Scala Package/namespace change

[kellen sunderland]

gt; 1.0 -> 1.0
>> > > >> >> 2.0 -> 1.1
>> > > >> >> 2.1 -> 1.2
>> > > >> >> 2.2 -> 1.3
>> > > >> >> 3.0 -> 2.0
>> > > >>
>> > > >> This is always an N-to-1 mapping, while N being the versions of our
>> > APIs
>> > > >> and 1 the MXNet Core version. From MXNets versioning perspective,
>> this
>> > > >> would then looking the following:
>> > > >> > MXNet-Version -> APIs
>> > > >> >> 1.0 -> Scala_1.0; R_1.0
>> > > >> >> 1.1 -> Scala_1.1; R_2.0
>> > > >> >> 1.2 -> Scala_2.0; R_2.1
>> > > >> >> 1.3 -> Scala_2.1; R_2.2
>> > > >> >> 2.0 -> Scala_3.0; R_3.0
>> > > >>
>> > > >> This would give us the liberty to develop MXNet without
>> restricting us
>> > > too
>> > > >> much - of course, major version increments will still have to be
>> > > >> considered
>> > > >> carefully. I don't think that this would harm transparency too much
>> > and
>> > > >> there's no need to write big documentation.
>> > > >>
>> > > >> -Marco
>> > > >>
>> > > >>
>> > > >> On Sun, Mar 11, 2018 at 12:16 PM, YiZhi Liu <liuyi...@apache.org>
>> > > wrote:
>> > > >>
>> > > >> > I have no idea how separating Scala API version can solve the
>> > > >> > 'compatibility' problem. Do you mean we have different version
>> for
>> > > >> > 'ml.dmlc' namespace and 'org.apache' namespace? Do these two
>> > versions
>> > > >> > have same behavior? How to tell which Scala API version works
>> with
>> > > >> > which MXNet core version? By document? How many users will read
>> the
>> > > >> > whole document and carefully pair the version id before they run
>> > into
>> > > >> > a strange error and give up?
>> > > >> >
>> > > >> > Moreover, changing namespace is an issue that is really rare and
>> > > >> > hardly happens. For other 'compatibility' problem, for example,
>> the
>> > > >> > class/function definitions, should handle the compatibility
>> itself.
>> > > >> > You'll never expect a project to have a different version for
>> > changing
>> > > >> > 'calculate(int)' to 'calculate(float)', it should just add a new
>> > > >> > function 'calculate(float)'.
>> > > >> >
>> > > >> > Regarding 'In this case the Scala interface is clearly a separate
>> > > >> > entity from the C API.'. Everything can be seen as a separate
>> > entity,
>> > > >> > the mxnet engine, the graph description, operators, python API,
>> > gluon
>> > > >> > API, etc. We should think carefully what we want to provide, and
>> > what
>> > > >> > our users need.
>> > > >> >
>> > > >> > As an example, Apache Spark, still has SparkR (R API), PySpark
>> > (Python
>> > > >> > API), MLLib, GraphX ... as part of its release, and have the same
>> > > >> > version as Spark core as well as its Scala/Java API.
>> > > >> >
>> > > >> > 2018-03-10 23:58 GMT-08:00 kellen sunderland <
>> > > >> kellen.sunderl...@gmail.com
>> > > >> > >:
>> > > >> > > +1 (non-binding) to what Marco is describing.  +1
>> (non-binding) to
>> > > >> > getting the Scala bindings with the namespace change into Maven.
>> > > >> > >
>> > > >> > > The general best practice for SemVer, which is used by most
>> > projects
>> > > >> > that employ SemVer, is to apply SemVer to the public APIs of
>> > packages
>> > > >> that
>> > > >> > ship with your project.  If you have several independent APIs
>> this
>> > > could
>> > > >> > mean that they are versioned separately from each other, and from
>> > the
>> > > >> > overall project versioning mechanism.
>> > > >> > >
>> > > >> > > F

Re: Publishing Scala Package/namespace change

[kellen sunderland]

ould harm transparency too much
> > and
> > > >> there's no need to write big documentation.
> > > >>
> > > >> -Marco
> > > >>
> > > >>
> > > >> On Sun, Mar 11, 2018 at 12:16 PM, YiZhi Liu <liuyi...@apache.org>
> > > wrote:
> > > >>
> > > >> > I have no idea how separating Scala API version can solve the
> > > >> > 'compatibility' problem. Do you mean we have different version for
> > > >> > 'ml.dmlc' namespace and 'org.apache' namespace? Do these two
> > versions
> > > >> > have same behavior? How to tell which Scala API version works with
> > > >> > which MXNet core version? By document? How many users will read
> the
> > > >> > whole document and carefully pair the version id before they run
> > into
> > > >> > a strange error and give up?
> > > >> >
> > > >> > Moreover, changing namespace is an issue that is really rare and
> > > >> > hardly happens. For other 'compatibility' problem, for example,
> the
> > > >> > class/function definitions, should handle the compatibility
> itself.
> > > >> > You'll never expect a project to have a different version for
> > changing
> > > >> > 'calculate(int)' to 'calculate(float)', it should just add a new
> > > >> > function 'calculate(float)'.
> > > >> >
> > > >> > Regarding 'In this case the Scala interface is clearly a separate
> > > >> > entity from the C API.'. Everything can be seen as a separate
> > entity,
> > > >> > the mxnet engine, the graph description, operators, python API,
> > gluon
> > > >> > API, etc. We should think carefully what we want to provide, and
> > what
> > > >> > our users need.
> > > >> >
> > > >> > As an example, Apache Spark, still has SparkR (R API), PySpark
> > (Python
> > > >> > API), MLLib, GraphX ... as part of its release, and have the same
> > > >> > version as Spark core as well as its Scala/Java API.
> > > >> >
> > > >> > 2018-03-10 23:58 GMT-08:00 kellen sunderland <
> > > >> kellen.sunderl...@gmail.com
> > > >> > >:
> > > >> > > +1 (non-binding) to what Marco is describing.  +1 (non-binding)
> to
> > > >> > getting the Scala bindings with the namespace change into Maven.
> > > >> > >
> > > >> > > The general best practice for SemVer, which is used by most
> > projects
> > > >> > that employ SemVer, is to apply SemVer to the public APIs of
> > packages
> > > >> that
> > > >> > ship with your project.  If you have several independent APIs this
> > > could
> > > >> > mean that they are versioned separately from each other, and from
> > the
> > > >> > overall project versioning mechanism.
> > > >> > >
> > > >> > > For example, the .NET Core library ships with a number of
> > binaries,
> > > >> each
> > > >> > with their own SemVerioned APIs.  They also have a high-level,
> easy
> > to
> > > >> > understand version for the package as a whole:
> > > >> > https://docs.microsoft.com/en-us/dotnet/core/versions/.
> > > >> > >
> > > >> > > Nodesource has a good description of this:
> > > >> http://nodesource.com/blog/
> > > >> > semver-a-primer/
> > > >> > > “Semver is a scheme for interface versioning for the benefit of
> > > >> > interface consumers, thus if a tool has multiple interfaces, e.g.
> an
> > > API
> > > >> > and a CLI, these interfaces may evolve independent versioning.”
> > > >> > >
> > > >> > > SemVer at its core is a communication mechanism to inform
> > developers
> > > >> of
> > > >> > incompatibilities. In this case the Scala interface is clearly a
> > > >> separate
> > > >> > entity from the C API.  I.e. changing the Scala namespace isn’t
> > going
> > > to
> > > >> > break C API users.  It does not communicate anything useful to
> these
> > > >> users
> > > >> > if we up their major version in response to a Scal

Re: Publishing Scala Package/namespace change

[YiZhi Liu]

 with the namespace change into Maven.
> > >> > >
> > >> > > The general best practice for SemVer, which is used by most
> projects
> > >> > that employ SemVer, is to apply SemVer to the public APIs of
> packages
> > >> that
> > >> > ship with your project.  If you have several independent APIs this
> > could
> > >> > mean that they are versioned separately from each other, and from
> the
> > >> > overall project versioning mechanism.
> > >> > >
> > >> > > For example, the .NET Core library ships with a number of
> binaries,
> > >> each
> > >> > with their own SemVerioned APIs.  They also have a high-level, easy
> to
> > >> > understand version for the package as a whole:
> > >> > https://docs.microsoft.com/en-us/dotnet/core/versions/.
> > >> > >
> > >> > > Nodesource has a good description of this:
> > >> http://nodesource.com/blog/
> > >> > semver-a-primer/
> > >> > > “Semver is a scheme for interface versioning for the benefit of
> > >> > interface consumers, thus if a tool has multiple interfaces, e.g. an
> > API
> > >> > and a CLI, these interfaces may evolve independent versioning.”
> > >> > >
> > >> > > SemVer at its core is a communication mechanism to inform
> developers
> > >> of
> > >> > incompatibilities. In this case the Scala interface is clearly a
> > >> separate
> > >> > entity from the C API.  I.e. changing the Scala namespace isn’t
> going
> > to
> > >> > break C API users.  It does not communicate anything useful to these
> > >> users
> > >> > if we up their major version in response to a Scala change, it
> simply
> > >> > breaks compatibility.  If we group all interfaces together, and
> > >> increment
> > >> > whenever any of them has a breaking change we’ll soon be at MXNet
> > >> version
> > >> > 587.  We’ll be forcing our users to check compatibility and update
> > their
> > >> > dependency tracking constantly.  The end result is that our users
> will
> > >> stop
> > >> > pulling in new versions of the library.
> > >> > >
> > >> > > What I would propose is that (1) we have a high-level SemVer
> system
> > >> that
> > >> > tracks our C_API.  This is the ‘MXNet’ version that we generally
> refer
> > >> to
> > >> > and emphasize for our public releases.  For each API we have an
> > >> independent
> > >> > versioning system that if we can, we fix to the MXNet version.  When
> > it
> > >> > makes sense we version these APIs independently.  So for example we
> > >> could
> > >> > have a MXNet 1.2 release that ships with a 2.0 Scala API / R API.
> > >> > >
> > >> > > In terms of Apache process I think shipping artifacts with a
> > >> non-Apache
> > >> > namespace is a bigger issue than whatever versioning conventions we
> > >> decide
> > >> > to use.
> > >> > >
> > >> > > -Kellen
> > >> > >
> > >> > > From: Carin Meier
> > >> > > Sent: Saturday, March 10, 2018 1:41 PM
> > >> > > To: dev@mxnet.incubator.apache.org
> > >> > > Cc: d...@mxnet.apache.org
> > >> > > Subject: Re: Publishing Scala Package/namespace change
> > >> > >
> > >> > > +1 as well. I'm actively developing a Clojure package for MXNet
> that
> > >> uses
> > >> > > the jars from the Scala package.
> > >> > >
> > >> > > - Carin
> > >> > >
> > >> > > On Fri, Mar 9, 2018 at 4:44 PM, YiZhi Liu <eazhi@gmail.com>
> > >> wrote:
> > >> > >
> > >> > >> +1 for changing the namespace asap. for the maven deploy, we can
> > have
> > >> > >> it build along with pip deployment.
> > >> > >>
> > >> > >>
> > >> > >> 2018-03-09 10:15 GMT-08:00 Naveen Swamy <mnnav...@gmail.com>:
> > >> > >> > Hi Guys,
> > >> > >> >
> > >> > >> > I am working on MXNet Scala Inference APIs
> > >> > >> > <https://issues.apache.org/jira/browse/MXNET-50> along with
> > >> another
> > >> > >> > contributor Roshani. A while back I noticed that we haven't
> been
> > >> > >> publishing
> > >> > >> > the scala package to Maven for a while now(last one being
> > v0.11.1a
> > >> > under
> > >> > >> > the dmlc namespace).
> > >> > >> > Currently users have to build the package manually and then use
> > it,
> > >> > this
> > >> > >> > hinders adoption and also is painful to build everything from
> > >> source.
> > >> > >> >
> > >> > >> > I also see that we haven't changed the namespace to org.apache
> > and
> > >> > >> instead
> > >> > >> > are still ml.dmlc namespace.
> > >> > >> >
> > >> > >> > I wanted to seek your opinion about changing the MXNet-Scala
> > >> package
> > >> > >> > namespace to org.apache for the Scala package and publish to
> > Maven
> > >> in
> > >> > the
> > >> > >> > upcoming release. I understand that this probably breaks the
> > Semver
> > >> > >> > semantics that is agreed upon, However I would like to point
> out
> > >> that
> > >> > the
> > >> > >> > Scala package has never been published to maven as 1.0 under
> > >> > org.apache.
> > >> > >> >
> > >> > >> > Open to suggestions.
> > >> > >> >
> > >> > >> > Thanks, Naveen
> > >> > >>
> > >> > >>
> > >> > >>
> > >> > >> --
> > >> > >> Yizhi Liu
> > >> > >> DMLC member
> > >> > >> Amazon Web Services
> > >> > >> Vancouver, Canada
> > >> > >>
> > >> > >
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Yizhi Liu
> > >> > DMLC member
> > >> > Amazon Web Services
> > >> > Vancouver, Canada
> > >> >
> > >>
> > >
> > >
> >
>



-- 
Yizhi Liu
DMLC member
Amazon Web Services
Vancouver, Canada

Re: Publishing Scala Package/namespace change

[Chris Olivier]

sion -> APIs
> >> >> 1.0 -> Scala_1.0; R_1.0
> >> >> 1.1 -> Scala_1.1; R_2.0
> >> >> 1.2 -> Scala_2.0; R_2.1
> >> >> 1.3 -> Scala_2.1; R_2.2
> >> >> 2.0 -> Scala_3.0; R_3.0
> >>
> >> This would give us the liberty to develop MXNet without restricting us
> too
> >> much - of course, major version increments will still have to be
> >> considered
> >> carefully. I don't think that this would harm transparency too much and
> >> there's no need to write big documentation.
> >>
> >> -Marco
> >>
> >>
> >> On Sun, Mar 11, 2018 at 12:16 PM, YiZhi Liu <liuyi...@apache.org>
> wrote:
> >>
> >> > I have no idea how separating Scala API version can solve the
> >> > 'compatibility' problem. Do you mean we have different version for
> >> > 'ml.dmlc' namespace and 'org.apache' namespace? Do these two versions
> >> > have same behavior? How to tell which Scala API version works with
> >> > which MXNet core version? By document? How many users will read the
> >> > whole document and carefully pair the version id before they run into
> >> > a strange error and give up?
> >> >
> >> > Moreover, changing namespace is an issue that is really rare and
> >> > hardly happens. For other 'compatibility' problem, for example, the
> >> > class/function definitions, should handle the compatibility itself.
> >> > You'll never expect a project to have a different version for changing
> >> > 'calculate(int)' to 'calculate(float)', it should just add a new
> >> > function 'calculate(float)'.
> >> >
> >> > Regarding 'In this case the Scala interface is clearly a separate
> >> > entity from the C API.'. Everything can be seen as a separate entity,
> >> > the mxnet engine, the graph description, operators, python API, gluon
> >> > API, etc. We should think carefully what we want to provide, and what
> >> > our users need.
> >> >
> >> > As an example, Apache Spark, still has SparkR (R API), PySpark (Python
> >> > API), MLLib, GraphX ... as part of its release, and have the same
> >> > version as Spark core as well as its Scala/Java API.
> >> >
> >> > 2018-03-10 23:58 GMT-08:00 kellen sunderland <
> >> kellen.sunderl...@gmail.com
> >> > >:
> >> > > +1 (non-binding) to what Marco is describing.  +1 (non-binding) to
> >> > getting the Scala bindings with the namespace change into Maven.
> >> > >
> >> > > The general best practice for SemVer, which is used by most projects
> >> > that employ SemVer, is to apply SemVer to the public APIs of packages
> >> that
> >> > ship with your project.  If you have several independent APIs this
> could
> >> > mean that they are versioned separately from each other, and from the
> >> > overall project versioning mechanism.
> >> > >
> >> > > For example, the .NET Core library ships with a number of binaries,
> >> each
> >> > with their own SemVerioned APIs.  They also have a high-level, easy to
> >> > understand version for the package as a whole:
> >> > https://docs.microsoft.com/en-us/dotnet/core/versions/.
> >> > >
> >> > > Nodesource has a good description of this:
> >> http://nodesource.com/blog/
> >> > semver-a-primer/
> >> > > “Semver is a scheme for interface versioning for the benefit of
> >> > interface consumers, thus if a tool has multiple interfaces, e.g. an
> API
> >> > and a CLI, these interfaces may evolve independent versioning.”
> >> > >
> >> > > SemVer at its core is a communication mechanism to inform developers
> >> of
> >> > incompatibilities. In this case the Scala interface is clearly a
> >> separate
> >> > entity from the C API.  I.e. changing the Scala namespace isn’t going
> to
> >> > break C API users.  It does not communicate anything useful to these
> >> users
> >> > if we up their major version in response to a Scala change, it simply
> >> > breaks compatibility.  If we group all interfaces together, and
> >> increment
> >> > whenever any of them has a breaking change we’ll soon be at MXNet
> >> version
> >> > 587.  We’ll be forcing our users to check compatibility and update
> their
> >> > depen

Re: Publishing Scala Package/namespace change

[kellen sunderland]

changing
> > 'calculate(int)' to 'calculate(float)', it should just add a new
> > function 'calculate(float)'.
> >
> > Regarding 'In this case the Scala interface is clearly a separate
> > entity from the C API.'. Everything can be seen as a separate entity,
> > the mxnet engine, the graph description, operators, python API, gluon
> > API, etc. We should think carefully what we want to provide, and what
> > our users need.
> >
> > As an example, Apache Spark, still has SparkR (R API), PySpark (Python
> > API), MLLib, GraphX ... as part of its release, and have the same
> > version as Spark core as well as its Scala/Java API.
> >
> > 2018-03-10 23:58 GMT-08:00 kellen sunderland <
> kellen.sunderl...@gmail.com
> > >:
> > > +1 (non-binding) to what Marco is describing.  +1 (non-binding) to
> > getting the Scala bindings with the namespace change into Maven.
> > >
> > > The general best practice for SemVer, which is used by most projects
> > that employ SemVer, is to apply SemVer to the public APIs of packages
> that
> > ship with your project.  If you have several independent APIs this could
> > mean that they are versioned separately from each other, and from the
> > overall project versioning mechanism.
> > >
> > > For example, the .NET Core library ships with a number of binaries,
> each
> > with their own SemVerioned APIs.  They also have a high-level, easy to
> > understand version for the package as a whole:
> > https://docs.microsoft.com/en-us/dotnet/core/versions/.
> > >
> > > Nodesource has a good description of this: http://nodesource.com/blog/
> > semver-a-primer/
> > > “Semver is a scheme for interface versioning for the benefit of
> > interface consumers, thus if a tool has multiple interfaces, e.g. an API
> > and a CLI, these interfaces may evolve independent versioning.”
> > >
> > > SemVer at its core is a communication mechanism to inform developers of
> > incompatibilities. In this case the Scala interface is clearly a separate
> > entity from the C API.  I.e. changing the Scala namespace isn’t going to
> > break C API users.  It does not communicate anything useful to these
> users
> > if we up their major version in response to a Scala change, it simply
> > breaks compatibility.  If we group all interfaces together, and increment
> > whenever any of them has a breaking change we’ll soon be at MXNet version
> > 587.  We’ll be forcing our users to check compatibility and update their
> > dependency tracking constantly.  The end result is that our users will
> stop
> > pulling in new versions of the library.
> > >
> > > What I would propose is that (1) we have a high-level SemVer system
> that
> > tracks our C_API.  This is the ‘MXNet’ version that we generally refer to
> > and emphasize for our public releases.  For each API we have an
> independent
> > versioning system that if we can, we fix to the MXNet version.  When it
> > makes sense we version these APIs independently.  So for example we could
> > have a MXNet 1.2 release that ships with a 2.0 Scala API / R API.
> > >
> > > In terms of Apache process I think shipping artifacts with a non-Apache
> > namespace is a bigger issue than whatever versioning conventions we
> decide
> > to use.
> > >
> > > -Kellen
> > >
> > > From: Carin Meier
> > > Sent: Saturday, March 10, 2018 1:41 PM
> > > To: dev@mxnet.incubator.apache.org
> > > Cc: d...@mxnet.apache.org
> > > Subject: Re: Publishing Scala Package/namespace change
> > >
> > > +1 as well. I'm actively developing a Clojure package for MXNet that
> uses
> > > the jars from the Scala package.
> > >
> > > - Carin
> > >
> > > On Fri, Mar 9, 2018 at 4:44 PM, YiZhi Liu <eazhi@gmail.com> wrote:
> > >
> > >> +1 for changing the namespace asap. for the maven deploy, we can have
> > >> it build along with pip deployment.
> > >>
> > >>
> > >> 2018-03-09 10:15 GMT-08:00 Naveen Swamy <mnnav...@gmail.com>:
> > >> > Hi Guys,
> > >> >
> > >> > I am working on MXNet Scala Inference APIs
> > >> > <https://issues.apache.org/jira/browse/MXNET-50> along with another
> > >> > contributor Roshani. A while back I noticed that we haven't been
> > >> publishing
> > >> > the scala package to Maven for a while now(last one being v0.11.1a
> > under
> > >> > the dmlc namespace).
> > >> > Currently users have to build the package manually and then use it,
> > this
> > >> > hinders adoption and also is painful to build everything from
> source.
> > >> >
> > >> > I also see that we haven't changed the namespace to org.apache and
> > >> instead
> > >> > are still ml.dmlc namespace.
> > >> >
> > >> > I wanted to seek your opinion about changing the MXNet-Scala package
> > >> > namespace to org.apache for the Scala package and publish to Maven
> in
> > the
> > >> > upcoming release. I understand that this probably breaks the Semver
> > >> > semantics that is agreed upon, However I would like to point out
> that
> > the
> > >> > Scala package has never been published to maven as 1.0 under
> > org.apache.
> > >> >
> > >> > Open to suggestions.
> > >> >
> > >> > Thanks, Naveen
> > >>
> > >>
> > >>
> > >> --
> > >> Yizhi Liu
> > >> DMLC member
> > >> Amazon Web Services
> > >> Vancouver, Canada
> > >>
> > >
> >
> >
> >
> > --
> > Yizhi Liu
> > DMLC member
> > Amazon Web Services
> > Vancouver, Canada
> >
>

Re: Publishing Scala Package/namespace change

[YiZhi Liu]

I have no idea how separating Scala API version can solve the
'compatibility' problem. Do you mean we have different version for
'ml.dmlc' namespace and 'org.apache' namespace? Do these two versions
have same behavior? How to tell which Scala API version works with
which MXNet core version? By document? How many users will read the
whole document and carefully pair the version id before they run into
a strange error and give up?

Moreover, changing namespace is an issue that is really rare and
hardly happens. For other 'compatibility' problem, for example, the
class/function definitions, should handle the compatibility itself.
You'll never expect a project to have a different version for changing
'calculate(int)' to 'calculate(float)', it should just add a new
function 'calculate(float)'.

Regarding 'In this case the Scala interface is clearly a separate
entity from the C API.'. Everything can be seen as a separate entity,
the mxnet engine, the graph description, operators, python API, gluon
API, etc. We should think carefully what we want to provide, and what
our users need.

As an example, Apache Spark, still has SparkR (R API), PySpark (Python
API), MLLib, GraphX ... as part of its release, and have the same
version as Spark core as well as its Scala/Java API.

2018-03-10 23:58 GMT-08:00 kellen sunderland <kellen.sunderl...@gmail.com>:
> +1 (non-binding) to what Marco is describing.  +1 (non-binding) to getting 
> the Scala bindings with the namespace change into Maven.
>
> The general best practice for SemVer, which is used by most projects that 
> employ SemVer, is to apply SemVer to the public APIs of packages that ship 
> with your project.  If you have several independent APIs this could mean that 
> they are versioned separately from each other, and from the overall project 
> versioning mechanism.
>
> For example, the .NET Core library ships with a number of binaries, each with 
> their own SemVerioned APIs.  They also have a high-level, easy to understand 
> version for the package as a whole: 
> https://docs.microsoft.com/en-us/dotnet/core/versions/.
>
> Nodesource has a good description of this: 
> http://nodesource.com/blog/semver-a-primer/
> “Semver is a scheme for interface versioning for the benefit of interface 
> consumers, thus if a tool has multiple interfaces, e.g. an API and a CLI, 
> these interfaces may evolve independent versioning.”
>
> SemVer at its core is a communication mechanism to inform developers of 
> incompatibilities. In this case the Scala interface is clearly a separate 
> entity from the C API.  I.e. changing the Scala namespace isn’t going to 
> break C API users.  It does not communicate anything useful to these users if 
> we up their major version in response to a Scala change, it simply breaks 
> compatibility.  If we group all interfaces together, and increment whenever 
> any of them has a breaking change we’ll soon be at MXNet version 587.  We’ll 
> be forcing our users to check compatibility and update their dependency 
> tracking constantly.  The end result is that our users will stop pulling in 
> new versions of the library.
>
> What I would propose is that (1) we have a high-level SemVer system that 
> tracks our C_API.  This is the ‘MXNet’ version that we generally refer to and 
> emphasize for our public releases.  For each API we have an independent 
> versioning system that if we can, we fix to the MXNet version.  When it makes 
> sense we version these APIs independently.  So for example we could have a 
> MXNet 1.2 release that ships with a 2.0 Scala API / R API.
>
> In terms of Apache process I think shipping artifacts with a non-Apache 
> namespace is a bigger issue than whatever versioning conventions we decide to 
> use.
>
> -Kellen
>
> From: Carin Meier
> Sent: Saturday, March 10, 2018 1:41 PM
> To: dev@mxnet.incubator.apache.org
> Cc: d...@mxnet.apache.org
> Subject: Re: Publishing Scala Package/namespace change
>
> +1 as well. I'm actively developing a Clojure package for MXNet that uses
> the jars from the Scala package.
>
> - Carin
>
> On Fri, Mar 9, 2018 at 4:44 PM, YiZhi Liu <eazhi@gmail.com> wrote:
>
>> +1 for changing the namespace asap. for the maven deploy, we can have
>> it build along with pip deployment.
>>
>>
>> 2018-03-09 10:15 GMT-08:00 Naveen Swamy <mnnav...@gmail.com>:
>> > Hi Guys,
>> >
>> > I am working on MXNet Scala Inference APIs
>> > <https://issues.apache.org/jira/browse/MXNET-50> along with another
>> > contributor Roshani. A while back I noticed that we haven't been
>> publishing
>> > the scala package to Maven for a while now(last one being v0.11.1a under
>> > the dmlc namespace).
>> > Currently users have 

RE: Publishing Scala Package/namespace change

[kellen sunderland]

+1 (non-binding) to what Marco is describing.  +1 (non-binding) to getting the 
Scala bindings with the namespace change into Maven.

The general best practice for SemVer, which is used by most projects that 
employ SemVer, is to apply SemVer to the public APIs of packages that ship with 
your project.  If you have several independent APIs this could mean that they 
are versioned separately from each other, and from the overall project 
versioning mechanism.  

For example, the .NET Core library ships with a number of binaries, each with 
their own SemVerioned APIs.  They also have a high-level, easy to understand 
version for the package as a whole: 
https://docs.microsoft.com/en-us/dotnet/core/versions/.

Nodesource has a good description of this: 
http://nodesource.com/blog/semver-a-primer/
“Semver is a scheme for interface versioning for the benefit of interface 
consumers, thus if a tool has multiple interfaces, e.g. an API and a CLI, these 
interfaces may evolve independent versioning.”

SemVer at its core is a communication mechanism to inform developers of 
incompatibilities. In this case the Scala interface is clearly a separate 
entity from the C API.  I.e. changing the Scala namespace isn’t going to break 
C API users.  It does not communicate anything useful to these users if we up 
their major version in response to a Scala change, it simply breaks 
compatibility.  If we group all interfaces together, and increment whenever any 
of them has a breaking change we’ll soon be at MXNet version 587.  We’ll be 
forcing our users to check compatibility and update their dependency tracking 
constantly.  The end result is that our users will stop pulling in new versions 
of the library.

What I would propose is that (1) we have a high-level SemVer system that tracks 
our C_API.  This is the ‘MXNet’ version that we generally refer to and 
emphasize for our public releases.  For each API we have an independent 
versioning system that if we can, we fix to the MXNet version.  When it makes 
sense we version these APIs independently.  So for example we could have a 
MXNet 1.2 release that ships with a 2.0 Scala API / R API.

In terms of Apache process I think shipping artifacts with a non-Apache 
namespace is a bigger issue than whatever versioning conventions we decide to 
use.

-Kellen

From: Carin Meier
Sent: Saturday, March 10, 2018 1:41 PM
To: dev@mxnet.incubator.apache.org
Cc: d...@mxnet.apache.org
Subject: Re: Publishing Scala Package/namespace change

+1 as well. I'm actively developing a Clojure package for MXNet that uses
the jars from the Scala package.

- Carin

On Fri, Mar 9, 2018 at 4:44 PM, YiZhi Liu <eazhi@gmail.com> wrote:

> +1 for changing the namespace asap. for the maven deploy, we can have
> it build along with pip deployment.
>
>
> 2018-03-09 10:15 GMT-08:00 Naveen Swamy <mnnav...@gmail.com>:
> > Hi Guys,
> >
> > I am working on MXNet Scala Inference APIs
> > <https://issues.apache.org/jira/browse/MXNET-50> along with another
> > contributor Roshani. A while back I noticed that we haven't been
> publishing
> > the scala package to Maven for a while now(last one being v0.11.1a under
> > the dmlc namespace).
> > Currently users have to build the package manually and then use it, this
> > hinders adoption and also is painful to build everything from source.
> >
> > I also see that we haven't changed the namespace to org.apache and
> instead
> > are still ml.dmlc namespace.
> >
> > I wanted to seek your opinion about changing the MXNet-Scala package
> > namespace to org.apache for the Scala package and publish to Maven in the
> > upcoming release. I understand that this probably breaks the Semver
> > semantics that is agreed upon, However I would like to point out that the
> > Scala package has never been published to maven as 1.0 under org.apache.
> >
> > Open to suggestions.
> >
> > Thanks, Naveen
>
>
>
> --
> Yizhi Liu
> DMLC member
> Amazon Web Services
> Vancouver, Canada
>

Re: Publishing Scala Package/namespace change

[Carin Meier]

+1 as well. I'm actively developing a Clojure package for MXNet that uses
the jars from the Scala package.

- Carin

On Fri, Mar 9, 2018 at 4:44 PM, YiZhi Liu  wrote:

> +1 for changing the namespace asap. for the maven deploy, we can have
> it build along with pip deployment.
>
>
> 2018-03-09 10:15 GMT-08:00 Naveen Swamy :
> > Hi Guys,
> >
> > I am working on MXNet Scala Inference APIs
> >  along with another
> > contributor Roshani. A while back I noticed that we haven't been
> publishing
> > the scala package to Maven for a while now(last one being v0.11.1a under
> > the dmlc namespace).
> > Currently users have to build the package manually and then use it, this
> > hinders adoption and also is painful to build everything from source.
> >
> > I also see that we haven't changed the namespace to org.apache and
> instead
> > are still ml.dmlc namespace.
> >
> > I wanted to seek your opinion about changing the MXNet-Scala package
> > namespace to org.apache for the Scala package and publish to Maven in the
> > upcoming release. I understand that this probably breaks the Semver
> > semantics that is agreed upon, However I would like to point out that the
> > Scala package has never been published to maven as 1.0 under org.apache.
> >
> > Open to suggestions.
> >
> > Thanks, Naveen
>
>
>
> --
> Yizhi Liu
> DMLC member
> Amazon Web Services
> Vancouver, Canada
>

Re: Publishing Scala Package/namespace change

[Chris Olivier]

user: “I’m having a problem with Scala API”.
dev: “What version?”
user: “1.4”
dev: “Is that Scala API version or MXNET version?”
user: “uhh... scala version, maybe?”
dev: “ok, which mxnet version?”
user: “i don’t know, how do I find that out?”
dev : “ummm go look in blah blah directory and tell me libmxnet.so file
date”
user: “nevermind, I’ll just use Caffe”


On Fri, Mar 9, 2018 at 4:43 PM Marco de Abreu 
wrote:

> My proposal was to have separate versioning but same release cycles -
> having separate cycles was just an idea.
>
> So how would we approach this problem in future if there's a big
> improvement on one of our language bindings? Like I said previously, it
> might be fine for this case, but we will run into the same issues for the
> next time and it's not a great user experience if we just do this
> "silently", provide no transitional solution and just replace an existing
> API with a new improved one.
>
> I would like to have a solution for this problem before we move on.
>
> -Marco
>
>
>
> On Sat, Mar 10, 2018 at 12:06 AM, Nan Zhu  wrote:
>
> > I think last time we postpone it because the release is a minor version
> >
> > but actually such a change is actually affordable for a jump from 1.1 -
> 1.2
> >
> > -1 on separate versions (not following apache rules)
> >
> >
> > On Fri, Mar 9, 2018 at 2:38 PM, Chris Olivier 
> > wrote:
> >
> > > IMHO, I don't think having separate versions and releases for the scala
> > API
> > > will work for the following reasons:
> > >
> > >- Scala API is not its own Apache project, so we don't really have a
> > >mechanism to release it separately and not the manpower of
> volunteers
> > to
> > >maintain all the BS that goes along with releases
> > >- We operate under the assumption (which is fair, I think) that the
> > >Scala API is only compatible with the exact C API for which it was
> > built
> > >against.  Since the Scala API ships with its own libmxnet.so, and
> > >libmxnet.so is "stable" only at release boundaries, then it would be
> > > risky
> > >to pick arbitrary points in the Scala evolution as "new versions"
> and
> > > birth
> > >them into the World with just whatever mxnet commit was at the top
> at
> > > that
> > >time
> > >
> > > I am also in favor of changing the namespace ASAP before the massive
> > > adoption of the Scala API that's about to happen, because it'll be way
> > > harder to do later.
> > >
> > > -Chris
> > >
> > >
> > > On Fri, Mar 9, 2018 at 2:09 PM, Naveen Swamy 
> wrote:
> > >
> > > > MXNet Scala APis already generate a MXNet Core Scala package based
> off
> > > the
> > > > cpp backend already. I think customers who are building from source
> > would
> > > > love to get Maven package given that it takes so much pain.
> > > >
> > > > Are you suggesting we take MXNet-Scala APIs into a separate release
> > > cycle,
> > > > it is possible and can start with this one but It would not make a
> lot
> > of
> > > > sense to start MXNet-Scala 1.0 depending on MXNet 1.0(cpp). This
> won't
> > > very
> > > > different from breaking backward compatibility when we release a new
> > > > package.
> > > >
> > > > IMO managing separate release cycles for different language bindings
> > > could
> > > > turn into a lot of work for the community unnecessarily especially
> > since
> > > > they are closely
> > > >
> > > > On Fri, Mar 9, 2018 at 1:56 PM, Marco de Abreu <
> > > > marco.g.ab...@googlemail.com
> > > > > wrote:
> > > >
> > > > > While it has never been published, there have still been releases
> > under
> > > > > Apache and - as you mentioned - customers that build off the
> source.
> > > This
> > > > > would cause compatibility issues.
> > > > >
> > > > > In general I actively support the idea of enhancing the Scala
> > package,
> > > > but
> > > > > I think that we have to solve another problem first. At the moment,
> > all
> > > > > APIs are bound to the MXNet core versioning and vica versa.
> > > > >
> > > > > In my opinion, we should first separate the APIs from the MXNet
> core,
> > > > start
> > > > > versioning them separately and then make changes like these. While
> it
> > > > would
> > > > > be possible (although not right) to make an exception here, we
> still
> > > > don't
> > > > > solve the root problem and we are going to run into the same issues
> > > with
> > > > > the next big API update.
> > > > >
> > > > > Just to mention another example: our team got a request to rewrite
> > the
> > > > Cpp
> > > > > package, but we actually would not be able to merge it into MXNet
> > since
> > > > it
> > > > > breaks the existing Cpp package API - means we would need a major
> > > version
> > > > > increase.
> > > > >
> > > > > We really should solve this problem once and for all, giving back a
> > lot
> > > > of
> > > > > freedom and reducing overhead in the long term.
> > > > >
> > > > > -Marco

Re: Publishing Scala Package/namespace change

[Marco de Abreu]

My proposal was to have separate versioning but same release cycles -
having separate cycles was just an idea.

So how would we approach this problem in future if there's a big
improvement on one of our language bindings? Like I said previously, it
might be fine for this case, but we will run into the same issues for the
next time and it's not a great user experience if we just do this
"silently", provide no transitional solution and just replace an existing
API with a new improved one.

I would like to have a solution for this problem before we move on.

-Marco



On Sat, Mar 10, 2018 at 12:06 AM, Nan Zhu  wrote:

> I think last time we postpone it because the release is a minor version
>
> but actually such a change is actually affordable for a jump from 1.1 - 1.2
>
> -1 on separate versions (not following apache rules)
>
>
> On Fri, Mar 9, 2018 at 2:38 PM, Chris Olivier 
> wrote:
>
> > IMHO, I don't think having separate versions and releases for the scala
> API
> > will work for the following reasons:
> >
> >- Scala API is not its own Apache project, so we don't really have a
> >mechanism to release it separately and not the manpower of volunteers
> to
> >maintain all the BS that goes along with releases
> >- We operate under the assumption (which is fair, I think) that the
> >Scala API is only compatible with the exact C API for which it was
> built
> >against.  Since the Scala API ships with its own libmxnet.so, and
> >libmxnet.so is "stable" only at release boundaries, then it would be
> > risky
> >to pick arbitrary points in the Scala evolution as "new versions" and
> > birth
> >them into the World with just whatever mxnet commit was at the top at
> > that
> >time
> >
> > I am also in favor of changing the namespace ASAP before the massive
> > adoption of the Scala API that's about to happen, because it'll be way
> > harder to do later.
> >
> > -Chris
> >
> >
> > On Fri, Mar 9, 2018 at 2:09 PM, Naveen Swamy  wrote:
> >
> > > MXNet Scala APis already generate a MXNet Core Scala package based off
> > the
> > > cpp backend already. I think customers who are building from source
> would
> > > love to get Maven package given that it takes so much pain.
> > >
> > > Are you suggesting we take MXNet-Scala APIs into a separate release
> > cycle,
> > > it is possible and can start with this one but It would not make a lot
> of
> > > sense to start MXNet-Scala 1.0 depending on MXNet 1.0(cpp). This won't
> > very
> > > different from breaking backward compatibility when we release a new
> > > package.
> > >
> > > IMO managing separate release cycles for different language bindings
> > could
> > > turn into a lot of work for the community unnecessarily especially
> since
> > > they are closely
> > >
> > > On Fri, Mar 9, 2018 at 1:56 PM, Marco de Abreu <
> > > marco.g.ab...@googlemail.com
> > > > wrote:
> > >
> > > > While it has never been published, there have still been releases
> under
> > > > Apache and - as you mentioned - customers that build off the source.
> > This
> > > > would cause compatibility issues.
> > > >
> > > > In general I actively support the idea of enhancing the Scala
> package,
> > > but
> > > > I think that we have to solve another problem first. At the moment,
> all
> > > > APIs are bound to the MXNet core versioning and vica versa.
> > > >
> > > > In my opinion, we should first separate the APIs from the MXNet core,
> > > start
> > > > versioning them separately and then make changes like these. While it
> > > would
> > > > be possible (although not right) to make an exception here, we still
> > > don't
> > > > solve the root problem and we are going to run into the same issues
> > with
> > > > the next big API update.
> > > >
> > > > Just to mention another example: our team got a request to rewrite
> the
> > > Cpp
> > > > package, but we actually would not be able to merge it into MXNet
> since
> > > it
> > > > breaks the existing Cpp package API - means we would need a major
> > version
> > > > increase.
> > > >
> > > > We really should solve this problem once and for all, giving back a
> lot
> > > of
> > > > freedom and reducing overhead in the long term.
> > > >
> > > > -Marco
> > > >
> > > > YiZhi Liu  schrieb am Fr., 9. März 2018, 22:44:
> > > >
> > > > > +1 for changing the namespace asap. for the maven deploy, we can
> have
> > > > > it build along with pip deployment.
> > > > >
> > > > >
> > > > > 2018-03-09 10:15 GMT-08:00 Naveen Swamy :
> > > > > > Hi Guys,
> > > > > >
> > > > > > I am working on MXNet Scala Inference APIs
> > > > > >  along with
> > another
> > > > > > contributor Roshani. A while back I noticed that we haven't been
> > > > > publishing
> > > > > > the scala package to Maven for a while now(last one being
> v0.11.1a
> > > > under
> > > > > > the dmlc namespace).

Re: Publishing Scala Package/namespace change

[Nan Zhu]

I think last time we postpone it because the release is a minor version

but actually such a change is actually affordable for a jump from 1.1 - 1.2

-1 on separate versions (not following apache rules)


On Fri, Mar 9, 2018 at 2:38 PM, Chris Olivier  wrote:

> IMHO, I don't think having separate versions and releases for the scala API
> will work for the following reasons:
>
>- Scala API is not its own Apache project, so we don't really have a
>mechanism to release it separately and not the manpower of volunteers to
>maintain all the BS that goes along with releases
>- We operate under the assumption (which is fair, I think) that the
>Scala API is only compatible with the exact C API for which it was built
>against.  Since the Scala API ships with its own libmxnet.so, and
>libmxnet.so is "stable" only at release boundaries, then it would be
> risky
>to pick arbitrary points in the Scala evolution as "new versions" and
> birth
>them into the World with just whatever mxnet commit was at the top at
> that
>time
>
> I am also in favor of changing the namespace ASAP before the massive
> adoption of the Scala API that's about to happen, because it'll be way
> harder to do later.
>
> -Chris
>
>
> On Fri, Mar 9, 2018 at 2:09 PM, Naveen Swamy  wrote:
>
> > MXNet Scala APis already generate a MXNet Core Scala package based off
> the
> > cpp backend already. I think customers who are building from source would
> > love to get Maven package given that it takes so much pain.
> >
> > Are you suggesting we take MXNet-Scala APIs into a separate release
> cycle,
> > it is possible and can start with this one but It would not make a lot of
> > sense to start MXNet-Scala 1.0 depending on MXNet 1.0(cpp). This won't
> very
> > different from breaking backward compatibility when we release a new
> > package.
> >
> > IMO managing separate release cycles for different language bindings
> could
> > turn into a lot of work for the community unnecessarily especially since
> > they are closely
> >
> > On Fri, Mar 9, 2018 at 1:56 PM, Marco de Abreu <
> > marco.g.ab...@googlemail.com
> > > wrote:
> >
> > > While it has never been published, there have still been releases under
> > > Apache and - as you mentioned - customers that build off the source.
> This
> > > would cause compatibility issues.
> > >
> > > In general I actively support the idea of enhancing the Scala package,
> > but
> > > I think that we have to solve another problem first. At the moment, all
> > > APIs are bound to the MXNet core versioning and vica versa.
> > >
> > > In my opinion, we should first separate the APIs from the MXNet core,
> > start
> > > versioning them separately and then make changes like these. While it
> > would
> > > be possible (although not right) to make an exception here, we still
> > don't
> > > solve the root problem and we are going to run into the same issues
> with
> > > the next big API update.
> > >
> > > Just to mention another example: our team got a request to rewrite the
> > Cpp
> > > package, but we actually would not be able to merge it into MXNet since
> > it
> > > breaks the existing Cpp package API - means we would need a major
> version
> > > increase.
> > >
> > > We really should solve this problem once and for all, giving back a lot
> > of
> > > freedom and reducing overhead in the long term.
> > >
> > > -Marco
> > >
> > > YiZhi Liu  schrieb am Fr., 9. März 2018, 22:44:
> > >
> > > > +1 for changing the namespace asap. for the maven deploy, we can have
> > > > it build along with pip deployment.
> > > >
> > > >
> > > > 2018-03-09 10:15 GMT-08:00 Naveen Swamy :
> > > > > Hi Guys,
> > > > >
> > > > > I am working on MXNet Scala Inference APIs
> > > > >  along with
> another
> > > > > contributor Roshani. A while back I noticed that we haven't been
> > > > publishing
> > > > > the scala package to Maven for a while now(last one being v0.11.1a
> > > under
> > > > > the dmlc namespace).
> > > > > Currently users have to build the package manually and then use it,
> > > this
> > > > > hinders adoption and also is painful to build everything from
> source.
> > > > >
> > > > > I also see that we haven't changed the namespace to org.apache and
> > > > instead
> > > > > are still ml.dmlc namespace.
> > > > >
> > > > > I wanted to seek your opinion about changing the MXNet-Scala
> package
> > > > > namespace to org.apache for the Scala package and publish to Maven
> in
> > > the
> > > > > upcoming release. I understand that this probably breaks the Semver
> > > > > semantics that is agreed upon, However I would like to point out
> that
> > > the
> > > > > Scala package has never been published to maven as 1.0 under
> > > org.apache.
> > > > >
> > > > > Open to suggestions.
> > > > >
> > > > > Thanks, Naveen
> > > >
> > > >
> > > >
> > > > --
> > > > Yizhi Liu
> > 

Re: Publishing Scala Package/namespace change

[Chris Olivier]

IMHO, I don't think having separate versions and releases for the scala API
will work for the following reasons:

   - Scala API is not its own Apache project, so we don't really have a
   mechanism to release it separately and not the manpower of volunteers to
   maintain all the BS that goes along with releases
   - We operate under the assumption (which is fair, I think) that the
   Scala API is only compatible with the exact C API for which it was built
   against.  Since the Scala API ships with its own libmxnet.so, and
   libmxnet.so is "stable" only at release boundaries, then it would be risky
   to pick arbitrary points in the Scala evolution as "new versions" and birth
   them into the World with just whatever mxnet commit was at the top at that
   time

I am also in favor of changing the namespace ASAP before the massive
adoption of the Scala API that's about to happen, because it'll be way
harder to do later.

-Chris


On Fri, Mar 9, 2018 at 2:09 PM, Naveen Swamy  wrote:

> MXNet Scala APis already generate a MXNet Core Scala package based off the
> cpp backend already. I think customers who are building from source would
> love to get Maven package given that it takes so much pain.
>
> Are you suggesting we take MXNet-Scala APIs into a separate release cycle,
> it is possible and can start with this one but It would not make a lot of
> sense to start MXNet-Scala 1.0 depending on MXNet 1.0(cpp). This won't very
> different from breaking backward compatibility when we release a new
> package.
>
> IMO managing separate release cycles for different language bindings could
> turn into a lot of work for the community unnecessarily especially since
> they are closely
>
> On Fri, Mar 9, 2018 at 1:56 PM, Marco de Abreu <
> marco.g.ab...@googlemail.com
> > wrote:
>
> > While it has never been published, there have still been releases under
> > Apache and - as you mentioned - customers that build off the source. This
> > would cause compatibility issues.
> >
> > In general I actively support the idea of enhancing the Scala package,
> but
> > I think that we have to solve another problem first. At the moment, all
> > APIs are bound to the MXNet core versioning and vica versa.
> >
> > In my opinion, we should first separate the APIs from the MXNet core,
> start
> > versioning them separately and then make changes like these. While it
> would
> > be possible (although not right) to make an exception here, we still
> don't
> > solve the root problem and we are going to run into the same issues with
> > the next big API update.
> >
> > Just to mention another example: our team got a request to rewrite the
> Cpp
> > package, but we actually would not be able to merge it into MXNet since
> it
> > breaks the existing Cpp package API - means we would need a major version
> > increase.
> >
> > We really should solve this problem once and for all, giving back a lot
> of
> > freedom and reducing overhead in the long term.
> >
> > -Marco
> >
> > YiZhi Liu  schrieb am Fr., 9. März 2018, 22:44:
> >
> > > +1 for changing the namespace asap. for the maven deploy, we can have
> > > it build along with pip deployment.
> > >
> > >
> > > 2018-03-09 10:15 GMT-08:00 Naveen Swamy :
> > > > Hi Guys,
> > > >
> > > > I am working on MXNet Scala Inference APIs
> > > >  along with another
> > > > contributor Roshani. A while back I noticed that we haven't been
> > > publishing
> > > > the scala package to Maven for a while now(last one being v0.11.1a
> > under
> > > > the dmlc namespace).
> > > > Currently users have to build the package manually and then use it,
> > this
> > > > hinders adoption and also is painful to build everything from source.
> > > >
> > > > I also see that we haven't changed the namespace to org.apache and
> > > instead
> > > > are still ml.dmlc namespace.
> > > >
> > > > I wanted to seek your opinion about changing the MXNet-Scala package
> > > > namespace to org.apache for the Scala package and publish to Maven in
> > the
> > > > upcoming release. I understand that this probably breaks the Semver
> > > > semantics that is agreed upon, However I would like to point out that
> > the
> > > > Scala package has never been published to maven as 1.0 under
> > org.apache.
> > > >
> > > > Open to suggestions.
> > > >
> > > > Thanks, Naveen
> > >
> > >
> > >
> > > --
> > > Yizhi Liu
> > > DMLC member
> > > Amazon Web Services
> > > Vancouver, Canada
> > >
> >
>

Re: Publishing Scala Package/namespace change

[Marco de Abreu]

I agree, separate release cycles would cause a lot of hassle and grant now
visible benefit, I'd bind these to the main release cycles. But I would
separate the versioning of each package.

At the moment, we are bound by the least common denominator in terms of API
changes and versions. While this gives stability to our users, it limits us
in our development since we have to consider backwards compatibility for
more than half a dozen languages. Especially considering this being an open
source project, it's hard to manage this without significant constraints.

With that proposal we would have the core of MXNet as some type of hardened
and "compact" but fully functional and backwards  compatible kernel with a
C/C++ API. This would be the main version of MXNet. If there is suddenly a
big community effort to heavily improve a certain API (like Scala), we
would not block them with semver being the reason but just increase the
version of that specific API. This gives us flexibility and our customers
the possibility to benefit from an extensively improved API.

Something we also could consider with this clear separation would be to
give the possibility to have a legacy API for some time and basically offer
two versions. We would be maintaining both for a few months to give
customers the possibility to migrate and then remove it. At the moment, a
layout like this is hard to pull off.

I think we should really work on this topic as a community and provide a
good and maintainable solution. In the end, it's a huge plus (that is being
appreciated by our users) for MXNet that we are offering such a variety of
APIs. Now we need a concept to maintain them properly.

-Marco

Naveen Swamy  schrieb am Fr., 9. März 2018, 23:10:

> MXNet Scala APis already generate a MXNet Core Scala package based off the
> cpp backend already. I think customers who are building from source would
> love to get Maven package given that it takes so much pain.
>
> Are you suggesting we take MXNet-Scala APIs into a separate release cycle,
> it is possible and can start with this one but It would not make a lot of
> sense to start MXNet-Scala 1.0 depending on MXNet 1.0(cpp). This won't very
> different from breaking backward compatibility when we release a new
> package.
>
> IMO managing separate release cycles for different language bindings could
> turn into a lot of work for the community unnecessarily especially since
> they are closely
>
> On Fri, Mar 9, 2018 at 1:56 PM, Marco de Abreu <
> marco.g.ab...@googlemail.com
> > wrote:
>
> > While it has never been published, there have still been releases under
> > Apache and - as you mentioned - customers that build off the source. This
> > would cause compatibility issues.
> >
> > In general I actively support the idea of enhancing the Scala package,
> but
> > I think that we have to solve another problem first. At the moment, all
> > APIs are bound to the MXNet core versioning and vica versa.
> >
> > In my opinion, we should first separate the APIs from the MXNet core,
> start
> > versioning them separately and then make changes like these. While it
> would
> > be possible (although not right) to make an exception here, we still
> don't
> > solve the root problem and we are going to run into the same issues with
> > the next big API update.
> >
> > Just to mention another example: our team got a request to rewrite the
> Cpp
> > package, but we actually would not be able to merge it into MXNet since
> it
> > breaks the existing Cpp package API - means we would need a major version
> > increase.
> >
> > We really should solve this problem once and for all, giving back a lot
> of
> > freedom and reducing overhead in the long term.
> >
> > -Marco
> >
> > YiZhi Liu  schrieb am Fr., 9. März 2018, 22:44:
> >
> > > +1 for changing the namespace asap. for the maven deploy, we can have
> > > it build along with pip deployment.
> > >
> > >
> > > 2018-03-09 10:15 GMT-08:00 Naveen Swamy :
> > > > Hi Guys,
> > > >
> > > > I am working on MXNet Scala Inference APIs
> > > >  along with another
> > > > contributor Roshani. A while back I noticed that we haven't been
> > > publishing
> > > > the scala package to Maven for a while now(last one being v0.11.1a
> > under
> > > > the dmlc namespace).
> > > > Currently users have to build the package manually and then use it,
> > this
> > > > hinders adoption and also is painful to build everything from source.
> > > >
> > > > I also see that we haven't changed the namespace to org.apache and
> > > instead
> > > > are still ml.dmlc namespace.
> > > >
> > > > I wanted to seek your opinion about changing the MXNet-Scala package
> > > > namespace to org.apache for the Scala package and publish to Maven in
> > the
> > > > upcoming release. I understand that this probably breaks the Semver
> > > > semantics that is agreed upon, However I would like to point out that

Re: Publishing Scala Package/namespace change

[Naveen Swamy]

MXNet Scala APis already generate a MXNet Core Scala package based off the
cpp backend already. I think customers who are building from source would
love to get Maven package given that it takes so much pain.

Are you suggesting we take MXNet-Scala APIs into a separate release cycle,
it is possible and can start with this one but It would not make a lot of
sense to start MXNet-Scala 1.0 depending on MXNet 1.0(cpp). This won't very
different from breaking backward compatibility when we release a new
package.

IMO managing separate release cycles for different language bindings could
turn into a lot of work for the community unnecessarily especially since
they are closely

On Fri, Mar 9, 2018 at 1:56 PM, Marco de Abreu  wrote:

> While it has never been published, there have still been releases under
> Apache and - as you mentioned - customers that build off the source. This
> would cause compatibility issues.
>
> In general I actively support the idea of enhancing the Scala package, but
> I think that we have to solve another problem first. At the moment, all
> APIs are bound to the MXNet core versioning and vica versa.
>
> In my opinion, we should first separate the APIs from the MXNet core, start
> versioning them separately and then make changes like these. While it would
> be possible (although not right) to make an exception here, we still don't
> solve the root problem and we are going to run into the same issues with
> the next big API update.
>
> Just to mention another example: our team got a request to rewrite the Cpp
> package, but we actually would not be able to merge it into MXNet since it
> breaks the existing Cpp package API - means we would need a major version
> increase.
>
> We really should solve this problem once and for all, giving back a lot of
> freedom and reducing overhead in the long term.
>
> -Marco
>
> YiZhi Liu  schrieb am Fr., 9. März 2018, 22:44:
>
> > +1 for changing the namespace asap. for the maven deploy, we can have
> > it build along with pip deployment.
> >
> >
> > 2018-03-09 10:15 GMT-08:00 Naveen Swamy :
> > > Hi Guys,
> > >
> > > I am working on MXNet Scala Inference APIs
> > >  along with another
> > > contributor Roshani. A while back I noticed that we haven't been
> > publishing
> > > the scala package to Maven for a while now(last one being v0.11.1a
> under
> > > the dmlc namespace).
> > > Currently users have to build the package manually and then use it,
> this
> > > hinders adoption and also is painful to build everything from source.
> > >
> > > I also see that we haven't changed the namespace to org.apache and
> > instead
> > > are still ml.dmlc namespace.
> > >
> > > I wanted to seek your opinion about changing the MXNet-Scala package
> > > namespace to org.apache for the Scala package and publish to Maven in
> the
> > > upcoming release. I understand that this probably breaks the Semver
> > > semantics that is agreed upon, However I would like to point out that
> the
> > > Scala package has never been published to maven as 1.0 under
> org.apache.
> > >
> > > Open to suggestions.
> > >
> > > Thanks, Naveen
> >
> >
> >
> > --
> > Yizhi Liu
> > DMLC member
> > Amazon Web Services
> > Vancouver, Canada
> >
>

Re: Publishing Scala Package/namespace change

[Marco de Abreu]

While it has never been published, there have still been releases under
Apache and - as you mentioned - customers that build off the source. This
would cause compatibility issues.

In general I actively support the idea of enhancing the Scala package, but
I think that we have to solve another problem first. At the moment, all
APIs are bound to the MXNet core versioning and vica versa.

In my opinion, we should first separate the APIs from the MXNet core, start
versioning them separately and then make changes like these. While it would
be possible (although not right) to make an exception here, we still don't
solve the root problem and we are going to run into the same issues with
the next big API update.

Just to mention another example: our team got a request to rewrite the Cpp
package, but we actually would not be able to merge it into MXNet since it
breaks the existing Cpp package API - means we would need a major version
increase.

We really should solve this problem once and for all, giving back a lot of
freedom and reducing overhead in the long term.

-Marco

YiZhi Liu  schrieb am Fr., 9. März 2018, 22:44:

> +1 for changing the namespace asap. for the maven deploy, we can have
> it build along with pip deployment.
>
>
> 2018-03-09 10:15 GMT-08:00 Naveen Swamy :
> > Hi Guys,
> >
> > I am working on MXNet Scala Inference APIs
> >  along with another
> > contributor Roshani. A while back I noticed that we haven't been
> publishing
> > the scala package to Maven for a while now(last one being v0.11.1a under
> > the dmlc namespace).
> > Currently users have to build the package manually and then use it, this
> > hinders adoption and also is painful to build everything from source.
> >
> > I also see that we haven't changed the namespace to org.apache and
> instead
> > are still ml.dmlc namespace.
> >
> > I wanted to seek your opinion about changing the MXNet-Scala package
> > namespace to org.apache for the Scala package and publish to Maven in the
> > upcoming release. I understand that this probably breaks the Semver
> > semantics that is agreed upon, However I would like to point out that the
> > Scala package has never been published to maven as 1.0 under org.apache.
> >
> > Open to suggestions.
> >
> > Thanks, Naveen
>
>
>
> --
> Yizhi Liu
> DMLC member
> Amazon Web Services
> Vancouver, Canada
>

Re: Publishing Scala Package/namespace change

[YiZhi Liu]

+1 for changing the namespace asap. for the maven deploy, we can have
it build along with pip deployment.


2018-03-09 10:15 GMT-08:00 Naveen Swamy :
> Hi Guys,
>
> I am working on MXNet Scala Inference APIs
>  along with another
> contributor Roshani. A while back I noticed that we haven't been publishing
> the scala package to Maven for a while now(last one being v0.11.1a under
> the dmlc namespace).
> Currently users have to build the package manually and then use it, this
> hinders adoption and also is painful to build everything from source.
>
> I also see that we haven't changed the namespace to org.apache and instead
> are still ml.dmlc namespace.
>
> I wanted to seek your opinion about changing the MXNet-Scala package
> namespace to org.apache for the Scala package and publish to Maven in the
> upcoming release. I understand that this probably breaks the Semver
> semantics that is agreed upon, However I would like to point out that the
> Scala package has never been published to maven as 1.0 under org.apache.
>
> Open to suggestions.
>
> Thanks, Naveen



-- 
Yizhi Liu
DMLC member
Amazon Web Services
Vancouver, Canada