autogenerated page like
>>>>>> https://download.pytorch.org/whl/nightly/cu101/torch_nightly.html
>>>>>>
>>>>>> Then "pip install -f URLTOPAGE mxnet" will install the latest
>> available
>>>>>> version.
>&
Mon, 2020-01-06 at 10:01 -0800, Lin Yuan wrote:
> > >>> +1 for a nightly pip with fixed name.
> > >>>
> > >>> We need this to track mxnet integration with other packages such as
> > >>> Horovod.
> > >>>
> > >>> Sam, when
gt; name?
> >>>
> >>> Thanks,
> >>>
> >>> Lin
> >>>
> >>> On Sun, Jan 5, 2020 at 7:48 PM Skalicky, Sam
> >>> mailto:sska...@amazon.com.invalid>>
> >>> wrote:
> >>>
> >>> Hi Tao,
> >>&g
ailto:sska...@amazon.com.invalid>>
>>> wrote:
>>>
>>> Hi Tao,
>>>
>>> We dont have this yet, but we did think about putting the latest
>>> wheels in
>>> a specific place in the s3 bucket so they are always updated.
>>> Initially we
>>
ic place in the s3 bucket so they are always updated.
> > Initially we
> > decided not to do this since the main MXNet CD should have been
> > fixed. But
> > since its still not fixed yet, we might try and go ahead and do
> > this.
> >
> > Sam
> >
> > On
to install the latest available build of a flavor without
> specifying
> the build date? Something like `pip install mxnet --pre`.
>
> Thanks,
> -tao
>
> -Original Message-
> From: Skalicky, Sam sska...@amazon.com.INVALID> sska...@amazon.com.INVALID<mailto:s
install mxnet --pre`.
Thanks,
-tao
-Original Message-
From: Skalicky, Sam
mailto:sska...@amazon.com.INVALID>mailto:sska...@amazon.com.INVALID>>>
Sent: Monday, January 6, 2020 2:09 AM
To:
dev@mxnet.incubator.apache.org<mailto:dev@mxnet.incubator.apache.org>mailto:dev@mxnet.
t; and our
> > > > > > apparent need for large binaries. Given this fact and that no
> > > objection was
> > > > > > raised by
> > > > > > 2019-12-05 at 05:42 UTC, I conclude we have lazy consensus on
> > > sto
; > >
> > > > > On Jan 5, 2020, at 6:02 PM, Lv, Tao A > > > > tao.a...@intel.com>> wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > How to install the latest available build of a flavor without
> > s
nd go ahead and do this.
> > > >
> > > > Sam
> > > >
> > > > On Jan 5, 2020, at 6:02 PM, Lv, Tao A > > > tao.a...@intel.com>> wrote:
> > > >
> > > > Hi,
> > > >
> > > > How to install the la
> > > the build date? Something like `pip install mxnet --pre`.
> > >
> > > Thanks,
> > > -tao
> > >
> > > -Original Message-
> > > From: Skalicky, Sam > > sska...@amazon.com.INVALID>>
> > > Sent: Monday, Januar
t; > > > > > we
> > > > > > > > > > > > have one
> > > > > > > > > > > > less blocker for the 1.6 release.
> > > > > > > > > > &g
gt; > > > > lose pip as a pretty common distribution channel where
> > > people
> > > > > > > consume
gt; >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://mxnet-dev.s3.amazonaws.com/mxnet_cu101-1.5.1.post0-py2.py3-none-manylinux1_x86_64.whl
> > > > > > > > > > pip install --no-cache-dir
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://mxnet-dev.s3-accelerate.amazonaws.com/mxnet_cu101-1.5.1.post0-py2.py3-none-manylinux1_x86_64.whl
> > > > > > > > > > pip install --no-cache-dir
> > > > > https://d19zq12jzu4w95.cloudfront.net/
> > > > > > > > > >
> mxnet_cu101-1.5.1.post0-py2.py3-none-manylinux1_x86_64.whl
> > > > > > > > > > <
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://d19zq12jzu4w95.cloudfront.net/mxnet_cu101-1.5.1.post0-py2.py3-none-manylinux1_x86_64.whl
> > > > > > > > > >
> > > > > > > > > > <
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://d19zq12jzu4w95.cloudfront.net/mxnet_cu101-1.5.1.post0-py2.py3-none-manylinux1_x86_64.whl
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > where --no-cache-dir prevents caching the downloaded
> file,
> > > for
> > > > > the
> > > > > > > > > purpose
> > > > > > > > > > of
> > > > > > > > > > testing. (cu101 chosen based on large size)
> > > > > > > > > >
> > > > > > > > > > The first URL uses standard S3 bucket in US. The second
> > uses
> > > S3
> > > > > > > > > Accelerate
> > > > > > > > > > based
> > > > > > > > > > on CloudFront CDN. And the third uses CloudFront CDN. I'm
> > > > adding
> > > > > > the
> > > > > > > > > third
> > > > > > > > > > URL,
> > > > > > > > > > as S3 Accelerate may or may not use all new CloudFront
> > > > endpoints
> > > > > > yet.
> > > > > > > > > >
> > > > > > > > > > Regarding voting: Uploading to Pypi is currently
> > impossible,
> > > > > which
> > > > > > > is a
> > > > > > > > > > reality
> > > > > > > > > > (so there is no option to continue as we do currently).
> > Pypi
> > > > > folks
> > > > > > > > > > indicated
> > > > > > > > > > they will unblock our uploads to Pypi once we stop
> > uploading
> > > > > > nightly
> > > > > > > > > > releases
> > > > > > > > > > and taking up 20% of their ressources [1].
> > > > > > > > > >
> > > > > > > > > > If there are any shortcomings or problems identified with
> > > > > uploading
> > > > > > > to
> > > > > > > > > S3,
> > > > > > > > > > we
> > > > > > > > > > can work to address them. But for now, status quo is
> broken
> > > and
> > > > > > this
> > > > > > > > > seems
> > > > > > > > > > the
> > > > > > > > > > only solution addressing Pypi's problem.
> > > > > > > > > >
> > > > > > > > > > I don't mind if you state that you object to lazy
> consensus
> > > and
> > > > > > > start a
> > > > > > > > > > vote. If
> > > > > > > > > > your "maybe [...] start a proper vote" was supposed to be
> > an
> > > > > > > objection
> > > > > > > > to
> > > > > > > > > > lazy
> > > > > > > > > > consensus, please state so clearly (I'm not sure if
> "maybe"
> > > > > > qualifies
> > > > > > > > as
> > > > > > > > > > objection). Though I think it only makes sense with at
> > least
> > > 2
> > > > > > > options
> > > > > > > > to
> > > > > > > > > > vote
> > > > > > > > > > on. Status quo is not a meaningful option, as it is
> already
> > > > > broken.
> > > > > > > > > >
> > > > > > > > > > Best regards
> > > > > > > > > > Leonard
> > > > > > > > > >
> > > > > > > > > > [1]:
> > > > > > > > >
> > > > > >
> > > https://github.com/pypa/pypi-support/issues/50#issuecomment-560479706
> > > > > > > > > >
> > > > > > > > > > On Tue, 2019-12-03 at 19:28 +0100, Marco de Abreu wrote:
> > > > > > > > > > Excellent! Could we maybe come up with a POC and a quick
> > > > writeup
> > > > > > and
> > > > > > > > then
> > > > > > > > > > start a proper vote after everyone verified that it
> covers
> > > > their
> > > > > > > > > > use-cases?
> > > > > > > > > > -Marco
> > > > > > > > > >
> > > > > > > > > > Sheng Zha schrieb am Di., 3. Dez.
> > > 2019,
> > > > > > 19:24:
> > > > > > > > > >
> > > > > > > > > > Yes, there is. We can also make it easier to access by
> > using
> > > a
> > > > > > > > > > geo-location based DNS server so that China users are
> > > directed
> > > > to
> > > > > > > that
> > > > > > > > > > local mirror. The rest of the world is already covered by
> > the
> > > > > > global
> > > > > > > > > > cloudfront.
> > > > > > > > > >
> > > > > > > > > > -sz
> > > > > > > > > >
> > > > > > > > > > On 2019/12/03 18:22:22, Marco de Abreu <
> > > > marco.g.ab...@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > > Isn't there an s3 endpoint in Beijing?
> > > > > > > > > >
> > > > > > > > > > It seems like this topic still warrants some discussion
> and
> > > > thus
> > > > > > I'd
> > > > > > > > > >
> > > > > > > > > > prefer
> > > > > > > > > > if we don't move forward with lazy consensus.
> > > > > > > > > >
> > > > > > > > > > -Marco
> > > > > > > > > >
> > > > > > > > > > Tao Lv schrieb am Di., 3. Dez.
> 2019,
> > > > 14:31:
> > > > > > > > > >
> > > > > > > > > > * For pypi, we can use mirrors.
> > > > > > > > > >
> > > > > > > > > > On Tue, Dec 3, 2019 at 9:28 PM Tao Lv <
> mutou...@gmail.com>
> > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > As we have many users in China, I'm considering the
> > > > > > > > > > accessibility of
> > > > > > > > > > S3.
> > > > > > > > > > For pip, we can mirrors.
> > > > > > > > > >
> > > > > > > > > > On Tue, Dec 3, 2019 at 3:24 PM Lausen, Leonard
> > > > > > > > > >
> > > > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > I would like to remind everyone that lazy consensus is
> > > assumed
> > > > > > > > > > if no
> > > > > > > > > > objections
> > > > > > > > > > are raised before 2019-12-05 at 05:42 UTC. There has been
> > > some
> > > > > > > > > >
> > > > > > > > > > discussion
> > > > > > > > > > about
> > > > > > > > > > the proposal, but to my understanding no objections were
> > > > > > > > > > raised.
> > > > > > > > > > If the proposal is accepted, MXNet releases would be
> > > installed
> > > > > > > > > > via
> > > > > > > > > > pip install mxnet
> > > > > > > > > >
> > > > > > > > > > And release candidates via
> > > > > > > > > >
> > > > > > > > > > pip install --pre mxnet
> > > > > > > > > >
> > > > > > > > > > (or with the respective cuda version specifier appended
> > etc.)
> > > > > > > > > >
> > > > > > > > > > To obtain releases built automatically from the master
> > > branch,
> > > > > > > > > > users
> > > > > > > > > > would need
> > > > > > > > > > to specify something like "-f
> > > > > > > > > > http://mxnet.s3.amazonaws.com/mxnet-X/nightly.html;
> option
> > > to
> > > > > > > > > > pip.
> > > > > > > > > > Best regards
> > > > > > > > > > Leonard
> > > > > > > > > >
> > > > > > > > > > On Mon, 2019-12-02 at 05:42 +, Lausen, Leonard wrote:
> > > > > > > > > > Hi MXNet Community,
> > > > > > > > > >
> > > > > > > > > > since more than 2 months our binary Python nightly
> releases
> > > > > > > > > >
> > > > > > > > > > published
> > > > > > > > > > on Pypi
> > > > > > > > > > are broken. The problem is that our binaries exceed
> Pypi's
> > > > > > > > > > size
> > > > > > > > > > limit.
> > > > > > > > > > Decreasing the binary size by adding compression breaks
> > > > > > > > > >
> > > > > > > > > > third-party
> > > > > > > > > > libraries
> > > > > > > > > > loading libmxnet.so
> > > > > > > > > >
> > > > > > > > > > https://github.com/apache/incubator-mxnet/issues/16193
> > > > > > > > > > Sheng requested Pypi to increase their size limit:
> > > > > > > > > > https://github.com/pypa/pypi-support/issues/50
> > > > > > > > > >
> > > > > > > > > > Currently "the biggest cost for PyPI from [the many MXNet
> > > > > > > > > > binaries
> > > > > > > > > > with
> > > > > > > > > > nightly
> > > > > > > > > > release to Pypi] is the bandwidth consumed when several
> > > > > > > > > > hundred
> > > > > > > > > > mirrors
> > > > > > > > > > attempt
> > > > > > > > > > to mirror each release immediately after it's published".
> > So
> > > > > > > > > > Pypi
> > > > > > > > > > is
> > > > > > > > > > not
> > > > > > > > > > inclined to allow us to upload even larger binaries on a
> > > > > > > > > > nightly
> > > > > > > > > > schedule.
> > > > > > > > > > Their compromise is to allow it on a weekly cadence.
> > > > > > > > > >
> > > > > > > > > > However, I would like the community to revisit the
> > necessity
> > > > > > > > > > of
> > > > > > > > > > releasing pre-
> > > > > > > > > > release binaries to Pypi on a nightly (or weekly)
> cadence.
> > > > > > > > > >
> > > > > > > > > > Instead, we
> > > > > > > > > > can
> > > > > > > > > > release nightly binaries ONLY to a public S3 bucket and
> > > > > > > > > > instruct
> > > > > > > > > > users
> > > > > > > > > > to
> > > > > > > > > > install from there. On our side, we only need to prepare
> a
> > > > > > > > > > html
> > > > > > > > > > document that
> > > > > > > > > > contains links to all released nightly binaries.
> > > > > > > > > > Finally users will install the nightly releases via
> > > > > > > > > >
> > > > > > > > > > pip install --pre mxnet-cu101 -f
> > > > > > > > > >
> > > > > > > > > > http://mxnet.s3.amazonaws.com/mxnet-cu101/
> > > > > > > > > > nightly.html
> > > > > > > > > >
> > > > > > > > > > Instead of
> > > > > > > > > >
> > > > > > > > > > pip install --pre mxnet-cu101
> > > > > > > > > >
> > > > > > > > > > Of course proper releases and release candidates should
> > > > > > > > > > still be
> > > > > > > > > > made
> > > > > > > > > > available
> > > > > > > > > > via Pypi. Thus releases would be installed via
> > > > > > > > > >
> > > > > > > > > > pip install mxnet-cu101
> > > > > > > > > >
> > > > > > > > > > And release candidates via
> > > > > > > > > >
> > > > > > > > > > pip install --pre mxnet-cu101
> > > > > > > > > >
> > > > > > > > > > This will substantially reduce the costs of the Pypi
> > project
> > > > > > > > > > and
> > > > > > > > > > in
> > > > > > > > > > fact
> > > > > > > > > > matches
> > > > > > > > > > the installation experience provided by PyTorch. I don't
> > > > > > > > > > think the
> > > > > > > > > > benefit of
> > > > > > > > > > not including "-f
> > > > > > > > > >
> > > > > > > > > > http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
> > > > > > > > > > matches the costs we currently externalize to the Pypi
> > team.
> > > > > > > > > >
> > > > > > > > > > This suggestion seems uncontroversial to me. Thus I would
> > > > > > > > > > like to
> > > > > > > > > > start
> > > > > > > > > > lazy
> > > > > > > > > > consensus. If there are no objections, I will assume lazy
> > > > > > > > > >
> > > > > > > > > > consensus on
> > > > > > > > > > stopping
> > > > > > > > > > nightly releases to Pypi in 72hrs.
> > > > > > > > > >
> > > > > > > > > > Best regards
> > > > > > > > > > Leonard
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
gt; >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://mxnet-dev.s3.amazonaws.com/mxnet_cu101-1.5.1.post0-py2.py3-none-manylinux1_x86_64.whl
> > > > > > > > > > pip install --no-cache-dir
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://mxnet-dev.s3-accelerate.amazonaws.com/mxnet_cu101-1.5.1.post0-py2.py3-none-manylinux1_x86_64.whl
> > > > > > > > > > pip install --no-cache-dir
> > > > > https://d19zq12jzu4w95.cloudfront.net/
> > > > > > > > > >
> mxnet_cu101-1.5.1.post0-py2.py3-none-manylinux1_x86_64.whl
> > > > > > > > > > <
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://d19zq12jzu4w95.cloudfront.net/mxnet_cu101-1.5.1.post0-py2.py3-none-manylinux1_x86_64.whl
> > > > > > > > > >
> > > > > > > > > > <
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://d19zq12jzu4w95.cloudfront.net/mxnet_cu101-1.5.1.post0-py2.py3-none-manylinux1_x86_64.whl
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > where --no-cache-dir prevents caching the downloaded
> file,
> > > for
> > > > > the
> > > > > > > > > purpose
> > > > > > > > > > of
> > > > > > > > > > testing. (cu101 chosen based on large size)
> > > > > > > > > >
> > > > > > > > > > The first URL uses standard S3 bucket in US. The second
> > uses
> > > S3
> > > > > > > > > Accelerate
> > > > > > > > > > based
> > > > > > > > > > on CloudFront CDN. And the third uses CloudFront CDN. I'm
> > > > adding
> > > > > > the
> > > > > > > > > third
> > > > > > > > > > URL,
> > > > > > > > > > as S3 Accelerate may or may not use all new CloudFront
> > > > endpoints
> > > > > > yet.
> > > > > > > > > >
> > > > > > > > > > Regarding voting: Uploading to Pypi is currently
> > impossible,
> > > > > which
> > > > > > > is a
> > > > > > > > > > reality
> > > > > > > > > > (so there is no option to continue as we do currently).
> > Pypi
> > > > > folks
> > > > > > > > > > indicated
> > > > > > > > > > they will unblock our uploads to Pypi once we stop
> > uploading
> > > > > > nightly
> > > > > > > > > > releases
> > > > > > > > > > and taking up 20% of their ressources [1].
> > > > > > > > > >
> > > > > > > > > > If there are any shortcomings or problems identified with
> > > > > uploading
> > > > > > > to
> > > > > > > > > S3,
> > > > > > > > > > we
> > > > > > > > > > can work to address them. But for now, status quo is
> broken
> > > and
> > > > > > this
> > > > > > > > > seems
> > > > > > > > > > the
> > > > > > > > > > only solution addressing Pypi's problem.
> > > > > > > > > >
> > > > > > > > > > I don't mind if you state that you object to lazy
> consensus
> > > and
> > > > > > > start a
> > > > > > > > > > vote. If
> > > > > > > > > > your "maybe [...] start a proper vote" was supposed to be
> > an
> > > > > > > objection
> > > > > > > > to
> > > > > > > > > > lazy
> > > > > > > > > > consensus, please state so clearly (I'm not sure if
> "maybe"
> > > > > > qualifies
> > > > > > > > as
> > > > > > > > > > objection). Though I think it only makes sense with at
> > least
> > > 2
> > > > > > > options
> > > > > > > > to
> > > > > > > > > > vote
> > > > > > > > > > on. Status quo is not a meaningful option, as it is
> already
> > > > > broken.
> > > > > > > > > >
> > > > > > > > > > Best regards
> > > > > > > > > > Leonard
> > > > > > > > > >
> > > > > > > > > > [1]:
> > > > > > > > >
> > > > > >
> > > https://github.com/pypa/pypi-support/issues/50#issuecomment-560479706
> > > > > > > > > >
> > > > > > > > > > On Tue, 2019-12-03 at 19:28 +0100, Marco de Abreu wrote:
> > > > > > > > > > Excellent! Could we maybe come up with a POC and a quick
> > > > writeup
> > > > > > and
> > > > > > > > then
> > > > > > > > > > start a proper vote after everyone verified that it
> covers
> > > > their
> > > > > > > > > > use-cases?
> > > > > > > > > > -Marco
> > > > > > > > > >
> > > > > > > > > > Sheng Zha schrieb am Di., 3. Dez.
> > > 2019,
> > > > > > 19:24:
> > > > > > > > > >
> > > > > > > > > > Yes, there is. We can also make it easier to access by
> > using
> > > a
> > > > > > > > > > geo-location based DNS server so that China users are
> > > directed
> > > > to
> > > > > > > that
> > > > > > > > > > local mirror. The rest of the world is already covered by
> > the
> > > > > > global
> > > > > > > > > > cloudfront.
> > > > > > > > > >
> > > > > > > > > > -sz
> > > > > > > > > >
> > > > > > > > > > On 2019/12/03 18:22:22, Marco de Abreu <
> > > > marco.g.ab...@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > > Isn't there an s3 endpoint in Beijing?
> > > > > > > > > >
> > > > > > > > > > It seems like this topic still warrants some discussion
> and
> > > > thus
> > > > > > I'd
> > > > > > > > > >
> > > > > > > > > > prefer
> > > > > > > > > > if we don't move forward with lazy consensus.
> > > > > > > > > >
> > > > > > > > > > -Marco
> > > > > > > > > >
> > > > > > > > > > Tao Lv schrieb am Di., 3. Dez.
> 2019,
> > > > 14:31:
> > > > > > > > > >
> > > > > > > > > > * For pypi, we can use mirrors.
> > > > > > > > > >
> > > > > > > > > > On Tue, Dec 3, 2019 at 9:28 PM Tao Lv <
> mutou...@gmail.com>
> > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > As we have many users in China, I'm considering the
> > > > > > > > > > accessibility of
> > > > > > > > > > S3.
> > > > > > > > > > For pip, we can mirrors.
> > > > > > > > > >
> > > > > > > > > > On Tue, Dec 3, 2019 at 3:24 PM Lausen, Leonard
> > > > > > > > > >
> > > > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > I would like to remind everyone that lazy consensus is
> > > assumed
> > > > > > > > > > if no
> > > > > > > > > > objections
> > > > > > > > > > are raised before 2019-12-05 at 05:42 UTC. There has been
> > > some
> > > > > > > > > >
> > > > > > > > > > discussion
> > > > > > > > > > about
> > > > > > > > > > the proposal, but to my understanding no objections were
> > > > > > > > > > raised.
> > > > > > > > > > If the proposal is accepted, MXNet releases would be
> > > installed
> > > > > > > > > > via
> > > > > > > > > > pip install mxnet
> > > > > > > > > >
> > > > > > > > > > And release candidates via
> > > > > > > > > >
> > > > > > > > > > pip install --pre mxnet
> > > > > > > > > >
> > > > > > > > > > (or with the respective cuda version specifier appended
> > etc.)
> > > > > > > > > >
> > > > > > > > > > To obtain releases built automatically from the master
> > > branch,
> > > > > > > > > > users
> > > > > > > > > > would need
> > > > > > > > > > to specify something like "-f
> > > > > > > > > > http://mxnet.s3.amazonaws.com/mxnet-X/nightly.html;
> option
> > > to
> > > > > > > > > > pip.
> > > > > > > > > > Best regards
> > > > > > > > > > Leonard
> > > > > > > > > >
> > > > > > > > > > On Mon, 2019-12-02 at 05:42 +, Lausen, Leonard wrote:
> > > > > > > > > > Hi MXNet Community,
> > > > > > > > > >
> > > > > > > > > > since more than 2 months our binary Python nightly
> releases
> > > > > > > > > >
> > > > > > > > > > published
> > > > > > > > > > on Pypi
> > > > > > > > > > are broken. The problem is that our binaries exceed
> Pypi's
> > > > > > > > > > size
> > > > > > > > > > limit.
> > > > > > > > > > Decreasing the binary size by adding compression breaks
> > > > > > > > > >
> > > > > > > > > > third-party
> > > > > > > > > > libraries
> > > > > > > > > > loading libmxnet.so
> > > > > > > > > >
> > > > > > > > > > https://github.com/apache/incubator-mxnet/issues/16193
> > > > > > > > > > Sheng requested Pypi to increase their size limit:
> > > > > > > > > > https://github.com/pypa/pypi-support/issues/50
> > > > > > > > > >
> > > > > > > > > > Currently "the biggest cost for PyPI from [the many MXNet
> > > > > > > > > > binaries
> > > > > > > > > > with
> > > > > > > > > > nightly
> > > > > > > > > > release to Pypi] is the bandwidth consumed when several
> > > > > > > > > > hundred
> > > > > > > > > > mirrors
> > > > > > > > > > attempt
> > > > > > > > > > to mirror each release immediately after it's published".
> > So
> > > > > > > > > > Pypi
> > > > > > > > > > is
> > > > > > > > > > not
> > > > > > > > > > inclined to allow us to upload even larger binaries on a
> > > > > > > > > > nightly
> > > > > > > > > > schedule.
> > > > > > > > > > Their compromise is to allow it on a weekly cadence.
> > > > > > > > > >
> > > > > > > > > > However, I would like the community to revisit the
> > necessity
> > > > > > > > > > of
> > > > > > > > > > releasing pre-
> > > > > > > > > > release binaries to Pypi on a nightly (or weekly)
> cadence.
> > > > > > > > > >
> > > > > > > > > > Instead, we
> > > > > > > > > > can
> > > > > > > > > > release nightly binaries ONLY to a public S3 bucket and
> > > > > > > > > > instruct
> > > > > > > > > > users
> > > > > > > > > > to
> > > > > > > > > > install from there. On our side, we only need to prepare
> a
> > > > > > > > > > html
> > > > > > > > > > document that
> > > > > > > > > > contains links to all released nightly binaries.
> > > > > > > > > > Finally users will install the nightly releases via
> > > > > > > > > >
> > > > > > > > > > pip install --pre mxnet-cu101 -f
> > > > > > > > > >
> > > > > > > > > > http://mxnet.s3.amazonaws.com/mxnet-cu101/
> > > > > > > > > > nightly.html
> > > > > > > > > >
> > > > > > > > > > Instead of
> > > > > > > > > >
> > > > > > > > > > pip install --pre mxnet-cu101
> > > > > > > > > >
> > > > > > > > > > Of course proper releases and release candidates should
> > > > > > > > > > still be
> > > > > > > > > > made
> > > > > > > > > > available
> > > > > > > > > > via Pypi. Thus releases would be installed via
> > > > > > > > > >
> > > > > > > > > > pip install mxnet-cu101
> > > > > > > > > >
> > > > > > > > > > And release candidates via
> > > > > > > > > >
> > > > > > > > > > pip install --pre mxnet-cu101
> > > > > > > > > >
> > > > > > > > > > This will substantially reduce the costs of the Pypi
> > project
> > > > > > > > > > and
> > > > > > > > > > in
> > > > > > > > > > fact
> > > > > > > > > > matches
> > > > > > > > > > the installation experience provided by PyTorch. I don't
> > > > > > > > > > think the
> > > > > > > > > > benefit of
> > > > > > > > > > not including "-f
> > > > > > > > > >
> > > > > > > > > > http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
> > > > > > > > > > matches the costs we currently externalize to the Pypi
> > team.
> > > > > > > > > >
> > > > > > > > > > This suggestion seems uncontroversial to me. Thus I would
> > > > > > > > > > like to
> > > > > > > > > > start
> > > > > > > > > > lazy
> > > > > > > > > > consensus. If there are no objections, I will assume lazy
> > > > > > > > > >
> > > > > > > > > > consensus on
> > > > > > > > > > stopping
> > > > > > > > > > nightly releases to Pypi in 72hrs.
> > > > > > > > > >
> > > > > > > > > > Best regards
> > > > > > > > > > Leonard
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
> > > >
> > > > > > > > > > pip install --no-cache-dir
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
&
> >
> > > >
> > >
> >
> https://d19zq12jzu4w95.cloudfront.net/mxnet_cu101-1.5.1.post0-py2.py3-none-manylinux1_x86_64.whl
> > > > > > > > >
> > > > > > > > > <
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://d19zq12jzu4w95.cloudfront.net/mxnet_cu101-1.5.1.post0-py2.py3-none-manylinux1_x86_64.whl
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > where --no-cache-dir prevents caching the downloaded file,
> > for
> > > > the
> > > > > > > > purpose
> > > > > > > > > of
> > > > > > > > > testing. (cu101 chosen based on large size)
> > > > > > > > >
> > > > > > > > > The first URL uses standard S3 bucket in US. The second
> uses
> > S3
> > > > > > > > Accelerate
> > > > > > > > > based
> > > > > > > > > on CloudFront CDN. And the third uses CloudFront CDN. I'm
> > > adding
> > > > > the
> > > > > > > > third
> > > > > > > > > URL,
> > > > > > > > > as S3 Accelerate may or may not use all new CloudFront
> > > endpoints
> > > > > yet.
> > > > > > > > >
> > > > > > > > > Regarding voting: Uploading to Pypi is currently
> impossible,
> > > > which
> > > > > > is a
> > > > > > > > > reality
> > > > > > > > > (so there is no option to continue as we do currently).
> Pypi
> > > > folks
> > > > > > > > > indicated
> > > > > > > > > they will unblock our uploads to Pypi once we stop
> uploading
> > > > > nightly
> > > > > > > > > releases
> > > > > > > > > and taking up 20% of their ressources [1].
> > > > > > > > >
> > > > > > > > > If there are any shortcomings or problems identified with
> > > > uploading
> > > > > > to
> > > > > > > > S3,
> > > > > > > > > we
> > > > > > > > > can work to address them. But for now, status quo is broken
> > and
> > > > > this
> > > > > > > > seems
> > > > > > > > > the
> > > > > > > > > only solution addressing Pypi's problem.
> > > > > > > > >
> > > > > > > > > I don't mind if you state that you object to lazy consensus
> > and
> > > > > > start a
> > > > > > > > > vote. If
> > > > > > > > > your "maybe [...] start a proper vote" was supposed to be
> an
> > > > > > objection
> > > > > > > to
> > > > > > > > > lazy
> > > > > > > > > consensus, please state so clearly (I'm not sure if "maybe"
> > > > > qualifies
> > > > > > > as
> > > > > > > > > objection). Though I think it only makes sense with at
> least
> > 2
> > > > > > options
> > > > > > > to
> > > > > > > > > vote
> > > > > > > > > on. Status quo is not a meaningful option, as it is already
> > > > broken.
> > > > > > > > >
> > > > > > > > > Best regards
> > > > > > > > > Leonard
> > > > > > > > >
> > > > > > > > > [1]:
> > > > > > > >
> > > > >
> > https://github.com/pypa/pypi-support/issues/50#issuecomment-560479706
> > > > > > > > >
> > > > > > > > > On Tue, 2019-12-03 at 19:28 +0100, Marco de Abreu wrote:
> > > > > > > > > Excellent! Could we maybe come up with a POC and a quick
> > > writeup
> > > > > and
> > > > > > > then
> > > > > > > > > start a proper vote after everyone verified that it covers
> > > their
> > > > > > > > > use-cases?
> > > > > > > > > -Marco
> > > > > > > > >
> > > > > > > > > Sheng Zha schrieb am Di., 3. Dez.
> > 2019,
> > > > > 19:24:
> > > > > > > > >
> > > > > > > > > Yes, there is. We can also make it easier to access by
> using
> > a
> > > > > > > > > geo-location based DNS server so that China users are
> > directed
> > > to
> > > > > > that
> > > > > > > > > local mirror. The rest of the world is already covered by
> the
> > > > > global
> > > > > > > > > cloudfront.
> > > > > > > > >
> > > > > > > > > -sz
> > > > > > > > >
> > > > > > > > > On 2019/12/03 18:22:22, Marco de Abreu <
> > > marco.g.ab...@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > > Isn't there an s3 endpoint in Beijing?
> > > > > > > > >
> > > > > > > > > It seems like this topic still warrants some discussion and
> > > thus
> > > > > I'd
> > > > > > > > >
> > > > > > > > > prefer
> > > > > > > > > if we don't move forward with lazy consensus.
> > > > > > > > >
> > > > > > > > > -Marco
> > > > > > > > >
> > > > > > > > > Tao Lv schrieb am Di., 3. Dez. 2019,
> > > 14:31:
> > > > > > > > >
> > > > > > > > > * For pypi, we can use mirrors.
> > > > > > > > >
> > > > > > > > > On Tue, Dec 3, 2019 at 9:28 PM Tao Lv
> > > > wrote:
> > > > > > > > >
> > > > > > > > > As we have many users in China, I'm considering the
> > > > > > > > > accessibility of
> > > > > > > > > S3.
> > > > > > > > > For pip, we can mirrors.
> > > > > > > > >
> > > > > > > > > On Tue, Dec 3, 2019 at 3:24 PM Lausen, Leonard
> > > > > > > > >
> > > > > > > > > > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > I would like to remind everyone that lazy consensus is
> > assumed
> > > > > > > > > if no
> > > > > > > > > objections
> > > > > > > > > are raised before 2019-12-05 at 05:42 UTC. There has been
> > some
> > > > > > > > >
> > > > > > > > > discussion
> > > > > > > > > about
> > > > > > > > > the proposal, but to my understanding no objections were
> > > > > > > > > raised.
> > > > > > > > > If the proposal is accepted, MXNet releases would be
> > installed
> > > > > > > > > via
> > > > > > > > > pip install mxnet
> > > > > > > > >
> > > > > > > > > And release candidates via
> > > > > > > > >
> > > > > > > > > pip install --pre mxnet
> > > > > > > > >
> > > > > > > > > (or with the respective cuda version specifier appended
> etc.)
> > > > > > > > >
> > > > > > > > > To obtain releases built automatically from the master
> > branch,
> > > > > > > > > users
> > > > > > > > > would need
> > > > > > > > > to specify something like "-f
> > > > > > > > > http://mxnet.s3.amazonaws.com/mxnet-X/nightly.html; option
> > to
> > > > > > > > > pip.
> > > > > > > > > Best regards
> > > > > > > > > Leonard
> > > > > > > > >
> > > > > > > > > On Mon, 2019-12-02 at 05:42 +, Lausen, Leonard wrote:
> > > > > > > > > Hi MXNet Community,
> > > > > > > > >
> > > > > > > > > since more than 2 months our binary Python nightly releases
> > > > > > > > >
> > > > > > > > > published
> > > > > > > > > on Pypi
> > > > > > > > > are broken. The problem is that our binaries exceed Pypi's
> > > > > > > > > size
> > > > > > > > > limit.
> > > > > > > > > Decreasing the binary size by adding compression breaks
> > > > > > > > >
> > > > > > > > > third-party
> > > > > > > > > libraries
> > > > > > > > > loading libmxnet.so
> > > > > > > > >
> > > > > > > > > https://github.com/apache/incubator-mxnet/issues/16193
> > > > > > > > > Sheng requested Pypi to increase their size limit:
> > > > > > > > > https://github.com/pypa/pypi-support/issues/50
> > > > > > > > >
> > > > > > > > > Currently "the biggest cost for PyPI from [the many MXNet
> > > > > > > > > binaries
> > > > > > > > > with
> > > > > > > > > nightly
> > > > > > > > > release to Pypi] is the bandwidth consumed when several
> > > > > > > > > hundred
> > > > > > > > > mirrors
> > > > > > > > > attempt
> > > > > > > > > to mirror each release immediately after it's published".
> So
> > > > > > > > > Pypi
> > > > > > > > > is
> > > > > > > > > not
> > > > > > > > > inclined to allow us to upload even larger binaries on a
> > > > > > > > > nightly
> > > > > > > > > schedule.
> > > > > > > > > Their compromise is to allow it on a weekly cadence.
> > > > > > > > >
> > > > > > > > > However, I would like the community to revisit the
> necessity
> > > > > > > > > of
> > > > > > > > > releasing pre-
> > > > > > > > > release binaries to Pypi on a nightly (or weekly) cadence.
> > > > > > > > >
> > > > > > > > > Instead, we
> > > > > > > > > can
> > > > > > > > > release nightly binaries ONLY to a public S3 bucket and
> > > > > > > > > instruct
> > > > > > > > > users
> > > > > > > > > to
> > > > > > > > > install from there. On our side, we only need to prepare a
> > > > > > > > > html
> > > > > > > > > document that
> > > > > > > > > contains links to all released nightly binaries.
> > > > > > > > > Finally users will install the nightly releases via
> > > > > > > > >
> > > > > > > > > pip install --pre mxnet-cu101 -f
> > > > > > > > >
> > > > > > > > > http://mxnet.s3.amazonaws.com/mxnet-cu101/
> > > > > > > > > nightly.html
> > > > > > > > >
> > > > > > > > > Instead of
> > > > > > > > >
> > > > > > > > > pip install --pre mxnet-cu101
> > > > > > > > >
> > > > > > > > > Of course proper releases and release candidates should
> > > > > > > > > still be
> > > > > > > > > made
> > > > > > > > > available
> > > > > > > > > via Pypi. Thus releases would be installed via
> > > > > > > > >
> > > > > > > > > pip install mxnet-cu101
> > > > > > > > >
> > > > > > > > > And release candidates via
> > > > > > > > >
> > > > > > > > > pip install --pre mxnet-cu101
> > > > > > > > >
> > > > > > > > > This will substantially reduce the costs of the Pypi
> project
> > > > > > > > > and
> > > > > > > > > in
> > > > > > > > > fact
> > > > > > > > > matches
> > > > > > > > > the installation experience provided by PyTorch. I don't
> > > > > > > > > think the
> > > > > > > > > benefit of
> > > > > > > > > not including "-f
> > > > > > > > >
> > > > > > > > > http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
> > > > > > > > > matches the costs we currently externalize to the Pypi
> team.
> > > > > > > > >
> > > > > > > > > This suggestion seems uncontroversial to me. Thus I would
> > > > > > > > > like to
> > > > > > > > > start
> > > > > > > > > lazy
> > > > > > > > > consensus. If there are no objections, I will assume lazy
> > > > > > > > >
> > > > > > > > > consensus on
> > > > > > > > > stopping
> > > > > > > > > nightly releases to Pypi in 72hrs.
> > > > > > > > >
> > > > > > > > > Best regards
> > > > > > > > > Leonard
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
The first URL uses standard S3 bucket in US. The second uses
> S3
> > > > > > > Accelerate
> > > > > > > > based
> > > > > > > > on CloudFront CDN. And the third uses CloudFront CDN. I'm
> > adding
> > > > the
> > > > >
eases, but higher cost due to 500MB ->
>> >>> 800MB limit increase. Assuming that the limit increase translates into
>> >>> actually larger binaries.
>> >>>
>> >>>
>> >>> On Wed, 2019-12-04 at 22:20 +0100, Marco de Abreu wrote:
>> >>> Are weekly releases an option? It was brought up as concern that we
>> might
>> >>> lose pip as a pretty common distribution channel where people consume
>> >>> nightly builds. I don't feel like that concern has been properly
>> addressed
>> >>> so far.
>> >>>
>> >>> -Marco
>> >>>
>> >>> Lausen, Leonard > >>> lau...@amazon.com.invalid>> >>> lau...@amazon.com.invalid<mailto:lau...@amazon.com.invalid>>>
>> schrieb am
>> >>> Mi., 4. Dez. 2019,
>> >>> 04:09:
>> >>>
>> >>> As a simple POC to test distribution, you can try installing MXNet
>> based
>> >>> on these 3 URLs:
>> >>>
>> >>> pip install --no-cache-dir
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> https://mxnet-dev.s3.amazonaws.com/mxnet_cu101-1.5.1.post0-py2.py3-none-manylinux1_x86_64.whl
>> >>> pip install --no-cache-dir
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> https://mxnet-dev.s3-accelerate.amazonaws.com/mxnet_cu101-1.5.1.post0-py2.py3-none-manylinux1_x86_64.whl
>> >>> pip install --no-cache-dir
>> >>> https://d19zq12jzu4w95.cloudfront.net/
>> >>> mxnet_cu101-1.5.1.post0-py2.py3-none-manylinux1_x86_64.whl
>> >>> <
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> https://d19zq12jzu4w95.cloudfront.net/mxnet_cu101-1.5.1.post0-py2.py3-none-manylinux1_x86_64.whl
>> >>>
>> >>> <
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> https://d19zq12jzu4w95.cloudfront.net/mxnet_cu101-1.5.1.post0-py2.py3-none-manylinux1_x86_64.whl
>> >>>
>> >>>
>> >>> where --no-cache-dir prevents caching the downloaded file, for the
>> purpose
>> >>> of testing. (cu101 chosen based on large size)
>> >>>
>> >>> The first URL uses standard S3 bucket in US. The second uses
>> >>> S3
>> >>> Accelerate
>> >>> based
>> >>> on CloudFront CDN. And the third uses CloudFront CDN. I'm adding the
>> third
>> >>> URL, as S3 Accelerate may or may not use all new CloudFront endpoints
>> yet.
>> >>>
>> >>> Regarding voting: Uploading to Pypi is currently impossible, which is
>> a
>> >>> reality (so there is no option to continue as we do currently). Pypi
>> folks
>> >>> indicated they will unblock our uploads to Pypi once we stop uploading
>> >>> nightly releases and taking up 20% of their ressources [1].
>> >>>
>> >>> If there are any shortcomings or problems identified with uploading
>> to S3,
>> >>> we can work to address them. But for now, status quo is broken and
>> this
>> >>> seems the only solution addressing Pypi's problem.
>> >>>
>> >>> I don't mind if you state that you object to lazy consensus and start
>> a
>> >>> vote. If your "maybe [...] start a proper vote" was supposed to be an
>> >>> objection to lazy consensus, please state so clearly (I'm not sure if
>> >>> "maybe"
>> >>> qualifies
>> >>> as
>> >>> objection). Though I think it only makes sense with at least 2
>> options to
>> >>> vote on. Status quo is not a meaningful option, as it is already
>> broken.
>> >>>
>> >>> Best regards
>> >>> Leonard
>> >>>
>> >>> [1]:
>> >>>
>> >>> https://github.com/pypa/pypi-support/issues/50#issuecomment-560479706
>> >>>
>> >>> On Tue, 2019-12-03 at 19:28 +0100, Marco de Abreu wrote:
>> >>> Excellent! Could we maybe come up with a POC and a quick writeup and
>> then
>> >>> start a proper vote after everyone verified that it covers their
>> use-cases?
>> >>> -Marco
>> >>>
>> >>> Sheng Zha mailto:zhash...@apache.org>> schrieb
>> am
>> >>> Di., 3. Dez. 2019,
>> >>> 19:24:
>> >>>
>> >>> Yes, there is. We can also make it easier to access by using a
>> >>> geo-location based DNS server so that China users are directed to that
>> >>> local mirror. The rest of the world is already covered by the global
>> >>> cloudfront.
>> >>>
>> >>> -sz
>> >>>
>> >>> On 2019/12/03 18:22:22, Marco de Abreu < marco.g.ab...@gmail.com
>> > >>> marco.g.ab...@gmail.com>
>> >>>
>> >>> wrote:
>> >>> Isn't there an s3 endpoint in Beijing?
>> >>>
>> >>> It seems like this topic still warrants some discussion and thus I'd
>> >>>
>> >>> prefer
>> >>> if we don't move forward with lazy consensus.
>> >>>
>> >>> -Marco
>> >>>
>> >>> Tao Lv mailto:mutou...@gmail.com>> schrieb am
>> Di., 3.
>> >>> Dez. 2019,
>> >>> 14:31:
>> >>>
>> >>> * For pypi, we can use mirrors.
>> >>>
>> >>> On Tue, Dec 3, 2019 at 9:28 PM Tao Lv > >>> mutou...@gmail.com>>
>> >>> wrote:
>> >>>
>> >>> As we have many users in China, I'm considering the accessibility of
>> S3.
>> >>> For pip, we can mirrors.
>> >>>
>> >>> On Tue, Dec 3, 2019 at 3:24 PM Lausen, Leonard
>> >>>
>> >>> mailto:lau...@amazon.com.invalid>
>> >>> wrote:
>> >>>
>> >>> I would like to remind everyone that lazy consensus is assumed if no
>> >>> objections are raised before 2019-12-05 at 05:42 UTC. There has been
>> some
>> >>>
>> >>> discussion
>> >>> about
>> >>> the proposal, but to my understanding no objections were raised.
>> >>> If the proposal is accepted, MXNet releases would be installed via
>> pip
>> >>> install mxnet
>> >>>
>> >>> And release candidates via
>> >>>
>> >>> pip install --pre mxnet
>> >>>
>> >>> (or with the respective cuda version specifier appended etc.)
>> >>>
>> >>> To obtain releases built automatically from the master branch, users
>> would
>> >>> need to specify something like "-f
>> >>> http://mxnet.s3.amazonaws.com/mxnet-X/nightly.html; option to pip.
>> >>> Best regards
>> >>> Leonard
>> >>>
>> >>> On Mon, 2019-12-02 at 05:42 +, Lausen, Leonard wrote:
>> >>> Hi MXNet Community,
>> >>>
>> >>> since more than 2 months our binary Python nightly releases
>> >>>
>> >>> published
>> >>> on Pypi
>> >>> are broken. The problem is that our binaries exceed Pypi's size limit.
>> >>> Decreasing the binary size by adding compression breaks
>> >>>
>> >>> third-party
>> >>> libraries
>> >>> loading libmxnet.so
>> >>>
>> >>> https://github.com/apache/incubator-mxnet/issues/16193
>> >>> Sheng requested Pypi to increase their size limit:
>> >>> https://github.com/pypa/pypi-support/issues/50
>> >>>
>> >>> Currently "the biggest cost for PyPI from [the many MXNet binaries
>> with
>> >>> nightly release to Pypi] is the bandwidth consumed when several
>> hundred
>> >>> mirrors attempt to mirror each release immediately after it's
>> published".
>> >>> So Pypi is not inclined to allow us to upload even larger binaries on
>> a
>> >>> nightly schedule.
>> >>> Their compromise is to allow it on a weekly cadence.
>> >>>
>> >>> However, I would like the community to revisit the necessity of
>> releasing
>> >>> pre- release binaries to Pypi on a nightly (or weekly) cadence.
>> >>>
>> >>> Instead, we
>> >>> can
>> >>> release nightly binaries ONLY to a public S3 bucket and instruct
>> users to
>> >>> install from there. On our side, we only need to prepare a html
>> document
>> >>> that contains links to all released nightly binaries.
>> >>> Finally users will install the nightly releases via
>> >>>
>> >>> pip install --pre mxnet-cu101 -f
>> >>>
>> >>> http://mxnet.s3.amazonaws.com/mxnet-cu101/
>> >>> nightly.html
>> >>>
>> >>> Instead of
>> >>>
>> >>> pip install --pre mxnet-cu101
>> >>>
>> >>> Of course proper releases and release candidates should still be made
>> >>> available via Pypi. Thus releases would be installed via
>> >>>
>> >>> pip install mxnet-cu101
>> >>>
>> >>> And release candidates via
>> >>>
>> >>> pip install --pre mxnet-cu101
>> >>>
>> >>> This will substantially reduce the costs of the Pypi project and in
>> fact
>> >>> matches the installation experience provided by PyTorch. I don't
>> think the
>> >>> benefit of not including "-f
>> >>>
>> >>> http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
>> >>> matches the costs we currently externalize to the Pypi team.
>> >>>
>> >>> This suggestion seems uncontroversial to me. Thus I would like to
>> start
>> >>> lazy consensus. If there are no objections, I will assume lazy
>> >>>
>> >>> consensus on
>> >>> stopping
>> >>> nightly releases to Pypi in 72hrs.
>> >>>
>> >>> Best regards
>> >>> Leonard
>> >>>
>> >>>
>>
>>
t 6:02 PM, Lv, Tao A >>> tao.a...@intel.com>> wrote:
> >>>
> >>> Hi,
> >>>
> >>> How to install the latest available build of a flavor without
> specifying
> >>> the build date? Something like `pip install mxnet -
t;>> How to install the latest available build of a flavor without specifying
>>> the build date? Something like `pip install mxnet --pre`.
>>>
>>> Thanks,
>>> -tao
>>>
>>> -Original Message-
>>> From: Skalicky, Sam >&
nal Message-
> From: Skalicky, Sam sska...@amazon.com.INVALID>>
> Sent: Monday, January 6, 2020 2:09 AM
> To: dev@mxnet.incubator.apache.org<mailto:dev@mxnet.incubator.apache.org>
> Subject: Re: Stopping nightly releases to Pypi
>
> Hi Haibin,
>
> You typed th
y, Sam
mailto:sska...@amazon.com.INVALID>>
Sent: Monday, January 6, 2020 2:09 AM
To: dev@mxnet.incubator.apache.org<mailto:dev@mxnet.incubator.apache.org>
Subject: Re: Stopping nightly releases to Pypi
Hi Haibin,
You typed the correct URLs, the cu100 build has been failing since December
30t
azonaws.com/mxnet-cu101/nightly.html;
matches the costs we currently externalize to the Pypi team.
This suggestion seems uncontroversial to me. Thus I would like to start lazy
consensus. If there are no objections, I will assume lazy
consensus on
stopping
nightly releases to Pypi in 72hrs.
Best regards
Leonard
rsial to me. Thus I would
like to
start
lazy
consensus. If there are no objections, I will assume lazy
consensus on
stopping
nightly releases to Pypi in 72hrs.
Best regards
Leonard
is broken and
> > > this
> > > > > > seems
> > > > > > > the
> > > > > > > only solution addressing Pypi's problem.
> > > > > > >
> > > > > > > I don't mind if you state that you object to lazy consensus and
> > > > start a
> > > > > > > vote. If
> > > > > > > your "maybe [...] start a proper vote" was supposed to be an
> > > > objection
> > > > > to
> > > > > > > lazy
> > > > > > > consensus, please state so clearly (I'm not sure if "maybe"
> > > qualifies
> > > > > as
> > > > > > > objection). Though I think it only makes sense with at least 2
> > > > options
> > > > > to
> > > > > > > vote
> > > > > > > on. Status quo is not a meaningful option, as it is already
> > broken.
> > > > > > >
> > > > > > > Best regards
> > > > > > > Leonard
> > > > > > >
> > > > > > > [1]:
> > > > > >
> > > https://github.com/pypa/pypi-support/issues/50#issuecomment-560479706
> > > > > > >
> > > > > > > On Tue, 2019-12-03 at 19:28 +0100, Marco de Abreu wrote:
> > > > > > > Excellent! Could we maybe come up with a POC and a quick
> writeup
> > > and
> > > > > then
> > > > > > > start a proper vote after everyone verified that it covers
> their
> > > > > > > use-cases?
> > > > > > > -Marco
> > > > > > >
> > > > > > > Sheng Zha schrieb am Di., 3. Dez. 2019,
> > > 19:24:
> > > > > > >
> > > > > > > Yes, there is. We can also make it easier to access by using a
> > > > > > > geo-location based DNS server so that China users are directed
> to
> > > > that
> > > > > > > local mirror. The rest of the world is already covered by the
> > > global
> > > > > > > cloudfront.
> > > > > > >
> > > > > > > -sz
> > > > > > >
> > > > > > > On 2019/12/03 18:22:22, Marco de Abreu <
> marco.g.ab...@gmail.com>
> > > > > > > wrote:
> > > > > > > Isn't there an s3 endpoint in Beijing?
> > > > > > >
> > > > > > > It seems like this topic still warrants some discussion and
> thus
> > > I'd
> > > > > > >
> > > > > > > prefer
> > > > > > > if we don't move forward with lazy consensus.
> > > > > > >
> > > > > > > -Marco
> > > > > > >
> > > > > > > Tao Lv schrieb am Di., 3. Dez. 2019,
> 14:31:
> > > > > > >
> > > > > > > * For pypi, we can use mirrors.
> > > > > > >
> > > > > > > On Tue, Dec 3, 2019 at 9:28 PM Tao Lv
> > wrote:
> > > > > > >
> > > > > > > As we have many users in China, I'm considering the
> > > > > > > accessibility of
> > > > > > > S3.
> > > > > > > For pip, we can mirrors.
> > > > > > >
> > > > > > > On Tue, Dec 3, 2019 at 3:24 PM Lausen, Leonard
> > > > > > >
> > > > > > > > > > > > > wrote:
> > > > > > >
> > > > > > > I would like to remind everyone that lazy consensus is assumed
> > > > > > > if no
> > > > > > > objections
> > > > > > > are raised before 2019-12-05 at 05:42 UTC. There has been some
> > > > > > >
> > > > > > > discussion
> > > > > > > about
> > > > > > > the proposal, but to my understanding no objections were
> > > > > > > raised.
> > > > > > > If the proposal is accepted, MXNet releases would be installed
> > > > > > > via
> > > > > > > pip install mxnet
> > > > > > >
> > > > > > > And release candidates via
> > > > > > >
> > > > > > > pip install --pre mxnet
> > > > > > >
> > > > > > > (or with the respective cuda version specifier appended etc.)
> > > > > > >
> > > > > > > To obtain releases built automatically from the master branch,
> > > > > > > users
> > > > > > > would need
> > > > > > > to specify something like "-f
> > > > > > > http://mxnet.s3.amazonaws.com/mxnet-X/nightly.html; option to
> > > > > > > pip.
> > > > > > > Best regards
> > > > > > > Leonard
> > > > > > >
> > > > > > > On Mon, 2019-12-02 at 05:42 +, Lausen, Leonard wrote:
> > > > > > > Hi MXNet Community,
> > > > > > >
> > > > > > > since more than 2 months our binary Python nightly releases
> > > > > > >
> > > > > > > published
> > > > > > > on Pypi
> > > > > > > are broken. The problem is that our binaries exceed Pypi's
> > > > > > > size
> > > > > > > limit.
> > > > > > > Decreasing the binary size by adding compression breaks
> > > > > > >
> > > > > > > third-party
> > > > > > > libraries
> > > > > > > loading libmxnet.so
> > > > > > >
> > > > > > > https://github.com/apache/incubator-mxnet/issues/16193
> > > > > > > Sheng requested Pypi to increase their size limit:
> > > > > > > https://github.com/pypa/pypi-support/issues/50
> > > > > > >
> > > > > > > Currently "the biggest cost for PyPI from [the many MXNet
> > > > > > > binaries
> > > > > > > with
> > > > > > > nightly
> > > > > > > release to Pypi] is the bandwidth consumed when several
> > > > > > > hundred
> > > > > > > mirrors
> > > > > > > attempt
> > > > > > > to mirror each release immediately after it's published". So
> > > > > > > Pypi
> > > > > > > is
> > > > > > > not
> > > > > > > inclined to allow us to upload even larger binaries on a
> > > > > > > nightly
> > > > > > > schedule.
> > > > > > > Their compromise is to allow it on a weekly cadence.
> > > > > > >
> > > > > > > However, I would like the community to revisit the necessity
> > > > > > > of
> > > > > > > releasing pre-
> > > > > > > release binaries to Pypi on a nightly (or weekly) cadence.
> > > > > > >
> > > > > > > Instead, we
> > > > > > > can
> > > > > > > release nightly binaries ONLY to a public S3 bucket and
> > > > > > > instruct
> > > > > > > users
> > > > > > > to
> > > > > > > install from there. On our side, we only need to prepare a
> > > > > > > html
> > > > > > > document that
> > > > > > > contains links to all released nightly binaries.
> > > > > > > Finally users will install the nightly releases via
> > > > > > >
> > > > > > > pip install --pre mxnet-cu101 -f
> > > > > > >
> > > > > > > http://mxnet.s3.amazonaws.com/mxnet-cu101/
> > > > > > > nightly.html
> > > > > > >
> > > > > > > Instead of
> > > > > > >
> > > > > > > pip install --pre mxnet-cu101
> > > > > > >
> > > > > > > Of course proper releases and release candidates should
> > > > > > > still be
> > > > > > > made
> > > > > > > available
> > > > > > > via Pypi. Thus releases would be installed via
> > > > > > >
> > > > > > > pip install mxnet-cu101
> > > > > > >
> > > > > > > And release candidates via
> > > > > > >
> > > > > > > pip install --pre mxnet-cu101
> > > > > > >
> > > > > > > This will substantially reduce the costs of the Pypi project
> > > > > > > and
> > > > > > > in
> > > > > > > fact
> > > > > > > matches
> > > > > > > the installation experience provided by PyTorch. I don't
> > > > > > > think the
> > > > > > > benefit of
> > > > > > > not including "-f
> > > > > > >
> > > > > > > http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
> > > > > > > matches the costs we currently externalize to the Pypi team.
> > > > > > >
> > > > > > > This suggestion seems uncontroversial to me. Thus I would
> > > > > > > like to
> > > > > > > start
> > > > > > > lazy
> > > > > > > consensus. If there are no objections, I will assume lazy
> > > > > > >
> > > > > > > consensus on
> > > > > > > stopping
> > > > > > > nightly releases to Pypi in 72hrs.
> > > > > > >
> > > > > > > Best regards
> > > > > > > Leonard
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
the
> > >> > > > third
> > >> > > > > URL,
> > >> > > > > as S3 Accelerate may or may not use all new CloudFront
> endpoints
> > >> yet.
> > >> > > > >
> > >> > > > > Regarding voting: Uploading to Pypi is currently impossible,
> > which
> > >> > is a
> > >> > > > > reality
> > >> > > > > (so there is no option to continue as we do currently). Pypi
> > folks
> > >> > > > > indicated
> > >> > > > > they will unblock our uploads to Pypi once we stop uploading
> > >> nightly
> > >> > > > > releases
> > >> > > > > and taking up 20% of their ressources [1].
> > >> > > > >
> > >> > > > > If there are any shortcomings or problems identified with
> > >> uploading
> > >> > to
> > >> > > > S3,
> > >> > > > > we
> > >> > > > > can work to address them. But for now, status quo is broken
> and
> > >> this
> > >> > > > seems
> > >> > > > > the
> > >> > > > > only solution addressing Pypi's problem.
> > >> > > > >
> > >> > > > > I don't mind if you state that you object to lazy consensus
> and
> > >> > start a
> > >> > > > > vote. If
> > >> > > > > your "maybe [...] start a proper vote" was supposed to be an
> > >> > objection
> > >> > > to
> > >> > > > > lazy
> > >> > > > > consensus, please state so clearly (I'm not sure if "maybe"
> > >> qualifies
> > >> > > as
> > >> > > > > objection). Though I think it only makes sense with at least 2
> > >> > options
> > >> > > to
> > >> > > > > vote
> > >> > > > > on. Status quo is not a meaningful option, as it is already
> > >> broken.
> > >> > > > >
> > >> > > > > Best regards
> > >> > > > > Leonard
> > >> > > > >
> > >> > > > > [1]:
> > >> > > >
> > >> https://github.com/pypa/pypi-support/issues/50#issuecomment-560479706
> > >> > > > >
> > >> > > > > On Tue, 2019-12-03 at 19:28 +0100, Marco de Abreu wrote:
> > >> > > > > Excellent! Could we maybe come up with a POC and a quick
> writeup
> > >> and
> > >> > > then
> > >> > > > > start a proper vote after everyone verified that it covers
> their
> > >> > > > > use-cases?
> > >> > > > > -Marco
> > >> > > > >
> > >> > > > > Sheng Zha schrieb am Di., 3. Dez. 2019,
> > >> 19:24:
> > >> > > > >
> > >> > > > > Yes, there is. We can also make it easier to access by using a
> > >> > > > > geo-location based DNS server so that China users are directed
> > to
> > >> > that
> > >> > > > > local mirror. The rest of the world is already covered by the
> > >> global
> > >> > > > > cloudfront.
> > >> > > > >
> > >> > > > > -sz
> > >> > > > >
> > >> > > > > On 2019/12/03 18:22:22, Marco de Abreu <
> marco.g.ab...@gmail.com
> > >
> > >> > > > > wrote:
> > >> > > > > Isn't there an s3 endpoint in Beijing?
> > >> > > > >
> > >> > > > > It seems like this topic still warrants some discussion and
> thus
> > >> I'd
> > >> > > > >
> > >> > > > > prefer
> > >> > > > > if we don't move forward with lazy consensus.
> > >> > > > >
> > >> > > > > -Marco
> > >> > > > >
> > >> > > > > Tao Lv schrieb am Di., 3. Dez. 2019,
> > 14:31:
> > >> > > > >
> > >> > > > > * For pypi, we can use mirrors.
> > >> > > > >
> > >> > > > > On Tue, Dec 3, 2019 at 9:28 PM Tao Lv
> > wrote:
> > >> > > > >
> > >> > > > > As we have many users in China, I'm considering the
> > >> > > > > accessibility of
> > >> > > > > S3.
> > >> > > > > For pip, we can mirrors.
> > >> > > > >
> > >> > > > > On Tue, Dec 3, 2019 at 3:24 PM Lausen, Leonard
> > >> > > > >
> > >> > > > > > >> > > > > wrote:
> > >> > > > >
> > >> > > > > I would like to remind everyone that lazy consensus is assumed
> > >> > > > > if no
> > >> > > > > objections
> > >> > > > > are raised before 2019-12-05 at 05:42 UTC. There has been some
> > >> > > > >
> > >> > > > > discussion
> > >> > > > > about
> > >> > > > > the proposal, but to my understanding no objections were
> > >> > > > > raised.
> > >> > > > > If the proposal is accepted, MXNet releases would be installed
> > >> > > > > via
> > >> > > > > pip install mxnet
> > >> > > > >
> > >> > > > > And release candidates via
> > >> > > > >
> > >> > > > > pip install --pre mxnet
> > >> > > > >
> > >> > > > > (or with the respective cuda version specifier appended etc.)
> > >> > > > >
> > >> > > > > To obtain releases built automatically from the master branch,
> > >> > > > > users
> > >> > > > > would need
> > >> > > > > to specify something like "-f
> > >> > > > > http://mxnet.s3.amazonaws.com/mxnet-X/nightly.html; option to
> > >> > > > > pip.
> > >> > > > > Best regards
> > >> > > > > Leonard
> > >> > > > >
> > >> > > > > On Mon, 2019-12-02 at 05:42 +, Lausen, Leonard wrote:
> > >> > > > > Hi MXNet Community,
> > >> > > > >
> > >> > > > > since more than 2 months our binary Python nightly releases
> > >> > > > >
> > >> > > > > published
> > >> > > > > on Pypi
> > >> > > > > are broken. The problem is that our binaries exceed Pypi's
> > >> > > > > size
> > >> > > > > limit.
> > >> > > > > Decreasing the binary size by adding compression breaks
> > >> > > > >
> > >> > > > > third-party
> > >> > > > > libraries
> > >> > > > > loading libmxnet.so
> > >> > > > >
> > >> > > > > https://github.com/apache/incubator-mxnet/issues/16193
> > >> > > > > Sheng requested Pypi to increase their size limit:
> > >> > > > > https://github.com/pypa/pypi-support/issues/50
> > >> > > > >
> > >> > > > > Currently "the biggest cost for PyPI from [the many MXNet
> > >> > > > > binaries
> > >> > > > > with
> > >> > > > > nightly
> > >> > > > > release to Pypi] is the bandwidth consumed when several
> > >> > > > > hundred
> > >> > > > > mirrors
> > >> > > > > attempt
> > >> > > > > to mirror each release immediately after it's published". So
> > >> > > > > Pypi
> > >> > > > > is
> > >> > > > > not
> > >> > > > > inclined to allow us to upload even larger binaries on a
> > >> > > > > nightly
> > >> > > > > schedule.
> > >> > > > > Their compromise is to allow it on a weekly cadence.
> > >> > > > >
> > >> > > > > However, I would like the community to revisit the necessity
> > >> > > > > of
> > >> > > > > releasing pre-
> > >> > > > > release binaries to Pypi on a nightly (or weekly) cadence.
> > >> > > > >
> > >> > > > > Instead, we
> > >> > > > > can
> > >> > > > > release nightly binaries ONLY to a public S3 bucket and
> > >> > > > > instruct
> > >> > > > > users
> > >> > > > > to
> > >> > > > > install from there. On our side, we only need to prepare a
> > >> > > > > html
> > >> > > > > document that
> > >> > > > > contains links to all released nightly binaries.
> > >> > > > > Finally users will install the nightly releases via
> > >> > > > >
> > >> > > > > pip install --pre mxnet-cu101 -f
> > >> > > > >
> > >> > > > > http://mxnet.s3.amazonaws.com/mxnet-cu101/
> > >> > > > > nightly.html
> > >> > > > >
> > >> > > > > Instead of
> > >> > > > >
> > >> > > > > pip install --pre mxnet-cu101
> > >> > > > >
> > >> > > > > Of course proper releases and release candidates should
> > >> > > > > still be
> > >> > > > > made
> > >> > > > > available
> > >> > > > > via Pypi. Thus releases would be installed via
> > >> > > > >
> > >> > > > > pip install mxnet-cu101
> > >> > > > >
> > >> > > > > And release candidates via
> > >> > > > >
> > >> > > > > pip install --pre mxnet-cu101
> > >> > > > >
> > >> > > > > This will substantially reduce the costs of the Pypi project
> > >> > > > > and
> > >> > > > > in
> > >> > > > > fact
> > >> > > > > matches
> > >> > > > > the installation experience provided by PyTorch. I don't
> > >> > > > > think the
> > >> > > > > benefit of
> > >> > > > > not including "-f
> > >> > > > >
> > >> > > > > http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
> > >> > > > > matches the costs we currently externalize to the Pypi team.
> > >> > > > >
> > >> > > > > This suggestion seems uncontroversial to me. Thus I would
> > >> > > > > like to
> > >> > > > > start
> > >> > > > > lazy
> > >> > > > > consensus. If there are no objections, I will assume lazy
> > >> > > > >
> > >> > > > > consensus on
> > >> > > > > stopping
> > >> > > > > nightly releases to Pypi in 72hrs.
> > >> > > > >
> > >> > > > > Best regards
> > >> > > > > Leonard
> > >> > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> >
>
tatus quo is not a meaningful option, as it is already
> broken.
> > > > > >
> > > > > > Best regards
> > > > > > Leonard
> > > > > >
> > > > > > [1]:
> > > > >
> > https://github.com/pypa/
> > > Best regards
>> > > > > Leonard
>> > > > >
>> > > > > [1]:
>> > > >
>> https://github.com/pypa/pypi-support/issues/50#issuecomment-560479706
>> > > > >
>> > > > > On Tue, 2019-12-03 at 19:28 +0100, Marco de Abreu wrote:
>> > > > > Excellent! Could we maybe come up with a POC and a quick writeup
>> and
>> > > then
>> > > > > start a proper vote after everyone verified that it covers their
>> > > > > use-cases?
>> > > > > -Marco
>> > > > >
>> > > > > Sheng Zha schrieb am Di., 3. Dez. 2019,
>> 19:24:
>> > > > >
>> > > > > Yes, there is. We can also make it easier to access by using a
>> > > > > geo-location based DNS server so that China users are directed to
>> > that
>> > > > > local mirror. The rest of the world is already covered by the
>> global
>> > > > > cloudfront.
>> > > > >
>> > > > > -sz
>> > > > >
>> > > > > On 2019/12/03 18:22:22, Marco de Abreu
>> > > > > wrote:
>> > > > > Isn't there an s3 endpoint in Beijing?
>> > > > >
>> > > > > It seems like this topic still warrants some discussion and thus
>> I'd
>> > > > >
>> > > > > prefer
>> > > > > if we don't move forward with lazy consensus.
>> > > > >
>> > > > > -Marco
>> > > > >
>> > > > > Tao Lv schrieb am Di., 3. Dez. 2019, 14:31:
>> > > > >
>> > > > > * For pypi, we can use mirrors.
>> > > > >
>> > > > > On Tue, Dec 3, 2019 at 9:28 PM Tao Lv wrote:
>> > > > >
>> > > > > As we have many users in China, I'm considering the
>> > > > > accessibility of
>> > > > > S3.
>> > > > > For pip, we can mirrors.
>> > > > >
>> > > > > On Tue, Dec 3, 2019 at 3:24 PM Lausen, Leonard
>> > > > >
>> > > > > > > > > > wrote:
>> > > > >
>> > > > > I would like to remind everyone that lazy consensus is assumed
>> > > > > if no
>> > > > > objections
>> > > > > are raised before 2019-12-05 at 05:42 UTC. There has been some
>> > > > >
>> > > > > discussion
>> > > > > about
>> > > > > the proposal, but to my understanding no objections were
>> > > > > raised.
>> > > > > If the proposal is accepted, MXNet releases would be installed
>> > > > > via
>> > > > > pip install mxnet
>> > > > >
>> > > > > And release candidates via
>> > > > >
>> > > > > pip install --pre mxnet
>> > > > >
>> > > > > (or with the respective cuda version specifier appended etc.)
>> > > > >
>> > > > > To obtain releases built automatically from the master branch,
>> > > > > users
>> > > > > would need
>> > > > > to specify something like "-f
>> > > > > http://mxnet.s3.amazonaws.com/mxnet-X/nightly.html; option to
>> > > > > pip.
>> > > > > Best regards
>> > > > > Leonard
>> > > > >
>> > > > > On Mon, 2019-12-02 at 05:42 +, Lausen, Leonard wrote:
>> > > > > Hi MXNet Community,
>> > > > >
>> > > > > since more than 2 months our binary Python nightly releases
>> > > > >
>> > > > > published
>> > > > > on Pypi
>> > > > > are broken. The problem is that our binaries exceed Pypi's
>> > > > > size
>> > > > > limit.
>> > > > > Decreasing the binary size by adding compression breaks
>> > > > >
>> > > > > third-party
>> > > > > libraries
>> > > > > loading libmxnet.so
>> > > > >
>> > > > > https://github.com/apache/incubator-mxnet/issues/16193
>> > > > > Sheng requested Pypi to increase their size limit:
>> > > > > https://github.com/pypa/pypi-support/issues/50
>> > > > >
>> > > > > Currently "the biggest cost for PyPI from [the many MXNet
>> > > > > binaries
>> > > > > with
>> > > > > nightly
>> > > > > release to Pypi] is the bandwidth consumed when several
>> > > > > hundred
>> > > > > mirrors
>> > > > > attempt
>> > > > > to mirror each release immediately after it's published". So
>> > > > > Pypi
>> > > > > is
>> > > > > not
>> > > > > inclined to allow us to upload even larger binaries on a
>> > > > > nightly
>> > > > > schedule.
>> > > > > Their compromise is to allow it on a weekly cadence.
>> > > > >
>> > > > > However, I would like the community to revisit the necessity
>> > > > > of
>> > > > > releasing pre-
>> > > > > release binaries to Pypi on a nightly (or weekly) cadence.
>> > > > >
>> > > > > Instead, we
>> > > > > can
>> > > > > release nightly binaries ONLY to a public S3 bucket and
>> > > > > instruct
>> > > > > users
>> > > > > to
>> > > > > install from there. On our side, we only need to prepare a
>> > > > > html
>> > > > > document that
>> > > > > contains links to all released nightly binaries.
>> > > > > Finally users will install the nightly releases via
>> > > > >
>> > > > > pip install --pre mxnet-cu101 -f
>> > > > >
>> > > > > http://mxnet.s3.amazonaws.com/mxnet-cu101/
>> > > > > nightly.html
>> > > > >
>> > > > > Instead of
>> > > > >
>> > > > > pip install --pre mxnet-cu101
>> > > > >
>> > > > > Of course proper releases and release candidates should
>> > > > > still be
>> > > > > made
>> > > > > available
>> > > > > via Pypi. Thus releases would be installed via
>> > > > >
>> > > > > pip install mxnet-cu101
>> > > > >
>> > > > > And release candidates via
>> > > > >
>> > > > > pip install --pre mxnet-cu101
>> > > > >
>> > > > > This will substantially reduce the costs of the Pypi project
>> > > > > and
>> > > > > in
>> > > > > fact
>> > > > > matches
>> > > > > the installation experience provided by PyTorch. I don't
>> > > > > think the
>> > > > > benefit of
>> > > > > not including "-f
>> > > > >
>> > > > > http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
>> > > > > matches the costs we currently externalize to the Pypi team.
>> > > > >
>> > > > > This suggestion seems uncontroversial to me. Thus I would
>> > > > > like to
>> > > > > start
>> > > > > lazy
>> > > > > consensus. If there are no objections, I will assume lazy
>> > > > >
>> > > > > consensus on
>> > > > > stopping
>> > > > > nightly releases to Pypi in 72hrs.
>> > > > >
>> > > > > Best regards
>> > > > > Leonard
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
t; > >
> > > > > On 2019/12/03 18:22:22, Marco de Abreu
> > > > > wrote:
> > > > > Isn't there an s3 endpoint in Beijing?
> > > > >
> > > > > It seems like this topic still warrants some discussion and thus
> I'd
> > > > >
> > > > > prefer
> > > > > if we don't move forward with lazy consensus.
> > > > >
> > > > > -Marco
> > > > >
> > > > > Tao Lv schrieb am Di., 3. Dez. 2019, 14:31:
> > > > >
> > > > > * For pypi, we can use mirrors.
> > > > >
> > > > > On Tue, Dec 3, 2019 at 9:28 PM Tao Lv wrote:
> > > > >
> > > > > As we have many users in China, I'm considering the
> > > > > accessibility of
> > > > > S3.
> > > > > For pip, we can mirrors.
> > > > >
> > > > > On Tue, Dec 3, 2019 at 3:24 PM Lausen, Leonard
> > > > >
> > > > > > > > > wrote:
> > > > >
> > > > > I would like to remind everyone that lazy consensus is assumed
> > > > > if no
> > > > > objections
> > > > > are raised before 2019-12-05 at 05:42 UTC. There has been some
> > > > >
> > > > > discussion
> > > > > about
> > > > > the proposal, but to my understanding no objections were
> > > > > raised.
> > > > > If the proposal is accepted, MXNet releases would be installed
> > > > > via
> > > > > pip install mxnet
> > > > >
> > > > > And release candidates via
> > > > >
> > > > > pip install --pre mxnet
> > > > >
> > > > > (or with the respective cuda version specifier appended etc.)
> > > > >
> > > > > To obtain releases built automatically from the master branch,
> > > > > users
> > > > > would need
> > > > > to specify something like "-f
> > > > > http://mxnet.s3.amazonaws.com/mxnet-X/nightly.html; option to
> > > > > pip.
> > > > > Best regards
> > > > > Leonard
> > > > >
> > > > > On Mon, 2019-12-02 at 05:42 +, Lausen, Leonard wrote:
> > > > > Hi MXNet Community,
> > > > >
> > > > > since more than 2 months our binary Python nightly releases
> > > > >
> > > > > published
> > > > > on Pypi
> > > > > are broken. The problem is that our binaries exceed Pypi's
> > > > > size
> > > > > limit.
> > > > > Decreasing the binary size by adding compression breaks
> > > > >
> > > > > third-party
> > > > > libraries
> > > > > loading libmxnet.so
> > > > >
> > > > > https://github.com/apache/incubator-mxnet/issues/16193
> > > > > Sheng requested Pypi to increase their size limit:
> > > > > https://github.com/pypa/pypi-support/issues/50
> > > > >
> > > > > Currently "the biggest cost for PyPI from [the many MXNet
> > > > > binaries
> > > > > with
> > > > > nightly
> > > > > release to Pypi] is the bandwidth consumed when several
> > > > > hundred
> > > > > mirrors
> > > > > attempt
> > > > > to mirror each release immediately after it's published". So
> > > > > Pypi
> > > > > is
> > > > > not
> > > > > inclined to allow us to upload even larger binaries on a
> > > > > nightly
> > > > > schedule.
> > > > > Their compromise is to allow it on a weekly cadence.
> > > > >
> > > > > However, I would like the community to revisit the necessity
> > > > > of
> > > > > releasing pre-
> > > > > release binaries to Pypi on a nightly (or weekly) cadence.
> > > > >
> > > > > Instead, we
> > > > > can
> > > > > release nightly binaries ONLY to a public S3 bucket and
> > > > > instruct
> > > > > users
> > > > > to
> > > > > install from there. On our side, we only need to prepare a
> > > > > html
> > > > > document that
> > > > > contains links to all released nightly binaries.
> > > > > Finally users will install the nightly releases via
> > > > >
> > > > > pip install --pre mxnet-cu101 -f
> > > > >
> > > > > http://mxnet.s3.amazonaws.com/mxnet-cu101/
> > > > > nightly.html
> > > > >
> > > > > Instead of
> > > > >
> > > > > pip install --pre mxnet-cu101
> > > > >
> > > > > Of course proper releases and release candidates should
> > > > > still be
> > > > > made
> > > > > available
> > > > > via Pypi. Thus releases would be installed via
> > > > >
> > > > > pip install mxnet-cu101
> > > > >
> > > > > And release candidates via
> > > > >
> > > > > pip install --pre mxnet-cu101
> > > > >
> > > > > This will substantially reduce the costs of the Pypi project
> > > > > and
> > > > > in
> > > > > fact
> > > > > matches
> > > > > the installation experience provided by PyTorch. I don't
> > > > > think the
> > > > > benefit of
> > > > > not including "-f
> > > > >
> > > > > http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
> > > > > matches the costs we currently externalize to the Pypi team.
> > > > >
> > > > > This suggestion seems uncontroversial to me. Thus I would
> > > > > like to
> > > > > start
> > > > > lazy
> > > > > consensus. If there are no objections, I will assume lazy
> > > > >
> > > > > consensus on
> > > > > stopping
> > > > > nightly releases to Pypi in 72hrs.
> > > > >
> > > > > Best regards
> > > > > Leonard
> > > > >
> > > > >
> > > >
> > >
> >
>
e" was supposed to be an objection
>>>> to
>>>>>> lazy
>>>>>> consensus, please state so clearly (I'm not sure if "maybe" qualifies
>>>> as
>>>>>> objection). Though I think it only makes sense with at least 2
co
> >>>>
> >>>> Sheng Zha schrieb am Di., 3. Dez. 2019, 19:24:
> >>>>
> >>>> Yes, there is. We can also make it easier to access by using a
> >>>> geo-location based DNS server so that China users are directed to t
;> On Tue, Dec 3, 2019 at 9:28 PM Tao Lv wrote:
>>>>
>>>> As we have many users in China, I'm considering the
>>>> accessibility of
>>>> S3.
>>>> For pip, we can mirrors.
>>>>
>>>> On Tue, Dec 3, 2019 at 3:24 PM Lausen, Leonard
>>>>
>>>> >>> wrote:
>>>>
>>>> I would like to remind everyone that lazy consensus is assumed
>>>> if no
>>>> objections
>>>> are raised before 2019-12-05 at 05:42 UTC. There has been some
>>>>
>>>> discussion
>>>> about
>>>> the proposal, but to my understanding no objections were
>>>> raised.
>>>> If the proposal is accepted, MXNet releases would be installed
>>>> via
>>>> pip install mxnet
>>>>
>>>> And release candidates via
>>>>
>>>> pip install --pre mxnet
>>>>
>>>> (or with the respective cuda version specifier appended etc.)
>>>>
>>>> To obtain releases built automatically from the master branch,
>>>> users
>>>> would need
>>>> to specify something like "-f
>>>> http://mxnet.s3.amazonaws.com/mxnet-X/nightly.html; option to
>>>> pip.
>>>> Best regards
>>>> Leonard
>>>>
>>>> On Mon, 2019-12-02 at 05:42 +, Lausen, Leonard wrote:
>>>> Hi MXNet Community,
>>>>
>>>> since more than 2 months our binary Python nightly releases
>>>>
>>>> published
>>>> on Pypi
>>>> are broken. The problem is that our binaries exceed Pypi's
>>>> size
>>>> limit.
>>>> Decreasing the binary size by adding compression breaks
>>>>
>>>> third-party
>>>> libraries
>>>> loading libmxnet.so
>>>>
>>>> https://github.com/apache/incubator-mxnet/issues/16193
>>>> Sheng requested Pypi to increase their size limit:
>>>> https://github.com/pypa/pypi-support/issues/50
>>>>
>>>> Currently "the biggest cost for PyPI from [the many MXNet
>>>> binaries
>>>> with
>>>> nightly
>>>> release to Pypi] is the bandwidth consumed when several
>>>> hundred
>>>> mirrors
>>>> attempt
>>>> to mirror each release immediately after it's published". So
>>>> Pypi
>>>> is
>>>> not
>>>> inclined to allow us to upload even larger binaries on a
>>>> nightly
>>>> schedule.
>>>> Their compromise is to allow it on a weekly cadence.
>>>>
>>>> However, I would like the community to revisit the necessity
>>>> of
>>>> releasing pre-
>>>> release binaries to Pypi on a nightly (or weekly) cadence.
>>>>
>>>> Instead, we
>>>> can
>>>> release nightly binaries ONLY to a public S3 bucket and
>>>> instruct
>>>> users
>>>> to
>>>> install from there. On our side, we only need to prepare a
>>>> html
>>>> document that
>>>> contains links to all released nightly binaries.
>>>> Finally users will install the nightly releases via
>>>>
>>>> pip install --pre mxnet-cu101 -f
>>>>
>>>> http://mxnet.s3.amazonaws.com/mxnet-cu101/
>>>> nightly.html
>>>>
>>>> Instead of
>>>>
>>>> pip install --pre mxnet-cu101
>>>>
>>>> Of course proper releases and release candidates should
>>>> still be
>>>> made
>>>> available
>>>> via Pypi. Thus releases would be installed via
>>>>
>>>> pip install mxnet-cu101
>>>>
>>>> And release candidates via
>>>>
>>>> pip install --pre mxnet-cu101
>>>>
>>>> This will substantially reduce the costs of the Pypi project
>>>> and
>>>> in
>>>> fact
>>>> matches
>>>> the installation experience provided by PyTorch. I don't
>>>> think the
>>>> benefit of
>>>> not including "-f
>>>>
>>>> http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
>>>> matches the costs we currently externalize to the Pypi team.
>>>>
>>>> This suggestion seems uncontroversial to me. Thus I would
>>>> like to
>>>> start
>>>> lazy
>>>> consensus. If there are no objections, I will assume lazy
>>>>
>>>> consensus on
>>>> stopping
>>>> nightly releases to Pypi in 72hrs.
>>>>
>>>> Best regards
>>>> Leonard
>>>>
>>>>
>>>
>>
gt; To obtain releases built automatically from the master branch,
> > > users
> > > would need
> > > to specify something like "-f
> > > http://mxnet.s3.amazonaws.com/mxnet-X/nightly.html; option to
> > > pip.
> > > Best regards
> > > Leonard
> > >
> > > On Mon, 2019-12-02 at 05:42 +, Lausen, Leonard wrote:
> > > Hi MXNet Community,
> > >
> > > since more than 2 months our binary Python nightly releases
> > >
> > > published
> > > on Pypi
> > > are broken. The problem is that our binaries exceed Pypi's
> > > size
> > > limit.
> > > Decreasing the binary size by adding compression breaks
> > >
> > > third-party
> > > libraries
> > > loading libmxnet.so
> > >
> > > https://github.com/apache/incubator-mxnet/issues/16193
> > > Sheng requested Pypi to increase their size limit:
> > > https://github.com/pypa/pypi-support/issues/50
> > >
> > > Currently "the biggest cost for PyPI from [the many MXNet
> > > binaries
> > > with
> > > nightly
> > > release to Pypi] is the bandwidth consumed when several
> > > hundred
> > > mirrors
> > > attempt
> > > to mirror each release immediately after it's published". So
> > > Pypi
> > > is
> > > not
> > > inclined to allow us to upload even larger binaries on a
> > > nightly
> > > schedule.
> > > Their compromise is to allow it on a weekly cadence.
> > >
> > > However, I would like the community to revisit the necessity
> > > of
> > > releasing pre-
> > > release binaries to Pypi on a nightly (or weekly) cadence.
> > >
> > > Instead, we
> > > can
> > > release nightly binaries ONLY to a public S3 bucket and
> > > instruct
> > > users
> > > to
> > > install from there. On our side, we only need to prepare a
> > > html
> > > document that
> > > contains links to all released nightly binaries.
> > > Finally users will install the nightly releases via
> > >
> > > pip install --pre mxnet-cu101 -f
> > >
> > > http://mxnet.s3.amazonaws.com/mxnet-cu101/
> > > nightly.html
> > >
> > > Instead of
> > >
> > > pip install --pre mxnet-cu101
> > >
> > > Of course proper releases and release candidates should
> > > still be
> > > made
> > > available
> > > via Pypi. Thus releases would be installed via
> > >
> > > pip install mxnet-cu101
> > >
> > > And release candidates via
> > >
> > > pip install --pre mxnet-cu101
> > >
> > > This will substantially reduce the costs of the Pypi project
> > > and
> > > in
> > > fact
> > > matches
> > > the installation experience provided by PyTorch. I don't
> > > think the
> > > benefit of
> > > not including "-f
> > >
> > > http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
> > > matches the costs we currently externalize to the Pypi team.
> > >
> > > This suggestion seems uncontroversial to me. Thus I would
> > > like to
> > > start
> > > lazy
> > > consensus. If there are no objections, I will assume lazy
> > >
> > > consensus on
> > > stopping
> > > nightly releases to Pypi in 72hrs.
> > >
> > > Best regards
> > > Leonard
> > >
> > >
> >
>
s://github.com/pypa/pypi-support/issues/50
> >
> > Currently "the biggest cost for PyPI from [the many MXNet
> > binaries
> > with
> > nightly
> > release to Pypi] is the bandwidth consumed when several
> > hundred
> > mirrors
> > attempt
> > to mirror each release immediately after it's published". So
> > Pypi
> > is
> > not
> > inclined to allow us to upload even larger binaries on a
> > nightly
> > schedule.
> > Their compromise is to allow it on a weekly cadence.
> >
> > However, I would like the community to revisit the necessity
> > of
> > releasing pre-
> > release binaries to Pypi on a nightly (or weekly) cadence.
> >
> > Instead, we
> > can
> > release nightly binaries ONLY to a public S3 bucket and
> > instruct
> > users
> > to
> > install from there. On our side, we only need to prepare a
> > html
> > document that
> > contains links to all released nightly binaries.
> > Finally users will install the nightly releases via
> >
> > pip install --pre mxnet-cu101 -f
> >
> > http://mxnet.s3.amazonaws.com/mxnet-cu101/
> > nightly.html
> >
> > Instead of
> >
> > pip install --pre mxnet-cu101
> >
> > Of course proper releases and release candidates should
> > still be
> > made
> > available
> > via Pypi. Thus releases would be installed via
> >
> > pip install mxnet-cu101
> >
> > And release candidates via
> >
> > pip install --pre mxnet-cu101
> >
> > This will substantially reduce the costs of the Pypi project
> > and
> > in
> > fact
> > matches
> > the installation experience provided by PyTorch. I don't
> > think the
> > benefit of
> > not including "-f
> >
> > http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
> > matches the costs we currently externalize to the Pypi team.
> >
> > This suggestion seems uncontroversial to me. Thus I would
> > like to
> > start
> > lazy
> > consensus. If there are no objections, I will assume lazy
> >
> > consensus on
> > stopping
> > nightly releases to Pypi in 72hrs.
> >
> > Best regards
> > Leonard
> >
> >
>
.org
> Cc: d...@mxnet.apache.org
> Subject: Re: Stopping nightly releases to Pypi
>
> Shall we update the website installation page with nightly build information
> as well (after we figure out the CD details)?
>
> Best,
> Haibin
>
> On Tue, Dec 10, 2019 at 10:15 PM
> > > if we don't move forward with lazy consensus.
> > > > >
> > > > > -Marco
> > > > >
> > > > > Tao Lv schrieb am Di., 3. Dez. 2019, 14:31:
> > > > >
> > > > > * For pypi, we can use mirrors.
> >
would be installed
> > > > via
> > > > pip install mxnet
> > > >
> > > > And release candidates via
> > > >
> > > > pip install --pre mxnet
> > > >
> > > > (or with the respective cuda version specifier appended etc.)
> > > >
> > > > To obtain releases built automatically from the master branch,
> > > > users
> > > > would need
> > > > to specify something like "-f
> > > > http://mxnet.s3.amazonaws.com/mxnet-X/nightly.html; option to
> > > > pip.
> > > > Best regards
> > > > Leonard
> > > >
> > > > On Mon, 2019-12-02 at 05:42 +, Lausen, Leonard wrote:
> > > > Hi MXNet Community,
> > > >
> > > > since more than 2 months our binary Python nightly releases
> > > >
> > > > published
> > > > on Pypi
> > > > are broken. The problem is that our binaries exceed Pypi's
> > > > size
> > > > limit.
> > > > Decreasing the binary size by adding compression breaks
> > > >
> > > > third-party
> > > > libraries
> > > > loading libmxnet.so
> > > >
> > > > https://github.com/apache/incubator-mxnet/issues/16193
> > > > Sheng requested Pypi to increase their size limit:
> > > > https://github.com/pypa/pypi-support/issues/50
> > > >
> > > > Currently "the biggest cost for PyPI from [the many MXNet
> > > > binaries
> > > > with
> > > > nightly
> > > > release to Pypi] is the bandwidth consumed when several
> > > > hundred
> > > > mirrors
> > > > attempt
> > > > to mirror each release immediately after it's published". So
> > > > Pypi
> > > > is
> > > > not
> > > > inclined to allow us to upload even larger binaries on a
> > > > nightly
> > > > schedule.
> > > > Their compromise is to allow it on a weekly cadence.
> > > >
> > > > However, I would like the community to revisit the necessity
> > > > of
> > > > releasing pre-
> > > > release binaries to Pypi on a nightly (or weekly) cadence.
> > > >
> > > > Instead, we
> > > > can
> > > > release nightly binaries ONLY to a public S3 bucket and
> > > > instruct
> > > > users
> > > > to
> > > > install from there. On our side, we only need to prepare a
> > > > html
> > > > document that
> > > > contains links to all released nightly binaries.
> > > > Finally users will install the nightly releases via
> > > >
> > > > pip install --pre mxnet-cu101 -f
> > > >
> > > > http://mxnet.s3.amazonaws.com/mxnet-cu101/
> > > > nightly.html
> > > >
> > > > Instead of
> > > >
> > > > pip install --pre mxnet-cu101
> > > >
> > > > Of course proper releases and release candidates should
> > > > still be
> > > > made
> > > > available
> > > > via Pypi. Thus releases would be installed via
> > > >
> > > > pip install mxnet-cu101
> > > >
> > > > And release candidates via
> > > >
> > > > pip install --pre mxnet-cu101
> > > >
> > > > This will substantially reduce the costs of the Pypi project
> > > > and
> > > > in
> > > > fact
> > > > matches
> > > > the installation experience provided by PyTorch. I don't
> > > > think the
> > > > benefit of
> > > > not including "-f
> > > >
> > > > http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
> > > > matches the costs we currently externalize to the Pypi team.
> > > >
> > > > This suggestion seems uncontroversial to me. Thus I would
> > > > like to
> > > > start
> > > > lazy
> > > > consensus. If there are no objections, I will assume lazy
> > > >
> > > > consensus on
> > > > stopping
> > > > nightly releases to Pypi in 72hrs.
> > > >
> > > > Best regards
> > > > Leonard
> > > >
> > > >
e cuda version specifier appended etc.)
> > >
> > > To obtain releases built automatically from the master branch,
> > > users
> > > would need
> > > to specify something like "-f
> > > http://mxnet.s3.amazonaws.com/mxnet-X/nightly.html; option to
to increase their size limit:
> > https://github.com/pypa/pypi-support/issues/50
> >
> > Currently "the biggest cost for PyPI from [the many MXNet
> > binaries
> > with
> > nightly
> > release to Pypi] is the bandwidth consumed when several
> > hundred
> > mirrors
> > attempt
> > to mirror each release immediately after it's published". So
> > Pypi
> > is
> > not
> > inclined to allow us to upload even larger binaries on a
> > nightly
> > schedule.
> > Their compromise is to allow it on a weekly cadence.
> >
> > However, I would like the community to revisit the necessity
> > of
> > releasing pre-
> > release binaries to Pypi on a nightly (or weekly) cadence.
> >
> > Instead, we
> > can
> > release nightly binaries ONLY to a public S3 bucket and
> > instruct
> > users
> > to
> > install from there. On our side, we only need to prepare a
> > html
> > document that
> > contains links to all released nightly binaries.
> > Finally users will install the nightly releases via
> >
> > pip install --pre mxnet-cu101 -f
> >
> > http://mxnet.s3.amazonaws.com/mxnet-cu101/
> > nightly.html
> >
> > Instead of
> >
> > pip install --pre mxnet-cu101
> >
> > Of course proper releases and release candidates should
> > still be
> > made
> > available
> > via Pypi. Thus releases would be installed via
> >
> > pip install mxnet-cu101
> >
> > And release candidates via
> >
> > pip install --pre mxnet-cu101
> >
> > This will substantially reduce the costs of the Pypi project
> > and
> > in
> > fact
> > matches
> > the installation experience provided by PyTorch. I don't
> > think the
> > benefit of
> > not including "-f
> >
> > http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
> > matches the costs we currently externalize to the Pypi team.
> >
> > This suggestion seems uncontroversial to me. Thus I would
> > like to
> > start
> > lazy
> > consensus. If there are no objections, I will assume lazy
> >
> > consensus on
> > stopping
> > nightly releases to Pypi in 72hrs.
> >
> > Best regards
> > Leonard
> >
> >
currently externalize to the Pypi team.
This suggestion seems uncontroversial to me. Thus I would
like to
start
lazy
consensus. If there are no objections, I will assume lazy
consensus on
stopping
nightly releases to Pypi in 72hrs.
Best regards
Leonard
instruct
> users
> to
> install from there. On our side, we only need to prepare a
> html
> document that
> contains links to all released nightly binaries.
> Finally users will install the nightly releases via
>
> pip install --pre mxnet-cu101 -f
>
> http://mxnet.s3.amazonaws.com/mxnet-cu101/
> nightly.html
>
> Instead of
>
> pip install --pre mxnet-cu101
>
> Of course proper releases and release candidates should
> still be
> made
> available
> via Pypi. Thus releases would be installed via
>
> pip install mxnet-cu101
>
> And release candidates via
>
> pip install --pre mxnet-cu101
>
> This will substantially reduce the costs of the Pypi project
> and
> in
> fact
> matches
> the installation experience provided by PyTorch. I don't
> think the
> benefit of
> not including "-f
>
> http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
> matches the costs we currently externalize to the Pypi team.
>
> This suggestion seems uncontroversial to me. Thus I would
> like to
> start
> lazy
> consensus. If there are no objections, I will assume lazy
>
> consensus on
> stopping
> nightly releases to Pypi in 72hrs.
>
> Best regards
> Leonard
>
>
sing pre-
> release binaries to Pypi on a nightly (or weekly) cadence.
>
> Instead, we
> can
> release nightly binaries ONLY to a public S3 bucket and
> instruct
> users
> to
> install from there. On our side, we only need to prepare a
> html
> document that
> con
nk the
benefit of
not including "-f
http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
matches the costs we currently externalize to the Pypi team.
This suggestion seems uncontroversial to me. Thus I would
like to
start
lazy
consensus. If there are no objections, I will assume lazy
conse
hed". So
> > Pypi
> > > > is
> > > > > > not
> > > > > > > > > inclined to allow us to upload even larger binaries on a
> > nightly
> > > > > > > > schedule.
> > > > > > > &g
> > > > > > >
> > > > > > > > However, I would like the community to revisit the necessity
> of
> > > > > > >
> > > > > > > releasing pre-
> > > > > > > > release binaries to Pypi on a nightly (or weekly) cadence.
> > >
> > > Instead, we
> > > > > > > can
> > > > > > > > release nightly binaries ONLY to a public S3 bucket and
> instruct
> > >
> > > users
> > > > > > > to
> > > > > > > > install from there. On our side, we only need to prepare a
> html
> > > > > > >
> > > > > > > document that
> > > > > > > > contains links to all released nightly binaries.
> > > > > > > > Finally users will install the nightly releases via
> > > > > > > >
> > > > > > > > pip install --pre mxnet-cu101 -f
> > > > > > >
> > > > > > > http://mxnet.s3.amazonaws.com/mxnet-cu101/
> > > > > > > > nightly.html
> > > > > > > >
> > > > > > > > Instead of
> > > > > > > >
> > > > > > > > pip install --pre mxnet-cu101
> > > > > > > >
> > > > > > > > Of course proper releases and release candidates should
> still be
> > >
> > > made
> > > > > > > > available
> > > > > > > > via Pypi. Thus releases would be installed via
> > > > > > > >
> > > > > > > > pip install mxnet-cu101
> > > > > > > >
> > > > > > > > And release candidates via
> > > > > > > >
> > > > > > > > pip install --pre mxnet-cu101
> > > > > > > >
> > > > > > > > This will substantially reduce the costs of the Pypi project
> and
> > >
> > > in
> > > > > fact
> > > > > > > > matches
> > > > > > > > the installation experience provided by PyTorch. I don't
> think the
> > > > > > >
> > > > > > > benefit of
> > > > > > > > not including "-f
> > > > > > >
> > > > > > > http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
> > > > > > > > matches the costs we currently externalize to the Pypi team.
> > > > > > > >
> > > > > > > > This suggestion seems uncontroversial to me. Thus I would
> like to
> > > > >
> > > > > start
> > > > > > > lazy
> > > > > > > > consensus. If there are no objections, I will assume lazy
> > >
> > > consensus on
> > > > > > > > stopping
> > > > > > > > nightly releases to Pypi in 72hrs.
> > > > > > > >
> > > > > > > > Best regards
> > > > > > > > Leonard
>
cadence.
> >
> > Instead, we
> > > > > > can
> > > > > > > release nightly binaries ONLY to a public S3 bucket and instruct
> >
> > users
> > > > > > to
> > > > > > > install from there. On our
; > > > > > install from there. On our side, we only need to prepare a html
> > > > >
> > > > > document that
> > > > > > contains links to all released nightly binaries.
> > > > > > Finally users will install the nightly releases via
> > > > > >
> > > > > > pip install --pre mxnet-cu101 -f
> > > > >
> > > > > http://mxnet.s3.amazonaws.com/mxnet-cu101/
> > > > > > nightly.html
> > > > > >
> > > > > > Instead of
> > > > > >
> > > > > > pip install --pre mxnet-cu101
> > > > > >
> > > > > > Of course proper releases and release candidates should still be
> > > > > > made
> > > > > > available
> > > > > > via Pypi. Thus releases would be installed via
> > > > > >
> > > > > > pip install mxnet-cu101
> > > > > >
> > > > > > And release candidates via
> > > > > >
> > > > > > pip install --pre mxnet-cu101
> > > > > >
> > > > > > This will substantially reduce the costs of the Pypi project and in
> > >
> > > fact
> > > > > > matches
> > > > > > the installation experience provided by PyTorch. I don't think the
> > > > >
> > > > > benefit of
> > > > > > not including "-f
> > > > >
> > > > > http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
> > > > > > matches the costs we currently externalize to the Pypi team.
> > > > > >
> > > > > > This suggestion seems uncontroversial to me. Thus I would like to
> > >
> > > start
> > > > > lazy
> > > > > > consensus. If there are no objections, I will assume lazy consensus
> > > > > > on
> > > > > > stopping
> > > > > > nightly releases to Pypi in 72hrs.
> > > > > >
> > > > > > Best regards
> > > > > > Leonard
ompromise is to allow it on a weekly cadence.
> > > > > > >> >
> > > > > > >> > However, I would like the community to revisit the necessity
> > of
> > > > > > >> releasing pre-
> > > > > > >> > release binaries to Pypi on a nightly (or weekly) cadence.
> > > > Instead, we
> > > > > > >> can
> > > > > > >> > release nightly binaries ONLY to a public S3 bucket and
> > instruct
> > > > users
> > > > > > >> to
> > > > > > >> > install from there. On our side, we only need to prepare a
> > html
> > > > > > >> document that
> > > > > > >> > contains links to all released nightly binaries.
> > > > > > >> > Finally users will install the nightly releases via
> > > > > > >> >
> > > > > > >> > pip install --pre mxnet-cu101 -f
> > > > > > >> http://mxnet.s3.amazonaws.com/mxnet-cu101/
> > > > > > >> > nightly.html
> > > > > > >> >
> > > > > > >> > Instead of
> > > > > > >> >
> > > > > > >> > pip install --pre mxnet-cu101
> > > > > > >> >
> > > > > > >> > Of course proper releases and release candidates should still
> > be
> > > > made
> > > > > > >> > available
> > > > > > >> > via Pypi. Thus releases would be installed via
> > > > > > >> >
> > > > > > >> > pip install mxnet-cu101
> > > > > > >> >
> > > > > > >> > And release candidates via
> > > > > > >> >
> > > > > > >> > pip install --pre mxnet-cu101
> > > > > > >> >
> > > > > > >> > This will substantially reduce the costs of the Pypi project
> > and
> > > > in
> > > > > > fact
> > > > > > >> > matches
> > > > > > >> > the installation experience provided by PyTorch. I don't
> > think the
> > > > > > >> benefit of
> > > > > > >> > not including "-f
> > > > > > >> http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
> > > > > > >> > matches the costs we currently externalize to the Pypi team.
> > > > > > >> >
> > > > > > >> > This suggestion seems uncontroversial to me. Thus I would
> > like to
> > > > > > start
> > > > > > >> lazy
> > > > > > >> > consensus. If there are no objections, I will assume lazy
> > > > consensus on
> > > > > > >> > stopping
> > > > > > >> > nightly releases to Pypi in 72hrs.
> > > > > > >> >
> > > > > > >> > Best regards
> > > > > > >> > Leonard
> > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
> > > > > >> >
> > > > > >> > However, I would like the community to revisit the necessity
> of
> > > > > >> releasing pre-
> > > > > >> > release binaries to Pypi on a nightly (or weekly) cadence.
>
inaries ONLY to a public S3 bucket and instruct
> > users
> > > > >> to
> > > > >> > install from there. On our side, we only need to prepare a html
> > > > >> document that
> > > > >> > contains links to all released nightly binaries.
> > > > >> > Finally users will install the nightly releases via
> > > > >> >
> > > > >> > pip install --pre mxnet-cu101 -f
> > > > >> http://mxnet.s3.amazonaws.com/mxnet-cu101/
> > > > >> > nightly.html
> > > > >> >
> > > > >> > Instead of
> > > > >> >
> > > > >> > pip install --pre mxnet-cu101
> > > > >> >
> > > > >> > Of course proper releases and release candidates should still be
> > made
> > > > >> > available
> > > > >> > via Pypi. Thus releases would be installed via
> > > > >> >
> > > > >> > pip install mxnet-cu101
> > > > >> >
> > > > >> > And release candidates via
> > > > >> >
> > > > >> > pip install --pre mxnet-cu101
> > > > >> >
> > > > >> > This will substantially reduce the costs of the Pypi project and
> > in
> > > > fact
> > > > >> > matches
> > > > >> > the installation experience provided by PyTorch. I don't think the
> > > > >> benefit of
> > > > >> > not including "-f
> > > > >> http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
> > > > >> > matches the costs we currently externalize to the Pypi team.
> > > > >> >
> > > > >> > This suggestion seems uncontroversial to me. Thus I would like to
> > > > start
> > > > >> lazy
> > > > >> > consensus. If there are no objections, I will assume lazy
> > consensus on
> > > > >> > stopping
> > > > >> > nightly releases to Pypi in 72hrs.
> > > > >> >
> > > > >> > Best regards
> > > > >> > Leonard
> > > > >>
> > > > >
> > > >
> > >
> >
>
e only need to prepare a html
> > > >> document that
> > > >> > contains links to all released nightly binaries.
> > > >> > Finally users will install the nightly releases via
> > > >> >
> > > >> > pip install --pre mxnet-cu101 -f
> > > >> http://mxnet.s3.amazonaws.com/mxnet-cu101/
> > > >> > nightly.html
> > > >> >
> > > >> > Instead of
> > > >> >
> > > >> > pip install --pre mxnet-cu101
> > > >> >
> > > >> > Of course proper releases and release candidates should still be
> made
> > > >> > available
> > > >> > via Pypi. Thus releases would be installed via
> > > >> >
> > > >> > pip install mxnet-cu101
> > > >> >
> > > >> > And release candidates via
> > > >> >
> > > >> > pip install --pre mxnet-cu101
> > > >> >
> > > >> > This will substantially reduce the costs of the Pypi project and
> in
> > > fact
> > > >> > matches
> > > >> > the installation experience provided by PyTorch. I don't think the
> > > >> benefit of
> > > >> > not including "-f
> > > >> http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
> > > >> > matches the costs we currently externalize to the Pypi team.
> > > >> >
> > > >> > This suggestion seems uncontroversial to me. Thus I would like to
> > > start
> > > >> lazy
> > > >> > consensus. If there are no objections, I will assume lazy
> consensus on
> > > >> > stopping
> > > >> > nightly releases to Pypi in 72hrs.
> > > >> >
> > > >> > Best regards
> > > >> > Leonard
> > > >>
> > > >
> > >
> >
>
; > >> > nightly.html
> > >> >
> > >> > Instead of
> > >> >
> > >> > pip install --pre mxnet-cu101
> > >> >
> > >> > Of course proper releases and release candidates should still be made
> > >> > available
> > >> > via Pypi. Thus releases would be installed via
> > >> >
> > >> > pip install mxnet-cu101
> > >> >
> > >> > And release candidates via
> > >> >
> > >> > pip install --pre mxnet-cu101
> > >> >
> > >> > This will substantially reduce the costs of the Pypi project and in
> > fact
> > >> > matches
> > >> > the installation experience provided by PyTorch. I don't think the
> > >> benefit of
> > >> > not including "-f
> > >> http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
> > >> > matches the costs we currently externalize to the Pypi team.
> > >> >
> > >> > This suggestion seems uncontroversial to me. Thus I would like to
> > start
> > >> lazy
> > >> > consensus. If there are no objections, I will assume lazy consensus on
> > >> > stopping
> > >> > nightly releases to Pypi in 72hrs.
> > >> >
> > >> > Best regards
> > >> > Leonard
> > >>
> > >
> >
>
gt; >> > pip install --pre mxnet-cu101
> >> >
> >> > Of course proper releases and release candidates should still be made
> >> > available
> >> > via Pypi. Thus releases would be installed via
> >> >
> >> > pip install mxnet-cu101
> >> >
> >> > And release candidates via
> >> >
> >> > pip install --pre mxnet-cu101
> >> >
> >> > This will substantially reduce the costs of the Pypi project and in
> fact
> >> > matches
> >> > the installation experience provided by PyTorch. I don't think the
> >> benefit of
> >> > not including "-f
> >> http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
> >> > matches the costs we currently externalize to the Pypi team.
> >> >
> >> > This suggestion seems uncontroversial to me. Thus I would like to
> start
> >> lazy
> >> > consensus. If there are no objections, I will assume lazy consensus on
> >> > stopping
> >> > nightly releases to Pypi in 72hrs.
> >> >
> >> > Best regards
> >> > Leonard
> >>
> >
>
pip install --pre mxnet-cu101
>> >
>> > Of course proper releases and release candidates should still be made
>> > available
>> > via Pypi. Thus releases would be installed via
>> >
>> > pip install mxnet-cu101
>> >
>> > And
stantially reduce the costs of the Pypi project and in fact
> > matches
> > the installation experience provided by PyTorch. I don't think the
> benefit of
> > not including "-f http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html
> "
> > matches the costs we currently externalize to the Pypi team.
> >
> > This suggestion seems uncontroversial to me. Thus I would like to start
> lazy
> > consensus. If there are no objections, I will assume lazy consensus on
> > stopping
> > nightly releases to Pypi in 72hrs.
> >
> > Best regards
> > Leonard
>
m/mxnet-cu101/nightly.html;
> matches the costs we currently externalize to the Pypi team.
>
> This suggestion seems uncontroversial to me. Thus I would like to start lazy
> consensus. If there are no objections, I will assume lazy consensus on
> stopping
> nightly releases to Pypi in 72hrs.
>
> Best regards
> Leonard
we can once again provide the whole link, but getting directly
> > > from
> > > pip is the familiar experience for most devs.
> > >
> > > Yes, 1.6 is the target release, but I don't see a world where the team can
> > > create new operators, and then ge
;> ________
>> From: Lausen, Leonard
>> Sent: Sunday, December 1, 2019 10:08 PM
>> To: dev@mxnet.incubator.apache.org
>> Cc: Kamakoti, Balaji
>> Subject: Re: Stopping nightly releases to Pypi
>>
>> If we decide to do weekly pre
Leonard
> Sent: Sunday, December 1, 2019 10:08 PM
> To: dev@mxnet.incubator.apache.org
> Cc: Kamakoti, Balaji
> Subject: Re: Stopping nightly releases to Pypi
>
> If we decide to do weekly pre-release builds to Pypi, what's the benefit? To
> catch bugs and pinpoint when they we
pi project and in fact
> matches
> the installation experience provided by PyTorch. I don't think the benefit of
> not including "-f http://mxnet.s3.amazonaws.com/mxnet-cu101/nightly.html;
> matches the costs we currently externalize to the Pypi team.
>
> This suggestion se
@mxnet.incubator.apache.org
Cc: Kamakoti, Balaji
Subject: Re: Stopping nightly releases to Pypi
If we decide to do weekly pre-release builds to Pypi, what's the benefit? To
catch bugs and pinpoint when they were introduced, having weekly builds may be
too coarse. So people would likely prefer the nightly
racting a community that so far has been relying on the
> nightly builds in advance of stable.
>
> Sincerely,
>
> Alex Chung
>
>
> From: Lausen, Leonard
> Sent: Sunday, December 1, 2019 9:42 PM
> To: d...@mxnet.apache.org
&g
ems uncontroversial to me. Thus I would like to start lazy
consensus. If there are no objections, I will assume lazy consensus on stopping
nightly releases to Pypi in 72hrs.
Best regards
Leonard
that so far has been relying on the
nightly builds in advance of stable.
Sincerely,
Alex Chung
From: Lausen, Leonard
Sent: Sunday, December 1, 2019 9:42 PM
To: d...@mxnet.apache.org
Subject: Stopping nightly releases to Pypi
Hi MXNet Community,
since more
tches the costs we currently externalize to the Pypi team.
This suggestion seems uncontroversial to me. Thus I would like to start lazy
consensus. If there are no objections, I will assume lazy consensus on stopping
nightly releases to Pypi in 72hrs.
Best regards
Leonard
66 matches
Mail list logo
