Re: Global Search Now Available on MXNet Website

[Thomas DELTEIL]

Awesome job :+1:

Le mer. 20 mai 2020 à 22:36, Lin Yuan  a écrit :

> Awesome work! Thanks a lot for making this desirable feature happen.
>
> Lin
>
> On Wed, May 20, 2020 at 8:45 AM Yang Shi  wrote:
>
> > Hi MXNet Community,
> >
> > Global search feature is added to the main information pages of MXNet
> > website. It can search for contents across all site in any version.
> > Currently it is available on master website, and will be supported on
> v1.6
> > website shortly.
> >
> > Best regards,
> > Yang
> >
>

Re: new website, docs code freeze

[Thomas DELTEIL]

Update on the docs:

@Tao see comments on your PR for what needs to be updated in the docs for
the release.

Soji (thanks a lot) just contributed a big improvement to the docs. The doc
coverage is now at ~100%, which an improvement compared to the previous
website.
This should be taking care of all or most of the issues around missing
operators / packages / functions.
https://mxnet.apache.org/api/python/docs/api/index.html

Next improvements on the docs coming up:
- more broken links to be fixed, more missing tutorials to be added back
- usability fix: adding a direct link to the Python docs on the home page
(requested by Matthias Seeger, Haibin Lin)
- PR preview pipeline to be sorted out (Aaron is contributing a half-way
solution that would allow specific PRs to be deployed to the beta staging
env)

All the best,

Thomas


Le mar. 8 oct. 2019 à 19:40, Tao Lv  a écrit :

> I just sent out the announcement of 1.5.1 release for review. But still not
> sure how to change  http://mxnet.incubator.apache.org/get_started and
> http://mxnet.incubator.apache.org/get_started/download to accommodate.
>
> Can anyone help on that?
>
> thanks,
> -tao
>
> On Wed, Oct 9, 2019 at 10:24 AM Zhao, Patric 
> wrote:
>
> > Thanks, Thomas, it's good to have a site-wide search bar 😊
> >
> >
> >
> > FYI, the similar thing in https://pytorch.org/
> >
> >
> >
> > --Patric
> >
> >
> >
> > > -Original Message-
> >
> > > From: Thomas DELTEIL 
> >
> > > Sent: Wednesday, October 9, 2019 1:41 AM
> >
> > > To: dev@mxnet.incubator.apache.org
> >
> > > Subject: Re: new website, docs code freeze
> >
> > >
> >
> > > Hi Patric,
> >
> > >
> >
> > > The search bar is available in the python docs:
> >
> > > https://mxnet.apache.org/api/python/docs/api/ (on the top right).
> Since
> > the
> >
> > > homepage is not built by sphinx anymore there are no more search bar
> > there.
> >
> > > We are considering using an external plugin to maintain a site-wide
> > index and
> >
> > > provide better search experience than the sphinx one.
> >
> > > btw you were asking about the mkldnn tutorials, they are now here:
> >
> > > https://mxnet.apache.org/api/python/docs/tutorials/performance/backend
> >
> > > /mkldnn/index.html
> >
> > >
> >
> > > All the best,
> >
> > >
> >
> > > Thomas Delteil
> >
> > >
> >
> > > Le lun. 7 oct. 2019 à 19:58, Zhao, Patric  a
> > écrit :
> >
> > >
> >
> > > > I find there is no "search bar" in the website today.
> >
> > > >
> >
> > > > Could anyone check it?
> >
> > > >
> >
> > > > Thanks,
> >
> > > >
> >
> > > > --Patric
> >
> > > >
> >
> > > > > -Original Message-
> >
> > > > > From: Thomas DELTEIL 
> >
> > > > > Sent: Saturday, October 5, 2019 3:41 AM
> >
> > > > > To: dev@mxnet.incubator.apache.org
> >
> > > > > Subject: Re: new website, docs code freeze
> >
> > > > >
> >
> > > > > Hi Haibin,
> >
> > > > >
> >
> > > > > We are currently working with Soji on overhauling the way the
> python
> >
> > > > > docs are organized to get better and more consistent docs with full
> >
> > > > > coverage,
> >
> > > > the
> >
> > > > > current system is a brittle and hard to browse. We hope to finish
> >
> > > > > our dev work by tonight, ETA for early next week.
> >
> > > > > There is no ETA on bringing back the old docs, though that's the
> >
> > > > > next
> >
> > > > highest
> >
> > > > > priority feature on the list after improving the coverage of the
> >
> > > > > python
> >
> > > > API.
> >
> > > > >
> >
> > > > > All the best,
> >
> > > > >
> >
> > > > > Thomas Delteil
> >
> > > > >
> >
> > > > > On Fri, Oct 4, 2019, 12:34 Haibin Lin 
> > wrote:
> >
> > > > >
> >
> > > > > > Yes, that is the correct one.
> >
> > > > > >
> >
> > > > > > On a separate note, are we

Re: new website, docs code freeze

[Thomas DELTEIL]

Hi Patric,

The search bar is available in the python docs:
https://mxnet.apache.org/api/python/docs/api/ (on the top right). Since the
homepage is not built by sphinx anymore there are no more search bar there.
We are considering using an external plugin to maintain a site-wide index
and provide better search experience than the sphinx one.
btw you were asking about the mkldnn tutorials, they are now here:
https://mxnet.apache.org/api/python/docs/tutorials/performance/backend/mkldnn/index.html

All the best,

Thomas Delteil

Le lun. 7 oct. 2019 à 19:58, Zhao, Patric  a écrit :

> I find there is no "search bar" in the website today.
>
> Could anyone check it?
>
> Thanks,
>
> --Patric
>
> > -----Original Message-
> > From: Thomas DELTEIL 
> > Sent: Saturday, October 5, 2019 3:41 AM
> > To: dev@mxnet.incubator.apache.org
> > Subject: Re: new website, docs code freeze
> >
> > Hi Haibin,
> >
> > We are currently working with Soji on overhauling the way the python docs
> > are organized to get better and more consistent docs with full coverage,
> the
> > current system is a brittle and hard to browse. We hope to finish our dev
> > work by tonight, ETA for early next week.
> > There is no ETA on bringing back the old docs, though that's the next
> highest
> > priority feature on the list after improving the coverage of the python
> API.
> >
> > All the best,
> >
> > Thomas Delteil
> >
> > On Fri, Oct 4, 2019, 12:34 Haibin Lin  wrote:
> >
> > > Yes, that is the correct one.
> > >
> > > On a separate note, are we removing documentation versioning from the
> > > website? How do we switch between the master/nightly version and the
> > > stable version for the python API doc? Maybe there's a switch
> > > somewhere but I cannot find it.
> > >
> > > Also, I find that the API doc for many methods are missing, for
> > > example, the Dataset.transform function has detailed documentation on
> > > input and output types, but the doc only shows the one-line
> > > description of the method
> > >
> > >
> > https://mxnet.apache.org/api/python/docs/api/gluon/_autogen/mxnet.glu
> > o
> > > n.data.Dataset.html?highlight=dataset#
> > > .
> > > Same for other methods such as filter, shard, etc.
> > >
> > > Thanks.
> > >
> > > Best,
> > > Haibin
> > >
> > >
> > > On Thu, Oct 3, 2019 at 7:59 AM Aaron Markham
> > > 
> > > wrote:
> > >
> > > > Hi Haibin, you mean this one?
> > > >
> > > >
> > > https://github.com/apache/incubator-
> > mxnet/blob/master/docs/static_site
> > > /src/pages/api/faq/distributed_training.md
> > > > If so, it looks like a link update is needed.
> > > >
> > > > On Wed, Oct 2, 2019 at 9:42 PM Haibin Lin 
> > > > wrote:
> > > > >
> > > > > I find that the 'distributed training with KVStore' tutorial is
> gone.
> > > Are
> > > > > we adding it back?
> > > > >
> > > >
> > >
> > https://mxnet.apache.org/api/python/docs/tutorials/performance/index.h
> > > tml?highlight=distributed#distributed-training
> > > > >
> > > > >
> > > > > On Tue, Oct 1, 2019 at 4:54 AM Marco de Abreu
> > > > >  > > >
> > > > > wrote:
> > > > >
> > > > > > Thanks for the update, great job!
> > > > > >
> > > > > > Are the lighthouse results accessible somewhere? I'd be curious.
> > > > > >
> > > > > > -Marco
> > > > > >
> > > > > > Thomas DELTEIL  schrieb am Di., 1.
> Okt.
> > > > 2019,
> > > > > > 13:49:
> > > > > >
> > > > > > > Update on the website:
> > > > > > >
> > > > > > > Highlights:
> > > > > > > - Broken links: a good amount of them have been fixed in this
> > > #16284
> > > > > > > <https://github.com/apache/incubator-mxnet/pull/16284>,
> > > > > > > tutorials
> > > > have
> > > > > > > been
> > > > > > > reorganized (not yet deployed because of CI). A new 404 with
> > > > > > > quick
> > > > links
> > > > > > > has been added and deployed in #16287
> > 

Re: new website, docs code freeze

[Thomas DELTEIL]

@Marco Lighthouse Reports
https://cwiki.apache.org/confluence/display/MXNET/New+Website+Lighthouse+Reports,
forgot about that
@Haibin and others: WIP PR to re-reorganize and add full coverage of the
python docs API: https://github.com/apache/incubator-mxnet/pull/16377



Le ven. 4 oct. 2019 à 12:41, Thomas DELTEIL  a
écrit :

> Hi Haibin,
>
> We are currently working with Soji on overhauling the way the python docs
> are organized to get better and more consistent docs with full coverage,
> the current system is a brittle and hard to browse. We hope to finish our
> dev work by tonight, ETA for early next week.
> There is no ETA on bringing back the old docs, though that's the next
> highest priority feature on the list after improving the coverage of the
> python API.
>
> All the best,
>
> Thomas Delteil
>
> On Fri, Oct 4, 2019, 12:34 Haibin Lin  wrote:
>
>> Yes, that is the correct one.
>>
>> On a separate note, are we removing documentation versioning from the
>> website? How do we switch between the master/nightly version and the
>> stable
>> version for the python API doc? Maybe there's a switch somewhere but I
>> cannot find it.
>>
>> Also, I find that the API doc for many methods are missing, for example,
>> the Dataset.transform function has detailed documentation on input and
>> output types, but the doc only shows the one-line description of the
>> method
>>
>> https://mxnet.apache.org/api/python/docs/api/gluon/_autogen/mxnet.gluon.data.Dataset.html?highlight=dataset#
>> .
>> Same for other methods such as filter, shard, etc.
>>
>> Thanks.
>>
>> Best,
>> Haibin
>>
>>
>> On Thu, Oct 3, 2019 at 7:59 AM Aaron Markham 
>> wrote:
>>
>> > Hi Haibin, you mean this one?
>> >
>> >
>> https://github.com/apache/incubator-mxnet/blob/master/docs/static_site/src/pages/api/faq/distributed_training.md
>> > If so, it looks like a link update is needed.
>> >
>> > On Wed, Oct 2, 2019 at 9:42 PM Haibin Lin 
>> > wrote:
>> > >
>> > > I find that the 'distributed training with KVStore' tutorial is gone.
>> Are
>> > > we adding it back?
>> > >
>> >
>> https://mxnet.apache.org/api/python/docs/tutorials/performance/index.html?highlight=distributed#distributed-training
>> > >
>> > >
>> > > On Tue, Oct 1, 2019 at 4:54 AM Marco de Abreu <
>> marco.g.ab...@gmail.com>
>> > > wrote:
>> > >
>> > > > Thanks for the update, great job!
>> > > >
>> > > > Are the lighthouse results accessible somewhere? I'd be curious.
>> > > >
>> > > > -Marco
>> > > >
>> > > > Thomas DELTEIL  schrieb am Di., 1. Okt.
>> > 2019,
>> > > > 13:49:
>> > > >
>> > > > > Update on the website:
>> > > > >
>> > > > > Highlights:
>> > > > > - Broken links: a good amount of them have been fixed in this
>> #16284
>> > > > > <https://github.com/apache/incubator-mxnet/pull/16284>, tutorials
>> > have
>> > > > > been
>> > > > > reorganized (not yet deployed because of CI). A new 404 with quick
>> > links
>> > > > > has been added and deployed in #16287
>> > > > > <https://github.com/apache/incubator-mxnet/pull/16287>
>> > > > > - Broken Search: Search has been fixed in #16284
>> > > > > <https://github.com/apache/incubator-mxnet/pull/16284> and merged
>> > but
>> > > > not
>> > > > > deployed yet.
>> > > > > - Google index issues: new website seems to be indexed now, in
>> this
>> > > > google
>> > > > > search: "MXNet NDArray" <
>> > https://www.google.com/search?q=mxnet+ndarray>
>> > > > > new
>> > > > > website is the top result. Redirects for old links to new ones
>> have
>> > been
>> > > > > added to the .htaccess, thanks Soji for the help! #16342
>> > > > > <https://github.com/apache/incubator-mxnet/pull/16342> is still
>> > waiting
>> > > > to
>> > > > > be merged but a manual update to the website has been done in the
>> > > > > meanwhile.
>> > > > > - Automated analysis from google lighthouse scoring, compared to
>> the
>> > old
>> > 

Re: new website, docs code freeze

[Thomas DELTEIL]

Hi Haibin,

We are currently working with Soji on overhauling the way the python docs
are organized to get better and more consistent docs with full coverage,
the current system is a brittle and hard to browse. We hope to finish our
dev work by tonight, ETA for early next week.
There is no ETA on bringing back the old docs, though that's the next
highest priority feature on the list after improving the coverage of the
python API.

All the best,

Thomas Delteil

On Fri, Oct 4, 2019, 12:34 Haibin Lin  wrote:

> Yes, that is the correct one.
>
> On a separate note, are we removing documentation versioning from the
> website? How do we switch between the master/nightly version and the stable
> version for the python API doc? Maybe there's a switch somewhere but I
> cannot find it.
>
> Also, I find that the API doc for many methods are missing, for example,
> the Dataset.transform function has detailed documentation on input and
> output types, but the doc only shows the one-line description of the method
>
> https://mxnet.apache.org/api/python/docs/api/gluon/_autogen/mxnet.gluon.data.Dataset.html?highlight=dataset#
> .
> Same for other methods such as filter, shard, etc.
>
> Thanks.
>
> Best,
> Haibin
>
>
> On Thu, Oct 3, 2019 at 7:59 AM Aaron Markham 
> wrote:
>
> > Hi Haibin, you mean this one?
> >
> >
> https://github.com/apache/incubator-mxnet/blob/master/docs/static_site/src/pages/api/faq/distributed_training.md
> > If so, it looks like a link update is needed.
> >
> > On Wed, Oct 2, 2019 at 9:42 PM Haibin Lin 
> > wrote:
> > >
> > > I find that the 'distributed training with KVStore' tutorial is gone.
> Are
> > > we adding it back?
> > >
> >
> https://mxnet.apache.org/api/python/docs/tutorials/performance/index.html?highlight=distributed#distributed-training
> > >
> > >
> > > On Tue, Oct 1, 2019 at 4:54 AM Marco de Abreu  >
> > > wrote:
> > >
> > > > Thanks for the update, great job!
> > > >
> > > > Are the lighthouse results accessible somewhere? I'd be curious.
> > > >
> > > > -Marco
> > > >
> > > > Thomas DELTEIL  schrieb am Di., 1. Okt.
> > 2019,
> > > > 13:49:
> > > >
> > > > > Update on the website:
> > > > >
> > > > > Highlights:
> > > > > - Broken links: a good amount of them have been fixed in this
> #16284
> > > > > <https://github.com/apache/incubator-mxnet/pull/16284>, tutorials
> > have
> > > > > been
> > > > > reorganized (not yet deployed because of CI). A new 404 with quick
> > links
> > > > > has been added and deployed in #16287
> > > > > <https://github.com/apache/incubator-mxnet/pull/16287>
> > > > > - Broken Search: Search has been fixed in #16284
> > > > > <https://github.com/apache/incubator-mxnet/pull/16284> and merged
> > but
> > > > not
> > > > > deployed yet.
> > > > > - Google index issues: new website seems to be indexed now, in this
> > > > google
> > > > > search: "MXNet NDArray" <
> > https://www.google.com/search?q=mxnet+ndarray>
> > > > > new
> > > > > website is the top result. Redirects for old links to new ones have
> > been
> > > > > added to the .htaccess, thanks Soji for the help! #16342
> > > > > <https://github.com/apache/incubator-mxnet/pull/16342> is still
> > waiting
> > > > to
> > > > > be merged but a manual update to the website has been done in the
> > > > > meanwhile.
> > > > > - Automated analysis from google lighthouse scoring, compared to
> the
> > old
> > > > > website: Performance saw a > ~100% improvement, SEO saw a ~25%
> > > > improvement,
> > > > > and best practices improved by ~19%. Thanks Russell D. for running
> > the
> > > > > analysis.
> > > > >
> > > > > Remaining
> > > > > - *[high priority]* API docs are still missing some classes /
> > packages /
> > > > > methods. ETA for fix is EOW, root cause has been identified, we are
> > still
> > > > > deciding what's the best way forward for good discoverability as
> > well as
> > > > > good coverage and maintainability.
> > > > > - Adding quick links to directly access Python API docs on
> homepage.
> > > &g

Re: new website, docs code freeze

[Thomas DELTEIL]

Update on the website:

Highlights:
- Broken links: a good amount of them have been fixed in this #16284
<https://github.com/apache/incubator-mxnet/pull/16284>, tutorials have been
reorganized (not yet deployed because of CI). A new 404 with quick links
has been added and deployed in #16287
<https://github.com/apache/incubator-mxnet/pull/16287>
- Broken Search: Search has been fixed in #16284
<https://github.com/apache/incubator-mxnet/pull/16284> and merged but not
deployed yet.
- Google index issues: new website seems to be indexed now, in this google
search: "MXNet NDArray" <https://www.google.com/search?q=mxnet+ndarray> new
website is the top result. Redirects for old links to new ones have been
added to the .htaccess, thanks Soji for the help! #16342
<https://github.com/apache/incubator-mxnet/pull/16342> is still waiting to
be merged but a manual update to the website has been done in the meanwhile.
- Automated analysis from google lighthouse scoring, compared to the old
website: Performance saw a > ~100% improvement, SEO saw a ~25% improvement,
and best practices improved by ~19%. Thanks Russell D. for running the
analysis.

Remaining
- *[high priority]* API docs are still missing some classes / packages /
methods. ETA for fix is EOW, root cause has been identified, we are still
deciding what's the best way forward for good discoverability as well as
good coverage and maintainability.
- Adding quick links to directly access Python API docs on homepage.

All the best,

Thomas Delteil

Le mar. 24 sept. 2019 à 19:15, Thomas DELTEIL  a
écrit :

> @Philip Yes we're looking at link redirects for older links that might be
> hosted externally (using htaccess is my preferred way to handle it for now
> as you sugested) and we'll use a broken link checker to update the links
> that are hosted internally. We'll update the 404 to add an explanation on
> the website update. Google indexes will slowly update across the week so
> the google search issues will be less of a problem.
>
> If you find any such links yourself, or missing tutorials, please consider
> stepping up and helping fixing them. The more people get familiar with the
> new website architecture, the least likely it is to fall in a state of
> stalled updates like the previous one.
>
> For the sphinx issues in the python mini-website, missing API classes, if
> anybody is familiar with it, I'd love for us to bring back the automatic
> doc generation for each package so at least we have a list of all available
> classes in each sub package rather than relying on manual insertion of each
> class, which is brittle and not future proof. @Lin, Haibin
>  if you have experience with it, could we sync up
> offline on how you suggest to do that based on your gluon-nlp experience?
>
> @Marco, I'm currently traveling for ICDAR in Sydney, and Aaron is on PTO
> in Europe, I'll try make time today to help with the fixes since it is
> impacting a lot of users.
>
> In the meanwhile, any help is appreciated, and more than the value of the
> fixes, let me repeat that there is tremendous value in having more people
> familiar with the website build pipelines. Aaron is the main owner for the
> docs but he is already super busy with all his other responsibilities. I'm
> available to help if anybody is stuck. I believe Aaron has updated the
> READMEs on how to test the websites locally, if they're not clear, feel
> free to contribute your own explanations or ask for help directly to me by
> email or on the discuss forum.
>
> Good hunting!
>
> Thomas
>
>
>
> Le mer. 25 sept. 2019 à 10:10, Marco de Abreu  a
> écrit :
>
>> Good catch, Mu! Also good idea, Philip!
>>
>> Aaron and Thomas, are you going to work on this?
>>
>> -Marco
>>
>> On Wed, Sep 25, 2019 at 1:28 AM Mu Li  wrote:
>>
>> > The questions I found are:
>> >
>> > 1. Not ever page contains, especially the homepage
>> >
>> >
>> http://mxnet.incubator.apache.org/api/python/docs/_static/google_analytics.js
>> > 2. The correct tracking id is UA-96378503-1 instead of UA-96378503-11 in
>> >
>> >
>> http://mxnet.incubator.apache.org/api/python/docs/_static/google_analytics.js
>> >
>> > On Tue, Sep 24, 2019 at 4:23 PM Mu Li  wrote:
>> >
>> > > I think the reason is that the google tracker is not included in the
>> new
>> > > website.
>> > >
>> > > On Tue, Sep 24, 2019 at 4:17 PM Marco de Abreu <
>> marco.g.ab...@gmail.com>
>> > > wrote:
>> > >
>> > >> Hello,
>> > >>
>> > >> I checked the Google Analytics s

Re: new website, docs code freeze

[Thomas DELTEIL]

@Philip Yes we're looking at link redirects for older links that might be
hosted externally (using htaccess is my preferred way to handle it for now
as you sugested) and we'll use a broken link checker to update the links
that are hosted internally. We'll update the 404 to add an explanation on
the website update. Google indexes will slowly update across the week so
the google search issues will be less of a problem.

If you find any such links yourself, or missing tutorials, please consider
stepping up and helping fixing them. The more people get familiar with the
new website architecture, the least likely it is to fall in a state of
stalled updates like the previous one.

For the sphinx issues in the python mini-website, missing API classes, if
anybody is familiar with it, I'd love for us to bring back the automatic
doc generation for each package so at least we have a list of all available
classes in each sub package rather than relying on manual insertion of each
class, which is brittle and not future proof. @Lin, Haibin
 if you have experience with it, could we sync up
offline on how you suggest to do that based on your gluon-nlp experience?

@Marco, I'm currently traveling for ICDAR in Sydney, and Aaron is on PTO in
Europe, I'll try make time today to help with the fixes since it is
impacting a lot of users.

In the meanwhile, any help is appreciated, and more than the value of the
fixes, let me repeat that there is tremendous value in having more people
familiar with the website build pipelines. Aaron is the main owner for the
docs but he is already super busy with all his other responsibilities. I'm
available to help if anybody is stuck. I believe Aaron has updated the
READMEs on how to test the websites locally, if they're not clear, feel
free to contribute your own explanations or ask for help directly to me by
email or on the discuss forum.

Good hunting!

Thomas



Le mer. 25 sept. 2019 à 10:10, Marco de Abreu  a
écrit :

> Good catch, Mu! Also good idea, Philip!
>
> Aaron and Thomas, are you going to work on this?
>
> -Marco
>
> On Wed, Sep 25, 2019 at 1:28 AM Mu Li  wrote:
>
> > The questions I found are:
> >
> > 1. Not ever page contains, especially the homepage
> >
> >
> http://mxnet.incubator.apache.org/api/python/docs/_static/google_analytics.js
> > 2. The correct tracking id is UA-96378503-1 instead of UA-96378503-11 in
> >
> >
> http://mxnet.incubator.apache.org/api/python/docs/_static/google_analytics.js
> >
> > On Tue, Sep 24, 2019 at 4:23 PM Mu Li  wrote:
> >
> > > I think the reason is that the google tracker is not included in the
> new
> > > website.
> > >
> > > On Tue, Sep 24, 2019 at 4:17 PM Marco de Abreu <
> marco.g.ab...@gmail.com>
> > > wrote:
> > >
> > >> Hello,
> > >>
> > >> I checked the Google Analytics statistics and the launch of the new
> > >> website reduced the traffic by over 80%:
> > >>
> > >> [image: image.png]
> > >>
> > >> (Please let me know if the image is not visible)
> > >>
> > >> How shall we handle this?
> > >>
> > >> Best regards,
> > >> Marco
> > >>
> > >> On Mon, Sep 23, 2019 at 7:30 AM Zhao, Patric 
> > >> wrote:
> > >>
> > >>> For the install page [1], I suggest to add the selection of backend
> > >>> DeepNumpy [2] which will be more clean.
> > >>>
> > >>> [1] http://mxnet.incubator.apache.org/index.html
> > >>> [2] https://numpy.mxnet.io/#installation
> > >>>
> > >>>
> > >>>
> > >>> > -Original Message-
> > >>> > From: kellen sunderland 
> > >>> > Sent: Monday, September 23, 2019 12:47 PM
> > >>> > To: dev@mxnet.incubator.apache.org
> > >>> > Subject: Re: new website, docs code freeze
> > >>> >
> > >>> > New site looks good.  I do notice that a few tutorials from the old
> > >>> site are
> > >>> > missing (for example the TensorRT tutorial).  Any plans to bring
> them
> > >>> back?
> > >>> >
> > >>> > On Sun, Sep 22, 2019 at 10:04 AM Haibin Lin <
> > haibin.lin@gmail.com>
> > >>> > wrote:
> > >>> >
> > >>> > > Another issue I found with the current website: the Sphinx object
> > >>> > > inventory
> > >>> > > <https://www.sphinx-
> > >>> > doc.org/en/master/usage/extensions/intersph

Re: new website, docs code freeze

[Thomas DELTEIL]

Thanks all for the feedback,

We'll send an email next week with the list of missing features, content
and bugs that we plan to fix.
We took the option of releasing early, with some features missing, rather
than trying to be at feature parity with the old website before launching
the website.
The reason why we decided to do that is two-fold:
- playing catch-up with docs in master introduce daily conflicts that need
to be resolved and introduce opportunity for errors
- by releasing early, we can take advantage of the community contributions
in modifying whatever the community feels like a better way of doing things.

One of the goals of the new website was to disentangle the main website,
now called "static_site" to the auto-generated docs. Now the overall site
is made of a main static site, with easy to modify content and easy to
understand architecture for anybody familiar with basic html, and a
collection of mini-websites for each language bindings that can be built in
isolation and that are self-contained. Actually the new CI jobs builds all
of them in parallel independently.

There is PLENTY of room for improvement, it would be great if the community
can help contribute to bring the new website at the same level of content
richness as the old one, and then even further.

Missing features:
- As pointed by Haibin, the API docs do not have the full list of operators
and classes. There is a mix of auto-generated docs based on packages, and
some docs that are spelled out manually to improve the logical organization
of the package where there is a need. The drawback with manually listed
classes in a package is that it's very easy to miss some. If someone wanted
to build a sanity check that would automatically detect which classes are
not in the documentation, or if someone knew how to enable that with
sphinx, that would be a great addition to the python docs
- There is missing content in the python tutorials, and the discoverability
could be improved. Some old tutorials have not been migrated just yet.
- The nightly tests on tutorials have been disabled for now
- There is no "Download jupyter notebook" for tutorials just yet.
- Non-python tutorials might benefit from a blurb description and a better
content organization.
- Python tutorials could be better organized, have a picture accompanying
their description
- There is no site-wide search, this is not an easy problem to solve to be
fair given the static nature of the website, but maybe an external plugin
might be able to give a half-way solution
- There is no version selector for the docs
- There is bug in search box of the python docs, but this is just a small
JS bug that can be fixed easily (on my list for next week)
- Most old links have not had a redirect put in place.

We'll formalize this in github issues next week, but they are all fairly
small and helping out on these would be a great way of familiarizing
yourself with the new website build system and website architecture.

 Thanks all for the feedback, please keep it coming!

Thomas Delteil

Le sam. 21 sept. 2019 à 09:53, Haibin Lin  a
écrit :

> It looks like my previous email did not go through. Re-sending:
>
> Hi Aaron,
>
> The website looks cool. Thanks for pushing this to production. A few
> questions:
>
> - I was looking for the API doc for mx.sym.dot, but I find that most
> operators under mx.sym.* are missing. Is this expected?
> - I was also checking the search functionality, searching the keyword
> "ndarray" only returns one result "mxnet.ndarray.NDArray", which doesn't
> seem right. There animation keeps going (Searching. -> Searching.. ->
> Searching ...) and gives me an impression that the search is never
> completely done(?).
>
> Best,
> Haibin
>
>
> On Fri, Sep 20, 2019 at 4:50 PM Chaitanya Bapat 
> wrote:
>
> > Thanks Aaron and the team for launching new website!
> >
> > 1. There's no search button anywhere on the landing page.
> > 2. I wasn't able to find FAQ (and without search button I dont have
> option
> > but to go manually on each menu). Only when I go to Docs&Tutorials -> FAQ
> > -> Extend and Cotribute (that I got what I wanted).
> >
> > Suggestions
> > Might want to make this searchable and pop FAQ on the main page (or
> > somewhere prominent)
> >
> > Thanks,
> > Chai
> >
> >
> > On Fri, 20 Sep 2019 at 14:58, Przemysław Trędak 
> > wrote:
> >
> > > There seems to be a problem with (at least Python, did not check
> others)
> > > APIs. For example this page:
> > >
> > >
> >
> https://mxnet.incubator.apache.org/api/python/docs/api/symbol/_autogen/mxnet.symbol.Symbol.argmax.html
> > >
> > > says that it is a convenience method for argmax (with a

Re: Developer tools for MXNet

[Thomas DELTEIL]

Thanks Pedro! Great resource for debugging the MXNet engine. In the future
let's try to avoid fragmentation of resources and post videos to the Apache
MXNet youtube channel. https://www.youtube.com/apachemxnet

All the best,

Thomas

Le jeu. 28 mars 2019 à 14:08, Pedro Larroy  a
écrit :

> Hi developers!
>
> We did a session on working with developer tools for debugging and
> extending the MXNet engine in CLion and posted the screencast in Vimeo
> in case you are interested.
>
> https://vimeo.com/326697272
>
> I also added it to the wiki.
>
> Pedro.
>

Re: Embedded World 2019 Robotics Demo

[Thomas DELTEIL]

Just tweeted two videos about it:
https://twitter.com/thdelteil/status/1101012275991764993
https://twitter.com/thdelteil/status/1101012034051727360
We will write a blog post or two about the project when Anton, Pavel and
the others are back from the conference to give more details :)

To summarize and as can be inferred from the videos, two robotic arms are
moving according to the position of the wrists of the player. The pose of
the player is detected using a pose estimation neural network based on the
implementation of Simple Baselines for Pose Estimation paper
<http://openaccess.thecvf.com/content_ECCV_2018/html/Bin_Xiao_Simple_Baselines_for_ECCV_2018_paper.html>that
is going to be released in the next version of gluon-cv, shout out to Zhi
and He Tong for providing the training code. Another classifier detects the
hand as open or closed and that decides the position of the grips. The goal
of the game is to take a book from one side and drop it in the bucket on
the other side by passing it between the two arms. The robots are from
Nyrio, a French startup.

To answer your question Isabel, this project was a joint cooperation
between a few MXNet team members at AWS, including Anton, Pavel and myself
and some employees at the QT (the C++ library) company, in their industrial
automation department.

All the best,

Thomas Delteil

Le mer. 27 févr. 2019 à 22:27, Junru Shao  a
écrit :

> Hi Anton,
>
> Would love to know more about this great demo! Will you guys share video
> clips?
>
> Cheers,
> Junru
>
> On Wed, Feb 27, 2019 at 8:11 PM Isabel Drost-Fromm 
> wrote:
>
> > Hi,
> >
> > Sounds interesting. Can you share more on what you are showing and what
> > role mxnet plays?
> >
> > Also, who's that "we" behind "our booth"?
> >
> >
> > Have fun in Nürnberg,
> > Isabel
> >
> > Am 27. Februar 2019 11:38:57 MEZ schrieb Anton Chernov <
> > mecher...@gmail.com>:
> > >Dear MXNet Community,
> > >
> > >If you happens to be at the Embedded World exhibition in Nürnberg drop
> > >by
> > >our booth at the Qt stand in hall 4 to see a MXNet robotics demo.
> > >
> > >Looking forward to see you!
> > >
> > >Best
> > >Anton
> >
> > --
> > Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.
> >
>

Re: [Announcement] New Committer -- Steffen Rochel

[Thomas DELTEIL]

Welcome Steffen!

On Mon, Feb 4, 2019, 15:55 Marco de Abreu  Welcome!
>
> Am Di., 5. Feb. 2019, 00:45 hat Chris Olivier 
> geschrieben:
>
> > Dear Community:
> >
> > Please join me to welcome Steffen Rochel (steffenroc...@gmail.com) as a
> > new
> > committer of Apache (incubating) MXNet!
> >
> > Steffen has played a role in nearly every MXNet release in the past 18
> > months, managed several of the wiki pages and has contributed in
> expanding
> > the community by managing and hosting meetups in different parts of the
> > world.
> >
> > -Chris
> >
>

Re: [Announcement] New committer: Thomas Delteil

[Thomas DELTEIL]

Thanks Marco and PMC members, looking forward to the road ahead.

All the best,

Thomas Delteil

Le mar. 20 nov. 2018 à 14:40, Marco de Abreu
 a écrit :

> The Project Management Committee (PMC) for Apache MXNet (incubating)
> has invited Thomas Delteil to become a committer and we are pleased
> to announce that he has accepted.
>
> Thomas, thanks a lot for your helpful contributions and we are looking
> forward to
> working with you on further projects!
>
> Best regards,
> Marco de Abreu
> - On behalf of the Apache MXNet (incubating) PMC
>

Re: [VOTE] Separating PMC and Committership

[Thomas DELTEIL]

+1 (non-binding)

Le jeu. 8 nov. 2018 à 10:04, Carin Meier  a écrit :

> Reminder - Vote ends tomorrow-  Friday Nov 9th at 6:00 am EST
>
> On Mon, Nov 5, 2018 at 11:29 AM Carin Meier  wrote:
>
> > This is a procedural vote on whether to separate the committer and PPMC
> > levels in the project. The current state is that a user is considered as
> > both a committer and a PPMC member at the same time. This vote is to
> change
> > that to be able to invite a person in as a committer separately from a
> PPMC
> > member.
> >
> > Document reference:
> >
> https://cwiki.apache.org/confluence/display/MXNET/Become+an+Apache+MXNet+%28incubating%29+Committer+and+PPMC+Member
> >
> > Discussion thread:
> >
> https://lists.apache.org/thread.html/9c6ecda02e081aa6b689c92badc9dcf05ced6fb3691fd370471773d1@%3Cdev.mxnet.apache.org%3E
> >
> > The vote will be a procedural issue vote as defined
> > https://www.apache.org/foundation/voting.html
> >
> > Votes on procedural issues follow the common format of majority rule
> unless
> > otherwise stated. That is, if there are more favourable votes than
> > unfavourable ones, the issue is considered to have passed -- regardless
> of
> > the number of votes in each category. (If the number of votes seems too
> > small to be representative of a community consensus, the issue is
> typically
> > not pursued. However, see the description of lazy consensus
> >  for a
> > modifying factor.)
> >
> > The vote will run until Friday Nov 9th at 6:00 am EST
> >
> > Thanks,
> > Carin
> >
> >
> >
>

Re: [VOTE] Release MXNet version 1.3.0.RC0

[Thomas DELTEIL]

-0
(non-binding)

If I may add some nuancing plus a personal data point as one of the users
commenting in the bug report in question:

- Performance vs. Basic functionality => I don't think high performance
use-cases and basic functionality are two obviously opposed concepts and
see no contradiction in Hagay's and Sandeep's statements.
Float16 support is feature of MXNet that provides more than twice the
performance of Float32 on supported platforms, hence the high performance
use-case. The bug is that the basic functionality of reloading a saved
float16 models is currently broken.

- This bug vs Other bugs => Contrary the vast majority of the 140 open bugs
that are mentioned above, I would put to Sandeep's credit that this one bug
has a PR open that provides a fix for it. This would make it a better
candidate to get included in this release than a bug that has no fix ready
for it.

- Personal datapoint: I recently did some experimentation with float16 [1]
and actually coincidentally just published a video on optimizing
performance for Gluon. Float16 conversion is one of the most, if not the
most effective way to get performance out of MXNet [2]. I believe there is
a lot of value in publicizing more its use and hence making sure at least
the basic support for normal use-cases is present.

Of course this needs to be balanced with the overhead of preparing a new
release candidate once the fixed is reviewed and merged, which seems to be
a lengthy and complex process in its own right, and the delay with
providing the other features present in 1.3 for users that are not running
off the nightly builds.

All the best,

Thomas

[1] https://github.com/ThomasDelteil/PerformanceTricksMXNetGluon
[2]
https://www.youtube.com/watch?v=Cqo7FPftNyo&t=0s&list=PLkEvNnRk8uVk6U515Pj-jHQUxFC4eDi3m

Le mar. 4 sept. 2018 à 17:11, Sheng Zha  a écrit :

> Sandeep,
>
> Thanks for explaining your veto. We have open bugs that impacted a lot more
> than just 3 customers, just by referring to the number of commenters on the
> issue [1].
>
> You said that this is for "high performance use cases", which contradicts
> with Hagay's assement that this is "basic functionality broken". Given that
> this is for advanced use cases of using half-precision training, why is it
> so much more important than any other open bug reports, that for this
> specific bug fix, we have to delay the access of regular users to the new
> MXNet 1.3 release by at least another week?
>
> Honestly, I'm concerned that your vote is biased by Amazon involvement,
> given that you quoted Amazon Rekognition.
>
> -sz
>
> [1]
>
> https://github.com/apache/incubator-mxnet/issues?q=is%3Aissue+is%3Aopen+label%3ABug+sort%3Acomments-desc
>
> On Tue, Sep 4, 2018 at 4:51 PM sandeep krishnamurthy <
> sandeep.krishn...@gmail.com> wrote:
>
> > My initial vote of “-0” was due to lack of info from a user who had said,
> > he overcame this issue for FP16 model.
> >
> >
> > However, suggested workaround [1] for the issue is not straight forward
> and
> > generally usable for all users. Also, issue is not simple and isolated to
> > be listed in the Release Notes as known issue with a workaround.
> >
> >
> > Changing my vote to: "-1 (binding)" owing to the user impact [3]
> >
> >
> >
> > @Sheng:
> >
> > 1. Agreed, bug existed from long time. However, FP16 and such
> optimizations
> > were added later on. Followed by users [2] using this feature for high
> > performance use cases. It is not ok to measure severity of the bug based
> on
> > its past existence, rather we can see who is impacted now and is it a
> small
> > subset with a simple workaround or large user impacting issue.
> >
> > 2. Agreed bug was reported 7/21. However, I became aware of this issue on
> > 08/29 and submitted the fix on 08/30. Also, I did bring this to the
> notice
> > of community, you and 1.3 release manager (Roshani) on the RC0 proposal
> > thread. Also, I would focus on the issue and user impact than who
> > identified and who is fixing the issue.
> >
> >
> > Based on my discussion with 2 users, I think it is a important feature
> for
> > them to see in Apache MXNet v1.3.0.
> >
> >
> >
> > Best,
> >
> > Sandeep
> >
> >
> > [1] Workaround used by the user.
> >
> >
> > net_fp16 = mx.gluon.SymbolBlock.imports('resnet34_fp16-symbol.json',
> > ['data'])
> >
> > params_fp16 = mx.nd.load('resnet34_fp16-.params')
> >
> >
> > for k, v in params_fp16.items():
> >
> > new_key = k.split(':')[1]
> >
> > net_fp16.collect_params()[new_key].cast(v.dtype)
> >
> >
> > net_fp16.collect_params().load('resnet34_fp16-.params', ctx)
> >
> >
> > [2] Amazon Rekognition
> >
> >
> > [3] User story: Train a model -> Cast it to FP16 -> Save the model ->
> Load
> > back the model does not work. They have to cast every parameter with a
> > workaround mentioned above [1].
> >
> > On Tue, Sep 4, 2018 at 4:14 PM Hagay Lupesko  wrote:
> >
> > > Hi Sheng,
> > >
> > > Addressing your questions:
> > >
> > > - "why this specific bug i

Re: Allow SSL Verification to be off in mx.gluon.utils.download?

[Thomas DELTEIL]

Agree that we should never push code that has a download with the flag
disabled. But I don't see a problem having a flag to disable ssl
verification if users want to put themselves at risk. I don't think à
warning is necessary as long as the API wording is scary enough.

All the best,

Thomas

On Wed, Jul 4, 2018, 06:50 kellen sunderland 
wrote:

> I'd agree with Sheng and Pedro.  I would also not put a warning message in
> place when the function is explicitly called with SSL verification turned
> off.  I would assume if the code author intentionally disables verification
> that the message being displayed would not provide value.
>
> -Kellen
>
>
> On Wed, Jul 4, 2018 at 3:42 PM Pedro Larroy 
> wrote:
>
> > Agree with Sheng. Not always a website has trusted SSL cert, and you
> might
> > still want to download cat and elephant pictures from it. (I checked some
> > usages of this function).
> >
> > On Wed, Jul 4, 2018 at 9:47 AM Marco de Abreu
> >  wrote:
> >
> > > Thanks for raising this issue Sheng.
> > >
> > > My proposal would be to always print a warning message when this
> function
> > > is called with the ssl check disabled. This functionality would be
> tested
> > > by a unit test which mocks the network access.
> > >
> > > Additionally, I'd like to propose that we set a policy for ourselves
> that
> > > we as MXNet community never submit any code that has this flag disabled
> > and
> > > rather ensure that the servers we are using are properly secured with
> > > correct ssl certificates.
> > >
> > > -Marco
> > >
> > > Sheng Zha  schrieb am Mi., 4. Juli 2018, 08:58:
> > >
> > > > Hi,
> > > >
> > > > This is a follow-up discussion from PR-11546
> > > > <
> > > >
> > >
> >
> https://github.com/apache/incubator-mxnet/pull/11546#pullrequestreview-134215477
> > > > >
> > > > per
> > > > suggestion from Marco. The proposed approach is to add an option to
> > allow
> > > > users who call the download function to explicitly turn off ssl
> > > > verification. The default behavior is unchanged (i.e. always verify).
> > > From
> > > > the comments so far:
> > > >
> > > > Pros:
> > > > Users can use this function to download from trusted links that don't
> > > have
> > > > proper ssl cert set-up, only by disabling this option explicitly.
> > Without
> > > > this option, the download function cannot be used in such case.
> > > >
> > > > Cons:
> > > > Vulnerable to MITM when disabled.
> > > >
> > > > My take on this is that having such option is better, since download
> > > > function can be useful in more scenarios. I'd like to hear from
> others
> > if
> > > > there are scenarios that this approach is absolutely not acceptable.
> > > > Thanks.
> > > >
> > > > -sz
> > > >
> > >
> >
>

Re: users@mxnet

[Thomas DELTEIL]

I was actually the one stating that we didn't need a user mailing list
during the Seattle meetup, given all the reasons already exposed above.

However given what proponents of a mailing list said, I personally wouldn't
mind adding a new channel as a user mailing list, and monitoring it. There
seems to be a subset of users, used to apache projects, that wouldn't use
the forum but would use a mailing list. Though I think it is not as
feature-rich as the forum and there is a risk of dilution of information.
It is more about reaching those extra users. If we see a dilution of
traffic on the forum towards the mailing list (~currently 100 posts/week)
then maybe we can reconsider our assumptions?

All the best,

Thomas Delteil

On Mon, Jun 18, 2018, 17:30 Pedro Larroy 
wrote:

> I agree with Tianqi, Eric and others. We shouldn't dilute the community
> with another forum. Disqus is already working and has healthy
> participation, you can get an email digest if you so desire. Subscribing to
> a mailing list to get a question answered is quite a heavyweight investment
> for many people and users who might not have the resources nor mental
> bandwidth to receive more email volume in their inboxes.
>
> On Mon, Jun 18, 2018 at 10:19 AM Tianqi Chen 
> wrote:
>
> > The problem of having multiple separate channels of communication is that
> > users get confused, and the cost of maintenance goes up(people have to
> > watch both). As the current community was at discuss forum and many users
> > prefer it, having a mail-list is only a burden we will bring
> >
> > Tianqi
> >
> > On Mon, Jun 18, 2018 at 9:48 AM, Jim Jagielski  wrote:
> >
> > > IMO, that is the wrong way to look at it.
> > >
> > > A users@ mailing list is a great, easy, low-cost and low-overhead way
> of
> > > *increasing* the user community and providing an extra level of
> support.
> > > Unless there is "strong evidence" that this is NOT the case, I would
> > > recommend we create the list.
> > >
> > > > On Jun 16, 2018, at 12:28 AM, Tianqi Chen 
> > > wrote:
> > > >
> > > > So unless there is a strong evidence that our community users prefers
> > the
> > > > mail-list, I would recommend we keep the current way
> > > >
> > > > Tianqi
> > > >
> > > > On Fri, Jun 15, 2018 at 9:25 PM, Sergio Fernández  >
> > > wrote:
> > > >
> > > >> Are we targeting just Seattle as our community? I really hope we are
> > > >> thinking a bit beyond that...
> > > >>
> > > >> On Fri, Jun 15, 2018, 21:22 Tianqi Chen 
> > > wrote:
> > > >>
> > > >>> I remember last time during the mxnet meetup in Seattle, we did a
> > > survey,
> > > >>> and most users preferred the current discuss forum. So I would say
> we
> > > >> stick
> > > >>> with that given the user community prefers that
> > > >>>
> > > >>> Tianqi
> > > >>>
> > > >>> On Fri, Jun 15, 2018 at 9:13 PM, Sergio Fernández <
> wik...@apache.org
> > >
> > > >>> wrote:
> > > >>>
> > > >>>> Then, if everybody agree, let's request the mailing list creation
> to
> > > >>> INFRA
> > > >>>> ;-)
> > > >>>>
> > > >>>> Marco, I wouldn't do that. Typically developers are also
> subscribed
> > > >>> there,
> > > >>>> since they may be the most informed people for answering users'
> > > >>> questions.
> > > >>>> But the topics discussed there may not be of the interest for pure
> > > >>>> development purposes. Some discussions will jump from users@ to
> > dev@,
> > > >>> but
> > > >>>> at a different level. So I wouldn't forward one mailing list to
> the
> > > >>> other.
> > > >>>>
> > > >>>> On Fri, Jun 15, 2018, 21:01 Marco de Abreu
> > > >>>>  wrote:
> > > >>>>
> > > >>>>> I think nobody was opposed to it in the past, right?
> > > >>>>>
> > > >>>>> I'd propose that all emails automatically get copied to dev@ to
> > > >> ensure
> > > >>>>> high
> > > >>>>> visibility initially. What do you think?
> > > >>>>&

Re: Reverting pull request

[Thomas DELTEIL]

Copying a comment I made on a flaky test introduced by this PR:
https://github.com/apache/incubator-mxnet/issues/11171

"

@piiswrong  you introduced this test in this
commit

[WIP] Do Not Merge. Static memory allocation for cached_op (#10817
) 2dbd143


and it seems to be flaky. I have seen it failing a few times in recent
builds. Can you take a look? e.g
http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/master/1002/pipeline/

Given all the talk about quality contributions going on the dev mailing
list, I am a little unsettled by the fact this PR went undocumented (no
design docs or explanation in the PR), unreviewed (1 question ignored), the
optimization wasn't tested or benchmarked (it was actually making things
slower), and the code was self-merged.

Should we enforce that each PR , especially the ones that introduce a
significant number of changes be properly documented and reviewed before
merging?

"

My main gripe is that there is literally no description of what the PR is
achieving, no design docs to refer to.

I spent time benchmarking the different optimization because I thought I
did something wrong when I found that it was slower with the optimization.
I would have hoped that a significant change like that would have at least
been benchmarked.

In French we say "Il ne faut pas confondre vitesse et precipitation" which
loosely translates to "Haste makes waste".

I would advocate to take the time to document PRs properly, benchmark and
thoroughly test significant changes. Because down the line, other users end
up losing so much more time trying to figure out what is happening when
things go wrong (like here).

Thanks,


Thomas

2018-06-15 14:27 GMT-07:00 Marco de Abreu <
marco.g.ab...@googlemail.com.invalid>:

> If it causes issues, I'd like to invite everybody to direct their requests
> to Eric since he merged the PR prematurely. The committer who merges a PR
> is responsible and can be held liable for any negative impact being the
> result of their action [1].
>
> [1]: https://www.apache.org/dev/committers.html#committer-responsibilities
>
> On Fri, Jun 15, 2018 at 2:23 PM Zheng, Da 
> wrote:
>
> > +1 The PR has been merged a while ago, so it has been tested by many
> > people.
> > Other people's work now depends on this PR. Reverting it at this point
> can
> > cause a lot of problems for many other people.
> >
> > Best,
> > Da
> >
> > On 6/15/18, 2:18 PM, "workc...@gmail.com on behalf of Tianqi Chen" <
> > workc...@gmail.com on behalf of tqc...@cs.washington.edu> wrote:
> >
> > +1   We would be stuck at local minimums if we just keep reverting
> the
> > PR
> > that brings improvements in the long term
> >
> > Tianqi
> >
> > On Fri, Jun 15, 2018 at 2:15 PM, Mu Li  wrote:
> >
> > > Why reverting instead of fixing the bugs? Static memory aims to
> > reduce
> > > memory allocation, it's a key feature to bridge the perf gap
> between
> > gluon
> > > and symbol.
> > >
> > > On Fri, Jun 15, 2018 at 2:06 PM, Marco de Abreu <
> > > marco.g.ab...@googlemail.com.invalid> wrote:
> > >
> > > > Hello,
> > > >
> > > > I'm reverting https://github.com/apache/
> incubator-mxnet/pull/10817
> > as of
> > > > https://github.com/apache/incubator-mxnet/pull/11311 due to
> > regressions
> > > > described in
> > https://github.com/apache/incubator-mxnet/issues/11171 and
> > > > https://github.com/apache/incubator-mxnet/pull/10817.
> > > >
> > > > The pull request has been self-merged without proper review and
> > > introduced
> > > > regressions. Committers should act as role models in this project
> > and
> > > > adhere to software engineer best practices.
> > > >
> > > > Best regards,
> > > > Marco
> > > >
> > >
> >
> >
> >
>

Re: Feature branches for ARM and Android

[Thomas DELTEIL]

Hi Pedro,

Is there a problem in working off a branch in your own fork and issue a
[WIP] PR ? This is a pattern I have seen a lot and personally I think it
works well, since it also gives some visibility if someone is interested in
looking at the progress of the work. You can add people collaborating with
you as collaborator to your own fork and that way your commits will be run
against the CI. Make sure to merge from apache/master and not larroy/master
if you have conflicts? Not sure why you got these conflicts otherwise.

All the best,

Thomas

2018-06-12 23:39 GMT-07:00 Pedro Larroy :

> Thanks a lot for creating these branches and proposing the idea, for the
> reasons you listed.
>
>
>  We tried during this week to work with these branches with @lebeg for
> Android and Arm support, for the reasons listed below these branches are
> not useful for us, so you can delete them.
>
> 1. We don't have permissions to commit to these development branches,
> 2. they show merge conflicts that have been solved locally before running
> CI (?). I'm pretty sure I merged and resolved conflicts locally. 3. It
> would also pollute the repository history with continuous merges to and
> from these branches. I prefer to have a linear history in master so
> changes, regressions and bisecting can be less painful when dealing with
> issues.
>
> I think is important to share development and integrate small, incremental
> patches towards architecture support, unfortunately these branches can't
> help us at this stage. We will share our work through a different means and
> without polluting the project with additional branches which are not meant
> for production or general use.
>
>
>
>
> On Mon, Jun 11, 2018 at 6:20 AM Marco de Abreu <
> marco.g.ab...@googlemail.com>
> wrote:
>
> > The problem with regular reviews here is that we might want to keep
> > temporary code or hacks as a temporary solution before we finalize it. A
> > regular review would have problems with that.
> >
> > The reason against a fork is the requirement of CI. Since multiple people
> > are working on the same branch and we have to file PRs against each
> other,
> > it would cause problems if CI is only triggered after the fact.
> >
> > Ideally, the branch will be in a good state and wouldn't need many
> chances
> > to be mergeable for master.
> >
> > Naveen Swamy  schrieb am So., 10. Juni 2018, 10:46:
> >
> > > I suggest you do a regular review and not a pass-through review for
> these
> > > branches as well. It would hard to manage a massive review at the end,
> if
> > > every commit to the branch goes with a proper PR, you could just merge
> to
> > > the master when its ready.
> > >
> > > If you want an experimental branch, why not just work it off of a fork?
> > >
> > >
> > > On Thu, Jun 7, 2018 at 5:32 PM, Marco de Abreu <
> > > marco.g.ab...@googlemail.com
> > > > wrote:
> > >
> > > > Hello,
> > > >
> > > > we are currently revisiting our setup for ARM and Android. Since
> we're
> > > not
> > > > in a stable state but need some way to collaborate while having
> support
> > > > from CI, we proposed the usage of feature branches. Thus, I have
> create
> > > the
> > > > two branches devel-arm and devel-android into which Anton and Pedro
> are
> > > > going to submit their PRs.
> > > >
> > > > @Reviewers: It'd be great if you could assist with the reviews.
> Please
> > > note
> > > > that the reviews for these branches can be less harsh, thus temporary
> > > > solutions and commented code are totally acceptable. We will do a
> final
> > > > review after the acceptance criteria (successful cross-compilation
> and
> > > > manual test execution on a physical device) are fulfilled. This is
> > > > necessary because we currently have no CI coverage and would like to
> > keep
> > > > these changes isolated from master until we found a stable solution.
> > > >
> > > > Best regards,
> > > > Marco
> > > >
> > >
> >
>

Re: Vote to stop using JIRA

[Thomas DELTEIL]

+0.5

Thanks Timur for the constructive feedback!

For me the problem is exactly as Anirudh raised, I am not a big fan of it
but I don't think JIRA has been given a fair shot. The problem lies not so
much with the tracking platform but rather with the contributors following
the project management guidelines.

All the best,

Thomas


2018-06-08 17:05 GMT-07:00 Eric Xie :

> Github has many project management tools. Any of them would be a better
> choice than JIRA.
>
> Thanks,
> Eric
>
> On 2018/06/09 00:02:55, Timur Shenkao  wrote:
> > Hello guys!
> > Let me add five cents step-by-step. Some kind of "external viewpoint" (I
> > don't work in Amazon or Intel).
> > 1) Thanks a lot of for interesting product / framework like MXNet.
> > Currently, I use pretty standard DL stack: Keras + TF and some other less
> > popular libraries. But I am not completely satisfied so I am looking for
> > something new for my upcoming projects. So, I "investigate" MXNet now.
> >
> > 2)* I completely agree with Naveen*.
> >
> > 3) If you really want to see wide adoption of MXNet (like Spark, Kafka,
> > Flink, etc.), then tasks should be documented in some task tracker.
> > In this case everyone would be able to see the progress of the project,
> > understand which features, bug fixes are included into the next release,
> > intended to appear in near future. Developers, contributors, committers
> can
> > see the progress, the cadence of the project. Links to docs, Github PRs,
> > roadmaps, proposals, build results, other JIRAs are present in a JIRA
> > ticket so it's easy to navigate for everyone.
> > Github is suitable for code review and management. But it can't
> substitute
> > tracker system. Even small team will have hundreds of PRs, issues pretty
> > soon and chaos arises. Besides, there is no much fun (for "external"
> user)
> > in jumping from one Github PR to another trying to understand why & when
> &
> > in which commit something appeared.
> > I understand that MXNet was promoted into Apache incubator to enlarge
> > community, build ecosystem around it and for some other "commercial
> > interests". Without clear vision and understanding of project's status &
> > future, nobody will take MXNet into production or build MXNetT "plugin".
> >
> > 4)  "*There are 900+ issues, that once in a while gets closed without any
> > reason has happened already once*". It also happens with other
> Github-based
> > projects. For example, Presto committers go through list of issues once
> or
> > twice a year. And close vast majority of them with resolutions like
> "Won't
> > fix", "I don't like it", etc.
> >
> > 5) JIRA is very configurable, one does not have to jump from board to
> board
> > to edit / enter ticket info. I have Apache's JIRA account. And I can
> read,
> > add comments & files / patches to tickets, create tickets in some
> projects.
> > But privileges can be configured.
> >
> >
> > Timur
> >
> > On Fri, Jun 8, 2018 at 10:48 PM, Hen  wrote:
> >
> > > Are you saying only committers can make JIRA accounts?
> > >
> > > On Fri, Jun 8, 2018 at 2:41 PM Qing Lan  wrote:
> > >
> > > > + 0
> > > > Can we keep this optional? Not totally abandon or support.
> > > > Pros:
> > > > I used JIRA to track most of my PRs and can place them together at
> one
> > > > place. It also being helpful if we find some issues and group them
> > > together
> > > > as one case.
> > > > Cons:
> > > > Currently JIRA does not allow someone who is not contributor to file
> an
> > > > issue which makes first-time contributor hard to submit a PR.
> > > >
> > > > Thanks,
> > > > Qing
> > > >
> > > >
> > > > On 6/8/18, 2:27 PM, "Hao Jin"  wrote:
> > > >
> > > > +0.5
> > > > I'm an SDE working for MXNet Engine team at AWS and I've been
> using
> > > > JIRA
> > > > for nearly every single PR (28 out of 29 PR at this moment) I
> created
> > > > since
> > > > I joined the team 3 months ago. I wouldn't say it's a really bad
> > > > experience, but definitely not good.
> > > > I do agree that we need something like JIRA and extra effort
> other
> > > than
> > > > writing code to manage the project, but the current user
> experience
> > > > with
> > > > JIRA really has room for improvement.
> > > > The more important question for the community is that, is JIRA
> good
> > > for
> > > > attracting and retaining open source contributors? Such a user
> > > > experience
> > > > of JIRA could drive contributors away if we're really enforcing
> it.
> > > > In conclusion, I think the usage of JIRA should be of one's or
> team's
> > > > own
> > > > choice, if you do have the need to manage some development
> process
> > > > (like
> > > > our team), just continue to use it, but we should no longer
> enforce
> > > or
> > > > persuade anyone to use it.
> > > > I'm also attaching a typical workflow of mine using JIRA with
> sprint
> > > > management and story point tracking for a single task, I think
> > > > eve

Re: PR validation and runtime of CI

[Thomas DELTEIL]

Thanks for bringing the issue of CI stability!

However I disagree with some points in this thread:

- "We are at approximately 3h for a full successful run."
=> Looking at Jenkins I see the last successful runs oscillating between
1h53 and 2h42 with a mean that seems to be at 2h20. Or are you talking
about something different than the jenkins CI run?

- "For this I propose working towards moving tests from CI to nightly,
specially
the ones that take most time or do black box testing with full training of
models. And addressing flaky tests by either fixing them or *disabling
them." *
=> Is there any evidence that some serious effort has been spent trying to
fix the flaky tests? I know Sheng and Marco have worked to consolidating a
list of Flaky tests, but I think simply disabling tests will just make the
platform weaker. Let's organize a flaky test week where we each take on a
couple of these flaky tests and hopefully we should make good progress
towards stabilizing the CI.

-"I'd like to disable flaky tests until they're fixed."
=> Wishful thinking IMO, we know this never happens, if we can't make time
now to fix them, we'll never go back and fix them.

"I would want a turnaround time of less than 30 minutes and 0% failure rate
on master."
 => With current timing, this means barely finishing the build step. Do we
propose dropping some platforms for building?

I agree with some points:

"Won't we end up in the same situation with so many flaky tests?" => pretty
sure it will
"This could be set to 100% for nightly, for example."[for the release] =>
That would be a given to me
"I'm also currently working on a system that tracks all test failures, so
this will also cover nightly tests. This will give us actionable data " =>
Awesome, that would be great to have data on that to help prioritize what
to fix!

I personally think if we disable most tests and move them to nightly tests,
we will decrease the trust and stability of the platform and it leaves the
door open to conflicting changes creating hard to debug failures. I think
the biggest potential win here is reducing test flakiness. That's the one
that is killing the productivity, we can redesign the test pipeline to run
integration and unit test in parallel and that would give us straight away
a 30 minutes reduced time in the CI run. Then we'd be always at <2h for a
build, which seems reasonable if it never fails for no reason.

Thomas

2018-06-07 8:27 GMT-07:00 Marco de Abreu :

> Yeah, I think we are at the point at which we have to disable tests..
>
> If a test fails in nightly, the commit would not be reverted since it's
> hard to pin a failure to a specific PR. We will have reporting for failures
> on nightly (they have proven to be stable, so we can enable it right from
> the beginning). I'm also currently working on a system that tracks all test
> failures, so this will also cover nightly tests. This will give us
> actionable data which allows us to define acceptance criteria for a
> release. E.g. if the test success rate is below X%, a release can not be
> made. This could be set to 100% for nightly, for example.
>
> It would definitely be good if we could determine which tests are required
> to run and which ones are unnecessary. I don't really like the flag in the
> comment (and also it's hard to integrate). A good idea would be some
> analytics on the changed file content. If we have this data, we could
> easily enable and disable different jobs. Since this behaviour is entirely
> defined in GitHub, I'd like to invite everybody to submit a PR.
>
> -Marco
>
>
>
> On Thu, Jun 7, 2018 at 5:20 PM Aaron Markham 
> wrote:
>
> > I'd like to disable flaky tests until they're fixed.
> > What would the process be for fixing a failure if the tests are done
> > nightly? Would the commit be reverted? Won't we end up in the same
> > situation with so many flaky tests?
> >
> > I'd like to see if we can separate the test pipelines based on the
> content
> > of the commit. I think that md, html, and js updates should fly through
> and
> > not have to go through GPU tests.
> >
> > Maybe some special flag added to the comment?
> > Is this possible?
> >
> >
> > On Wed, Jun 6, 2018 at 10:37 PM, Pedro Larroy <
> > pedro.larroy.li...@gmail.com>
> > wrote:
> >
> > > Hi Team
> > >
> > > The time to validate a PR is growing, due to our number of supported
> > > platforms and increased time spent in testing and running models.  We
> are
> > > at approximately 3h for a full successful run.
> > >
> > > This is compounded with the failure rate of builds due to flaky tests
> of
> > > more than 50% which is a big drag in developer productivity if you can
> > only
> > > get one or two CI runs to a change per day.
> > >
> > > I would want a turnaround time of less than 30 minutes and 0% failure
> > rate
> > > on master.
> > >
> > > For this I propose working towards moving tests from CI to nightly,
> > > specially the ones that take most time or do black box testing with
> full
> 

Re: home page highlights section updates

[Thomas DELTEIL]

 Nice work, I like wireframe 2 the best, it gives structure to an otherwise
loosely structured list.

2018-06-01 14:57 GMT-07:00 Aaron Markham :

> Hi everyone, happy Friday,
>
> I've mocked up a couple of modifications for the highlights on the home
> page.
> The goal is to expose more news, more tutorials, and other interesting
> things people have been working on. If you've contributed something
> interesting and want to highlight it, you should be able to suggest an
> update to the page.
> It would be great to have variety and fresh content on the site.
>
> Here are a couple of ideas with screenshots that shouldn't be to hard to
> implement:
> https://cwiki.apache.org/confluence/display/MXNET/
> Website+Interface+Updates
>
> You can reply with feedback here or make comments on the wiki! What are
> your thoughts on:
> 1) getting fresh content on the site
> 2) highlighting contributors and their contributions
> 3) flow on changes and what kind of approvals are expected for swapping out
> highlights
> 4) UX ideas that are not too challenging to implement or maintain and work
> well with mobile
>
> Cheers,
> Aaron
>

Re: backward compatibility of models saved with 1.2.0

[Thomas DELTEIL]

Created a github issue detailing the problem with a reproducible example:
https://github.com/apache/incubator-mxnet/issues/11091

Thanks,

Thomas

2018-05-29 11:50 GMT-07:00 Sergio Fernández :

> Hi Mario,
>
> I can't give you much details. But it looks there is a bug exporting the
> parameters' names to a JSON models.
>
> I wonder if anybody else in the community has faced this bug.
>
> Cheers,
>
>
> On Mon, May 28, 2018 at 5:42 PM, Marco de Abreu <
> marco.g.ab...@googlemail.com> wrote:
>
> > Hello Sergio,
> >
> > you are right. We are following semantic versioning and thus, every model
> > produced within the same major version (e.g.1.x) has to be backwards
> > compatible.
> >
> > Could you please provide a small example so we can reproduce this? We
> > definitely do not want our users to retrain their model if they update
> > MXNet. That's a serious issue and we'd love to follow up.
> >
> > Best regards,
> > Marco
> >
> > Sergio Fernández  schrieb am Di., 29. Mai 2018,
> 02:35:
> >
> > > Hi,
> > >
> > > I can't find anything related on that in the 1.2.-0-incubating
> changelog,
> > > so I assume models produced by the latest version would be backward
> > > compatible with old versions, such as 1.1.0. But we've found that the
> > > parameter model produced is very different and doesn't load.
> > >
> > > Can you point me to any documentation that could help us to load the
> > model
> > > in 1.1.0 without re-training?
> > >
> > > Thanks.
> > >
> >
>

Re: Auto scaling for MXNet CI

[Thomas DELTEIL]

Great news :) thanks Marco!

On Tue, May 15, 2018, 18:29 Steffen Rochel  wrote:

> Thanks Marco, good step forward.
> What is the expected, typical and worst case TAT time for PR checks?
>
> Steffen
>
> On Tue, May 15, 2018 at 10:49 AM Marco de Abreu <
> marco.g.ab...@googlemail.com> wrote:
>
> > Hello,
> >
> > I'd like to announce the deployment of auto scaling for our CI system
> (see
> > [1] for reference, setup documentation at [2]) for today at 11:00PM PST
> > 05/15/18. I expect no downtime since these changes are happening outside
> of
> > Jenkins.
> >
> > This system will increase the flexibility of our system to be able to
> > handle the increasing load, being a result of the growing number of great
> > contributions! In future, our CI will automatically adapt to the current
> > load and will thus support big tasks like the to-be-migrated nightly
> tests
> > or increased load before a release. Additionally, we're now able to
> provide
> > scalable p3.2xlarge instances and have the possibility to add new
> instance
> > types without much effort.
> >
> > In future, you will see that new slaves are being started up as the queue
> > grows and stopped if they go into idle. Your tasks will automatically be
> > picked up and our system makes sure every PR gets processes as fast as
> > possible.
> >
> > If you encounter any issues in the next week, please don't hesitate to
> > reach out to me. I'm looking forward to everyones feedback!
> >
> > Best regards,
> > Marco
> >
> >
> > [1]:
> >
> https://cwiki.apache.org/confluence/display/MXNET/Proposal%3A+Auto+Scaling
> > [2]: https://cwiki.apache.org/confluence/display/MXNET/Setup
> >
>

Tutorial tests in the CI

[Thomas DELTEIL]

Hi all,

I have updated the CI to make sure all tutorials run in python2 and python3
on GPU, thanks Marco for the help.

I have updated the instructions for writing tutorials here:
https://github.com/apache/incubator-mxnet/pull/10658

But just a few reminders for people contributing tutorials:
- Thank you! Your contributions are very appreciated
- The full instructions for contributing tutorials are here:
https://github.com/apache/incubator-mxnet/blob/master/example/README.md

Main points that are usually forgotten:
- Add  at the end of your python
tutorials to enable the download as a jupyter notebook (this now tested as
part of a sanity check on your PR)
- Add your tutorials to docs/tutorials/index.md for the tutorial to show up
on the website
- You can use  to prevent a line from being
generated in the notebook (useful for images ![](img_url) generated with
plt.imshow() for example)

New steps (Don't worry, the sanity check will catch it if you forget):
- (NEW) Add your tutorial test here: tests/tutorials/test_tutorials.py
- (NEW) Add your tutorial package dependencies here:
ci/docker/install/ubuntu_tutorials.sh

If you witness some flaky tutorials in the CI in the coming days, please
create an issue and tag me in it, I'll try to fix asap.

All the best,

Thomas

Re: [VOTE] Release Apache MXNet (incubating) version 1.2.0.RC0

[Thomas DELTEIL]

@Anirudh, thanks for looking into it! However I do not understand what you
mean by 'set as CPU and not GPU'? MXNet compiled with mkldnn and cuda is
supposed to be able to work with both context no? There are other tutorials
that are running successfully on both CPU and GPU context.

@Da to reproduce:

Download the source of 1.2.0.rc0 and extract it, cd into it.

make docs
make clean
make -j $(nproc) USE_OPENCV=1 USE_BLAS=openblas USE_CUDA=1
USE_CUDA_PATH=/usr/local/cuda USE_CUDNN=1 USE_MKLDNN=1
export PYTHONPATH=$(pwd)/python
cd tests/nightly
python test_tutorial.py --tutorial onnx/super_resolution

you can also start a jupyter notebook server and try to run
docs/_build/html/tutorials/onnx/super_resolution.ipynb



2018-04-21 15:08 GMT-07:00 Zheng, Da :

> @ThomasDelteil could you show me how to reproduce the problem? I'll take
> it a look as well.
>
> Best,
> Da
>
> Sent from my iPhone
>
> On Apr 21, 2018, at 1:12 PM, Anirudh Acharya  <mailto:anirudhk...@gmail.com>> wrote:
>
> @ThomasDelteil that might be due to the fact that in the example, the
> context is being set as CPU and not GPU.
> But I will still take a look as soon as possible.
>
>
> Regards
> Anirudh
>
> On Sat, Apr 21, 2018 at 11:10 AM, Thomas DELTEIL <
> thomas.delte...@gmail.com<mailto:thomas.delte...@gmail.com>>
> wrote:
>
> *-0*
>
> compiled from source on GPU CUDA/CUDNN, tutorials run fine.
>
> However:
> Compiled from source and adding USE_MKLDNN=1, the onnx/super_resolution
> tutorial is crashing on this line:
>
> ```
> from collections import namedtuple
> Batch = namedtuple('Batch', ['data'])
>
> # forward on the provided data batch
> mod.forward(Batch([mx.nd.array(test_image)]))
> ```
>
> Stack trace returned 8 entries:
> [bt] (0)
> /home/ubuntu/apache-mxnet-src-1.2.0.rc0-incubating/python/
> mxnet/../../lib/libmxnet.so(dmlc::StackTrace[abi:cxx11]()+0x5b)
> [0x7feef615721b]
> [bt] (1)
> /home/ubuntu/apache-mxnet-src-1.2.0.rc0-incubating/python/
> mxnet/../../lib/libmxnet.so(dmlc::LogMessageFatal::~
> LogMessageFatal()+0x28)
> [0x7feef6158258]
> [bt] (2)
> /home/ubuntu/apache-mxnet-src-1.2.0.rc0-incubating/python/
> mxnet/../../lib/libmxnet.so(mxnet::engine::ThreadedEngine:
> :ExecuteOprBlock(mxnet::RunContext,
> mxnet::engine::OprBlock*)+0xfa9) [0x7feef8b1ad49]
> [bt] (3)
> /home/ubuntu/apache-mxnet-src-1.2.0.rc0-incubating/python/
> mxnet/../../lib/libmxnet.so(std::_Function_handler (std::shared_ptr),
> mxnet::engine::ThreadedEnginePerDevice::PushToExecute(mxnet::engine::
> OprBlock*,
> bool)::{lambda()#1}::operator()()
> const::{lambda(std::shared_ptr)#1}>::_
> M_invoke(std::_Any_data
> const&, std::shared_ptr&&)+0xe2) [0x7feef8b30d82]
> [bt] (4)
> /home/ubuntu/apache-mxnet-src-1.2.0.rc0-incubating/python/
> mxnet/../../lib/libmxnet.so(std::thread::_Impl simple (std::shared_ptr)> (std::shared_ptr ManualEvent>)>
> ::_M_run()+0x4a) [0x7feef8b2af1a]
> [bt] (5) /home/ubuntu/anaconda3/bin/../lib/libstdc++.so.6(+0xafc5c)
> [0x7fef7cc79c5c]
> [bt] (6) /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba) [0x7fef7dec36ba]
> [bt] (7) /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fef7dbf941d]
>
> Depending on how experimental we consider MKLDNN, that could be a *-1 *for
> me.
>
> 2018-04-21 9:01 GMT-07:00 Jun Wu mailto:wu
> jun@gmail.com>>:
>
> +1
>
> Compiled from source. Ran the model quantization example. Both quantized
> model generation and inference can run successfully.
>
> On Fri, Apr 20, 2018 at 5:14 PM, Indhu  indhubhara...@gmail.com>> wrote:
>
> +1
>
> Compiled from source on P3 instance. Tested the SSD example and some
> Gluon
> examples.
>
> On Wed, Apr 18, 2018, 7:40 PM Anirudh  anirudh2...@gmail.com>> wrote:
>
> Hi everyone,
>
> This is a vote to release Apache MXNet (incubating) version 1.2.0.
> Voting
> will start now (Wednesday, April 18th) and end at 7:40 PM PDT,
> Saturday,
> April 21st.
>
> Link to the release notes:
>
>
> https://cwiki.apache.org/confluence/display/MXNET/
> Apache+MXNet+%28incubating%29+1.2.0+Release+Notes
>
> Link to the release candidate 1.2.0.rc0:
> https://github.com/apache/incubator-mxnet/releases/tag/1.2.0.rc0
>
> View this page, click on "Build from Source", and use the source code
> obtained from the 1.2.0.rc0 tag:
> https://mxnet.incubator.apache.org/install/index.html
>
> (Note: The README.md points to the 1.2.0 tag and does not work at the
> moment.)
>
> Please remember to TEST first before voting accordingly:
> +1 = approve
> +0 = no opinion
> -1 = disapprove (provide reason)
>
> Thanks,
>
> Anirudh
>
>
>
>
>

Re: [VOTE] Release Apache MXNet (incubating) version 1.2.0.RC0

[Thomas DELTEIL]

*-0*

compiled from source on GPU CUDA/CUDNN, tutorials run fine.

However:
Compiled from source and adding USE_MKLDNN=1, the onnx/super_resolution
tutorial is crashing on this line:

```
from collections import namedtuple
Batch = namedtuple('Batch', ['data'])

# forward on the provided data batch
mod.forward(Batch([mx.nd.array(test_image)]))
```

Stack trace returned 8 entries:
[bt] (0)
/home/ubuntu/apache-mxnet-src-1.2.0.rc0-incubating/python/mxnet/../../lib/libmxnet.so(dmlc::StackTrace[abi:cxx11]()+0x5b)
[0x7feef615721b]
[bt] (1)
/home/ubuntu/apache-mxnet-src-1.2.0.rc0-incubating/python/mxnet/../../lib/libmxnet.so(dmlc::LogMessageFatal::~LogMessageFatal()+0x28)
[0x7feef6158258]
[bt] (2)
/home/ubuntu/apache-mxnet-src-1.2.0.rc0-incubating/python/mxnet/../../lib/libmxnet.so(mxnet::engine::ThreadedEngine::ExecuteOprBlock(mxnet::RunContext,
mxnet::engine::OprBlock*)+0xfa9) [0x7feef8b1ad49]
[bt] (3)
/home/ubuntu/apache-mxnet-src-1.2.0.rc0-incubating/python/mxnet/../../lib/libmxnet.so(std::_Function_handler),
mxnet::engine::ThreadedEnginePerDevice::PushToExecute(mxnet::engine::OprBlock*,
bool)::{lambda()#1}::operator()()
const::{lambda(std::shared_ptr)#1}>::_M_invoke(std::_Any_data
const&, std::shared_ptr&&)+0xe2) [0x7feef8b30d82]
[bt] (4)
/home/ubuntu/apache-mxnet-src-1.2.0.rc0-incubating/python/mxnet/../../lib/libmxnet.so(std::thread::_Impl)> (std::shared_ptr)>
>::_M_run()+0x4a) [0x7feef8b2af1a]
[bt] (5) /home/ubuntu/anaconda3/bin/../lib/libstdc++.so.6(+0xafc5c)
[0x7fef7cc79c5c]
[bt] (6) /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba) [0x7fef7dec36ba]
[bt] (7) /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fef7dbf941d]

Depending on how experimental we consider MKLDNN, that could be a *-1 *for
me.

2018-04-21 9:01 GMT-07:00 Jun Wu :

> +1
>
> Compiled from source. Ran the model quantization example. Both quantized
> model generation and inference can run successfully.
>
> On Fri, Apr 20, 2018 at 5:14 PM, Indhu  wrote:
>
> > +1
> >
> > Compiled from source on P3 instance. Tested the SSD example and some
> Gluon
> > examples.
> >
> > On Wed, Apr 18, 2018, 7:40 PM Anirudh  wrote:
> >
> > > Hi everyone,
> > >
> > > This is a vote to release Apache MXNet (incubating) version 1.2.0.
> Voting
> > > will start now (Wednesday, April 18th) and end at 7:40 PM PDT,
> Saturday,
> > > April 21st.
> > >
> > > Link to the release notes:
> > >
> > >
> > > https://cwiki.apache.org/confluence/display/MXNET/
> > Apache+MXNet+%28incubating%29+1.2.0+Release+Notes
> > >
> > > Link to the release candidate 1.2.0.rc0:
> > > https://github.com/apache/incubator-mxnet/releases/tag/1.2.0.rc0
> > >
> > > View this page, click on “Build from Source”, and use the source code
> > > obtained from the 1.2.0.rc0 tag:
> > > https://mxnet.incubator.apache.org/install/index.html
> > >
> > > (Note: The README.md points to the 1.2.0 tag and does not work at the
> > > moment.)
> > >
> > > Please remember to TEST first before voting accordingly:
> > > +1 = approve
> > > +0 = no opinion
> > > -1 = disapprove (provide reason)
> > >
> > > Thanks,
> > >
> > > Anirudh
> > >
> >
>

Re: blog for MXNet

[Thomas DELTEIL]

Thanks Aaron, I like medium, a lot of projects seems to be posting their
articles there, as you mentioned.

Note that there is a newly created Chinese MXNet blog here:
https://zh.mxnet.io/blog/

I would be happy to contribute to the blogs, if you want to add me to the
writer/editor list.



Also, there is a mxnet subreddit r/mxnet which was created by Sebastian
(thanks!) and I am now a moderator as well. Feel free to cross-post any
interesting content there! https://www.reddit.com/r/mxnet/  Please
subscribe!


I will try to post one link a day at least, until I run out of links ☺ We
will also improve the look of it this week and add links to relevant
resources on the side bar, etc.



All the best,


Thomas


2018-04-11 14:45 GMT-07:00 Aaron Markham :

> Having a blog for MXNet would be very useful for conveying news,
> talking about features, demoing applications, and building awareness.
>
> Does anyone have particular preferences or recommendations on blog
> hosting or platform?
>
> I currently have editor access for an MXNet branded account on Medium.
> https://medium.com/mxnet
>
> There's nothing there at the moment, but at least with Medium we all
> could get started right away, and have a built-in syndication
> platform. Also, note that this is where the TensorFlow blog resides:
> https://medium.com/tensorflow
>
> Please make it known if you'd like to contribute, so you can get
> writer/editor access (to whichever platform we settle on.)
>
> Cheers,
> Aaron
>

Re: MXNet Name Change?

[Thomas DELTEIL]

FWIW Brainscript is actually a network definition language:
https://docs.microsoft.com/en-us/cognitive-toolkit/BrainScript-Network-Builder



Thomas


2018-04-11 12:13 GMT-07:00 Chiyuan Zhang :

> IIRC CNTK renamed to something like brainscript which does not seem to be
> very successful publicity campaign?
>
> Chiyuan
>
> On Wed, Apr 11, 2018 at 10:18 AM Chris Olivier 
> wrote:
>
> > Should we consider renaming MXNet to something more "friendly"?
> >
> > IMHO, I think this may be related to adoption problems.
> >
> > MXNet, CMTK -- both seem sort of sterile and hard to use, don't they?
> >
> > Tensorflow, PyTorch, Caffe -- sound cool.
> >
> --
> Semt ftom m ipohne
>

Re: Apache MXNet youtube channel

[Thomas DELTEIL]

Hi Marco,

Thanks! We have a couple of videos lined up already, but I like the concept
of having a schedule and releasing videos on a regular basis, even if it is
just a notebook run-through.

All the best,

Thomas

2018-04-07 6:56 GMT-07:00 Marco de Abreu :

> Hi Thomas,
>
> this is a GREAT idea! Thanks a lot for increasing our social media
> coverage.
>
> Do you know if there are any plans to create videos on a regular basis -
> e.g. once per week? I think a constant stream of new videos could increase
> our visibility over time and add a lot of value, especially for new users.
>
> Best regards,
> Marco
>
> On Sat, Apr 7, 2018 at 12:14 AM, Thomas DELTEIL  >
> wrote:
>
> > Hi all,
> >
> > We created an Apache MXNet youtube channel, to list and group youtube
> > content pertaining to MXNet.
> >
> > - If you know of content that should be listed there, please send me a
> list
> > of links and into which playlist they should be added (and new playlist
> > ideas).
> > - If you know of channels, or have a channel that promotes MXNet content,
> > please send me a list of them so I can subscribe the MXNet channel to
> them,
> > they will be featured on the landing page.
> > - If you produce new videos that you would like hosted directly on the
> > Apache MXNet page, we should be able to host them directly there too.
> > (still figuring out that part).
> >
> > https://www.youtube.com/channel/UCQua2ZAkbr_Shsgfk1LCy6A
> >
> > Please subscribe and share ( When we reach 100 subscribers and 30 month
> of
> > existence, we will be able to apply for a personalized youtube URL. e.g
> > https://youtube.com/channel/mxnet
> >
> > Thanks!
> >
> > Thomas Delteil
> >
>

Apache MXNet youtube channel

[Thomas DELTEIL]

Hi all,

We created an Apache MXNet youtube channel, to list and group youtube
content pertaining to MXNet.

- If you know of content that should be listed there, please send me a list
of links and into which playlist they should be added (and new playlist
ideas).
- If you know of channels, or have a channel that promotes MXNet content,
please send me a list of them so I can subscribe the MXNet channel to them,
they will be featured on the landing page.
- If you produce new videos that you would like hosted directly on the
Apache MXNet page, we should be able to host them directly there too.
(still figuring out that part).

https://www.youtube.com/channel/UCQua2ZAkbr_Shsgfk1LCy6A

Please subscribe and share ( When we reach 100 subscribers and 30 month of
existence, we will be able to apply for a personalized youtube URL. e.g
https://youtube.com/channel/mxnet

Thanks!

Thomas Delteil