Re: [apache/incubator-mxnet] [RFC] v1.8.0 release (#18800)
I would like to include the [BatchNorm performance improvement PR](https://github.com/apache/incubator-mxnet/pull/18676) for axis != 1 to 1.8 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/apache/incubator-mxnet/issues/18800#issuecomment-664642323
Re: [DISCUSS] Assigning Issues
+1 On Thu, Sep 12, 2019 at 1:37 PM Anirudh Subramanian wrote: > +1 > > On Thu, Sep 12, 2019 at 1:15 PM Zach Kimberg > wrote: > > > We had a discussion a while back about trying to improve the way we > handle > > issues by assigning them to users who are working on them. However, the > > discussion ended because issues could only be assigned to those with > write > > access (committers). > > > > I just came across a new Github feature where issues can also be assigned > > to any user who comments on an issue [ > > https://github.blog/2019-06-25-assign-issues-to-issue-commenters/]. > > Committers can then assign anyone from the community who wants to work on > > the issue so we can track which issues are assigned and which ones are > not. > > Assigned community members still have an "Unassign me" button if they no > > longer wish to work on an issue. It is also possible to assign up to 10 > > people to an issue (or PR). > > > > Given this, I think we should try to assign issues when possible to those > > working on them. What does everyone think? > > > > Zach > > >
Re: [Announcement] New Committer - Lai Wei
Congratulations! Lai On Sat, Aug 3, 2019 at 6:13 PM Sheng Zha wrote: > Hi all, > > Please join me in welcoming Lai (Roy) Wei as a new committer of Apache > MXNet (incubating)! > > Lai was one of the main contributor to the MXNet Keras frontend. More > recently, he contributed the > Gluon estimator API, which enables easy usage and better modularization > for the training scripts > of Gluon. He also persisted and helped driving the long-running 1.5.0 > release process. > > Welcome, Lai! > > -sz > >
Re: [Discuss] MXNet Python 2 Support Deprecation
+1 On Thu, Jul 18, 2019 at 11:27 AM Junru Shao wrote: > +1 > > On Thu, Jul 18, 2019 at 11:12 AM Anirudh Acharya > wrote: > > > +1 > > > > On Thu, Jul 18, 2019 at 11:03 AM Marco de Abreu > > > wrote: > > > > > +1 > > > > > > -Marco > > > > > > Sheng Zha schrieb am Do., 18. Juli 2019, 19:59: > > > > > > > Dear MXNet community, > > > > > > > > I'd like to reopen the discussion on deprecating python2 support. > This > > > > would help modernize the design and engineering practice in MXNet to > > help > > > > improve speed and quality. > > > > > > > > For this purpose, I reopened the issue on this here: > > > > https://github.com/apache/incubator-mxnet/issues/8703 > > > > > > > > If the consensus is towards the direction of dropping python2 > support, > > I > > > > suggest we announce our plan to drop python2 support in the next > > release, > > > > and actually drop the support in the next major version. Thanks. > > > > > > > > -sz > > > > > > > > > >
Re: CUDA recommendation
Hi Aaron, The performance regression that Sam mentioned above is this one [1]. Since CUDA 10.1 doesn’t have the regression, Nvidia suggested us to move to CUDA 10.1. Regarding documentation. I will raised a PR to update it. Thanks. Jake [1] https://github.com/apache/incubator-mxnet/issues/14725 Aaron Markham 於 2019年5月24日 週五，17:52寫道： > Sounds like we need an * near the CUDA 10.1 recommendation if there > are known performance issues. Is there a particular issue # tracking > the performance issues? > I'm seeing a CUDA 10 Windows issue here that seems unresolved: > https://github.com/apache/incubator-mxnet/issues/14479 > > On Fri, May 24, 2019 at 5:00 PM Sheng Zha wrote: > > > > 10.1 is recommended. The oldest CUDA version that we release is 8.0. > > > > -sz > > > > On 2019/05/24 23:29:38, Marco de Abreu wrote: > > > While we are at the topic, did we actually agree on dropping support > for > > > some versions? So far we are releasing all the way been to cuda 7.5 I > think > > > > > > -Marco > > > > > > Skalicky, Sam schrieb am Fr., 24. Mai > 2019, > > > 23:43: > > > > > > > Hi Aaron > > > > > > > > Right now, the most stable version is CUDA 9.2. CUDA 10 is supported > and > > > > some pip wheels are available, but there are known performance > issues. And > > > > we are quickly moving to CUDA 10.1. So things are still in flux now. > I > > > > think the best approach would be to wait a couple more weeks before > > > > updating this part of the docs. > > > > > > > > Sam > > > > > > > > > > > > > On May 24, 2019, at 2:09 PM, Aaron Markham < > aaron.s.mark...@gmail.com> > > > > wrote: > > > > > > > > > > What version of CUDA is currently recommended? > > > > > Now that there are packages for CUDA 10, shouldn't the build from > > > > > source (and other) documentation reflect the latest greatest > > > > > combinations? > > > > > > > > > > Specifically the Ubuntu guide [1] states: "CUDA 9.2 is > recommended." > > > > > > > > > > [1] > > > > > https://mxnet.incubator.apache.org/versions/master/install/ubuntu_setup.html#cuda-dependencies > > > > > > > > > > Cheers, > > > > > Aaron > > > > > > > > > > > >
Re: Dependency Update
Thanks Aaron that's a great suggestion. The reason why I put it under tools/dependencies is that the doc is intended for developers who want to contribe to update the dependencies of our PyPI package. Regarding the CI, I'm also working on upgrading CUDA/cuDNN version that CI use -PR[1][2]. Thanks, Jake [1] https://github.com/apache/incubator-mxnet/pull/14986 [2] https://github.com/apache/incubator-mxnet/pull/14950 On Wed, May 22, 2019 at 3:15 PM Aaron Markham wrote: > Thanks for doing a thorough look at the version ranges. I have this PR > [1] waiting for review that tries to pin graphviz and opencv, and it > updates CI as well as the docs that go on the website. > I think your updates would be beneficial in the docs that go on the > website and should also update CI. Is there a benefit to having them > as a readme in /tools? Doesn't this create extra maintenance with > these version numbers being in three places (website install > instructions, /tools folder, /ci folder)? > > [1] https://github.com/apache/incubator-mxnet/pull/14987 > > On Wed, May 22, 2019 at 2:31 PM Qing Lan wrote: > > > > > > Great work Jake! The content on CPU/GPU build instruction is really > helpful. > > > > Thanks, > > Qing > > > > > > From: Jake Lee > > Sent: Wednesday, May 22, 2019 17:26 > > To: dev@mxnet.incubator.apache.org > > Subject: Dependency Update > > > > Dear Community, > > > > I have been working on dependency udpate of MXNet. The goal is to upgrade > > the dependencies that have security vulnerability issues and make MXNet > > great again by being benefit from latest CUDA, cuDNN and NCCL software. I > > documented the process on PR< > > https://github.com/apache/incubator-mxnet/pull/15045>. Big thanks to > Sheng > > Zha(szha), Dick Carter(DickJC123), Anirudh Subramanian(anirudh2290), Qing > > Lan(lanking520), Per Goncalves da Silva(perdasilva) for supporting me! > > > > Thanks, > > Jake >
Dependency Update
Dear Community, I have been working on dependency udpate of MXNet. The goal is to upgrade the dependencies that have security vulnerability issues and make MXNet great again by being benefit from latest CUDA, cuDNN and NCCL software. I documented the process on PR< https://github.com/apache/incubator-mxnet/pull/15045>. Big thanks to Sheng Zha(szha), Dick Carter(DickJC123), Anirudh Subramanian(anirudh2290), Qing Lan(lanking520), Per Goncalves da Silva(perdasilva) for supporting me! Thanks, Jake
Re: Python2 End of Life
+1 Recently I upgraded the Numpy version and found out that Pylint had false alarm on it. The Pylint fix is only available on Python3. So I changed the default python version of 'make pylint' command to python3 (PR haven't been merged). It's time to drop support for Python2. On Mon, May 13, 2019 at 1:37 PM Junru Shao wrote: > +1 > > On Mon, May 13, 2019 at 1:34 PM Aaron Markham > wrote: > > > +1 for the pledge and to start moving things to Python 3. > > I think our installation instructions and tutorials can be updated to > > default to Python3 and we should update Python2-only tutorials. I know > > we have a handful of those, and when I spot them, I'll create an > > issue. > > I can also look at migrating the docs build to Python 3. > > Should we add a new label for issues relating to migrating to Python3? > > Cheers, > > Aaron > > > > On Mon, May 13, 2019 at 12:04 PM Zach Kimberg > > > wrote: > > > > > > Right now, the official date for ending support for Python 2.7 (and all > > of > > > python2) is set to January 1 [1]. As part of it, a number of projects > > have > > > pledged to drop support for Python2 in or before 2020 including > > Tensorflow, > > > requests, pandas, ipython, numpy, pillow, and Cython [2]. I believe we > > > should also join in this pledge on python3statement.org [2] because it > > > would help clean up our project and it would be difficult to continue > > > supporting Python2 anyway when some of our dependencies are dropping > > > support. > > > > > > As a concrete step, we should decide on a date to remove all usages of > > > Python2 from our CI and consider that officially dropping support. > > > Following that, we can expect PRs will end up breaking support for > > Python2. > > > I suggest just using the same date that Python is dropping support of > > > January 1. We may also need to update some examples or scripts that > were > > > written only for python2 that are around the project. Any thoughts? > > > > > > Zach > > > > > > > > > [1] - https://www.python.org/dev/peps/pep-0373/ > > > [2] - https://python3statement.org/ > > >
Re: [Announcement] New Committer - Zach Kimberg
Congrat! On Thu, May 9, 2019 at 10:37 AM Yuan Tang wrote: > Welcome! > > On Thu, May 9, 2019 at 1:36 PM Marco de Abreu > wrote: > > > Welcome! > > > > Hagay Lupesko schrieb am Do., 9. Mai 2019, 19:33: > > > > > Congratulations Zach - well deserved! > > > > > > On Thu, May 9, 2019, 13:26 Qing Lan wrote: > > > > > > > Hi All, > > > > > > > > Please join me in welcoming Zach Kimberg (https://github.com/zachgk) > > as > > > a > > > > new committer. > > > > > > > > He has been solving some important bugs in MXNet JVM with respect to > > > usage > > > > improvement, build issues and a lot more. He also created the Jenkins > > > based > > > > publish pipeline for us to have standard way to build and test > > > > static-linked package conveniently for everyone in the community. > > > Moreover, > > > > he solved a bunch of License problems we have in MXNet and brought > > > several > > > > fixes to let us get 1.4.0 release on time. > > > > > > > > Thanks, > > > > Qing > > > > > > > > > >
Re: [Announcement] New Committer - Patric Zhao
Congrats, Patric! Jake On Thu, Mar 21, 2019 at 12:03 PM Lin Yuan wrote: > Congrats, Patric! > > On Thu, Mar 21, 2019 at 10:32 AM Yuxi Hu wrote: > > > Congrats, Patric! Well deserved! > > > > On Wed, Mar 20, 2019 at 1:08 PM kellen sunderland < > > kellen.sunderl...@gmail.com> wrote: > > > > > Congrats Patric! > > > > > > On Sun, Mar 17, 2019 at 10:34 PM Hagay Lupesko > > wrote: > > > > > > > Congrats Patric! > > > > > > > > On Fri, Mar 15, 2019 at 7:49 AM Joshua Z. Zhang < > cheungc...@gmail.com> > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > Congrats Patrick! > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Zhi > > > > > > > > > > > > > > > > > On Mar 15, 2019 at 10:46 PM, > > > > marco.g.ab...@gmail.com)> wrote: > > > > > > > > > > > > > > > > > > > > > > > > Congratulations, great to have you on board! > > > > > > > > > > > > -Marco > > > > > > > > > > > > Lv, Tao Aschrieb am Fr., 15. März 2019, > > > 15:38: > > > > > > > > > > > > > Wow, congratulation Patric! > > > > > > > > > > > > > > -Original Message- > > > > > > > From: Steffen Rochel [mailto:steffenroc...@gmail.com] > > > > > > > Sent: Friday, March 15, 2019 10:25 PM > > > > > > > To: dev@mxnet.incubator.apache.org > > > > > > > Cc: patric zhao > > > > > > > Subject: Re: [Announcement] New Committer - Patric Zhao > > > > > > > > > > > > > > Congratulation Patrick! > > > > > > > Steffen > > > > > > > > > > > > > > On Fri, Mar 15, 2019 at 5:38 AM Zhao, Patric < > > > > patric.z...@intel.com> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > I am very glad to have this opportunity to contribute to > the > > > > > > > > Apache/MXNet community :) > > > > > > > > > > > > > > > > Thanks all of the supports from the community and Intel. > > > > > > > > > > > > > > > > BR, > > > > > > > > > > > > > > > > --Patric > > > > > > > > > > > > > > > > > > > > > > > > > -Original Message- > > > > > > > > > From: MiraiWK WKCN [mailto:w...@live.cn] > > > > > > > > > Sent: Friday, March 15, 2019 12:52 AM > > > > > > > > > To: dev@mxnet.incubator.apache.org; patric zhao > > > > > > > > > > > > > > > > > > Subject: Re: [Announcement] New Committer - Patric > Zhao > > > > > > > > > > > > > > > > > > Welcome Peng Zhao! > > > > > > > > > Peng is the AI Tech Leader in Intel Corporation. We > have > > > > > good > > > > > > > > > cooperation before. He is very professional and > > > contribute a > > > > > lot to > > > > > > > > > MXNet, > > > > > > > > especially deep > > > > > > > > > learning boost on CPU. > > > > > > > > > > > > > > > > > > > > > > > > > > > From: Anirudh Subramanian > > > > > > > > > Sent: Thursday, March 14, 2019 3:54:50 PM > > > > > > > > > To: dev@mxnet.incubator.apache.org; patric zhao > > > > > > > > > Subject: [Announcement] New Committer - Patric Zhao > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > Please join me to welcome Patric Zhao as a new > committer > > > of > > > > > Apache > > > > > > > > > (incubating) MXNet! > > > > > > > > > > > > > > > > > > Patric has put in great effort around MKLDNN > integration > > > > into > > > > > MXNet > > > > > > > > > and > > > > > > > > has > > > > > > > > > been involved in features like quantization, graph > > fusion > > > > and > > > > > fused > > > > > > > > > RNN operators for CPU. > > > > > > > > > > > > > > > > > > Dev List activity: > > > > > > > > > > > > > > > > > > > > > > > > https://lists.apache.org/list.html?d...@mxnet.apache.org:lte=3y:patric. > > > > > > > > zhao > > > > > > > > > > > > > > > > > > Issues: > > > > > > > > > https://github.com/apache/incubator- > > > > > > > > > > > > > > mxnet/issues?utf8=%E2%9C%93&q=is%3Aissue+involves%3Apengzhao-intel+ > > > > > > > > > > > > > > > > > > PR Reviews: > > > > > > > > > https://github.com/apache/incubator- > > > > > > > > > > > > > > mxnet/pulls?utf8=%E2%9C%93&q=is%3Apr+reviewed-by%3Apengzhao-intel > > > > > > > > > > > > > > > > > > Proposals involved in: > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/MXNET/MXNet+Graph+Optimi > > > > > > > > > z > > > > > > > > > ation+and+Quantization+based+on+subgraph+and+MKL-DNN > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/MXNET/Fused+RNN+Operator > > > > > > > > > s > > > > > > > > > +for+CPU > > > > > > > > > < > > > > > > https://cwiki.apache.org/confluence/display/MXNET/MXNet+Graph+Optim > > > > > > > > > i > > > > > > > > > zation+and+Quantization+based+on+subgraph+and+MKL-DNN> > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Anirudh > > > > > > > > > > > > > > > > > > > > >
Re: [Announcement] New Committer -- Lin Yuan
Congrats Lin! Tianqi Chen 於 2019年2月2日 週六 下午3:27寫道： > Dear Community: > > Please join me to welcome Lin Yuan(@apeforest) as a new committer of > Apache(incubating) MXNet! > > He has contributed to various improvements, including better compatibility > of larger arrays across the codebase. > > Commits: > https://github.com/apache/incubator-mxnet/commits?author=apeforest > > https://github.com/apache/incubator-mxnet/pulls?utf8=%E2%9C%93&q=is%3Apr+author%3Aapeforest > > > Reviews: > https://github.com/apache/incubator-mxnet/pulls?utf8=% > E2%9C%93&q=reviewed-by%3Aapeforest > > dev@ activitivity > https://lists.apache.org/list.html?*@mxnet.apache.org:lte=6M:Lin%20Yuan > > Tianqi >
Re: [Announce] Upcoming Apache MXNet (incubating) 1.4.0 release
Hi Steffen, It would be nice to add this PR to 1.4 release. https://github.com/apache/incubator-mxnet/pull/13550 It fixes the imagedetiter issue for mxnet 1.3 https://github.com/apache/incubator-mxnet/issues/13037. Thanks, Jake Lee Lin Yuan 於 2018年11月29日 週四 下午7:27寫道： > Hi Steffen, > > Can we add the following PR to 1.4.0 release: > > https://github.com/apache/incubator-mxnet/pull/13452 > > It's just a Python API returning header path so it should not cause any > regression issues. But it is required for Horovod to integrate MXNet. It's > better to have this in a minor release than patch release. > > Thanks, > > Lin > > On Thu, Nov 29, 2018 at 6:46 PM Steffen Rochel > wrote: > > > Hi Zhi - thanks for the improvement, which we should consider for 1.4.0. > > However, I don't see any tests with the PR and think it is too risky to > add > > changes without tests. I will add your PR to the tracking list, but would > > like to ask you to add functional tests before completing the PR to > master > > and v1.4.x branch. > > > > Steffen > > > > On Thu, Nov 29, 2018 at 5:01 PM Joshua Z. Zhang > > wrote: > > > > > Hi, I would like to bring a critical performance and stability patch of > > > existing gluon dataloader to 1.4.0: > > > https://github.com/apache/incubator-mxnet/pull/13447 < > > > https://github.com/apache/incubator-mxnet/pull/13447>. > > > > > > This PR is finished, waiting for CI to pass. > > > > > > Steffen, could you help me add that to the tracked list? > > > > > > Best, > > > Zhi > > > > > > > On Nov 29, 2018, at 4:25 PM, Naveen Swamy > wrote: > > > > > > > > the tests are randomly failing in different stages > > > > > > > > > > http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/incubator-mxnet/detail/PR-13105/ > > > > This PR has failed 8 times so far > > > > > > > > On Thu, Nov 29, 2018 at 3:43 PM Steffen Rochel < > > steffenroc...@gmail.com> > > > > wrote: > > > > > > > >> Pedro - ok. Please add PR to v1.4.x branch after merge to master and > > > please > > > >> update tracking page > > > >> < > > > >> > > > > > > https://cwiki.apache.org/confluence/display/MXNET/Apache+MXNet+%28incubating%29+1.4.0+Release+Plan+and+Status#ApacheMXNet(incubating)1.4.0ReleasePlanandStatus-OpenPRstotrack > > > >>> > > > >> . > > > >> Steffen > > > >> > > > >> On Thu, Nov 29, 2018 at 3:00 PM Pedro Larroy < > > > pedro.larroy.li...@gmail.com > > > >>> > > > >> wrote: > > > >> > > > >>> PR is ready from my side and passes the tests, unless somebody > raises > > > >>> any concerns it's good to go. > > > >>> On Thu, Nov 29, 2018 at 9:50 PM Steffen Rochel < > > > steffenroc...@gmail.com> > > > >>> wrote: > > > >>>> > > > >>>> Pedro - added to 1.4.0 tracking list > > > >>>> < > > > >>> > > > >> > > > > > > https://cwiki.apache.org/confluence/display/MXNET/Apache+MXNet+%28incubating%29+1.4.0+Release+Plan+and+Status#ApacheMXNet(incubating)1.4.0ReleasePlanandStatus-OpenPRstotrack > > > >>>> > > > >>>> > > > >>>> Do you have already ETA? > > > >>>> Steffen > > > >>>> > > > >>>> On Thu, Nov 29, 2018 at 6:13 AM Pedro Larroy < > > > >>> pedro.larroy.li...@gmail.com> > > > >>>> wrote: > > > >>>> > > > >>>>> Hi all. > > > >>>>> > > > >>>>> There are two important issues / fixes that should go in the next > > > >>>>> release in my radar: > > > >>>>> > > > >>>>> 1) https://github.com/apache/incubator-mxnet/pull/13409/files > > > >>>>> There is a bug in shape inference on CPU when not using MKL, also > > we > > > >>>>> are running activation on CPU via MKL when we compile > CUDNN+MKLDNN. > > > >>>>> I'm finishing a fix for these issues in the above PR. > > > >>>>> > > > >>>>> 2) https://github.com/apache/incubator-mxnet/issues/
Re: Regressions in NDArrayIter
I am looking into it. Could you provide the shape of NDArrayIter and minimal reproducible code? Thank you so much! Alfredo Luque 於 2018年9月11日 週二 下午6:05寫道： > Looks like https://github.com/apache/incubator-mxnet/pull/12285 broke a > ton > of our test cases iterating over 3D NDArray instances (eg; MNIST) by > creating an index out of range. > > Stacktrace: > > .com/airbnb/bighead/python/bighead/ml_frameworks/mxnet/gluon.py", line > 434, in transform > for batch in data_iter: > File "/anaconda3/envs/py36/lib/python3.6/site-packages/mxnet/io/io.py", > line 228, in __next__ > return self.next() > File "/anaconda3/envs/py36/lib/python3.6/site-packages/mxnet/io/io.py", > line 680, in next > label = self.getlabel() > File "/anaconda3/envs/py36/lib/python3.6/site-packages/mxnet/io/io.py", > line 750, in getlabel > return self._batchify(self.label) > File "/anaconda3/envs/py36/lib/python3.6/site-packages/mxnet/io/io.py", > line 732, in _batchify > first_data = self._getdata(data_source, start=self.cursor) > File "/anaconda3/envs/py36/lib/python3.6/site-packages/mxnet/io/io.py", > line 694, in _getdata > end = end if end is not None else data_source[0][1].shape[0] > IndexError: list index out of range > > I’ve created an issue at > https://github.com/apache/incubator-mxnet/issues/12526 > > > We’ll be pinning to the previous build until it’s reverted/patched, but let > us know if we can help provide more regression tests here. > > — > Alfredo Luque > Software Engineer > Machine Learning Infrastructure > Airbnb > San Francisco, CA >
