Re: [VOTE] Python 2 Removal for MXNet 1.6
+1 On Tue, Aug 27, 2019 at 3:49 AM Leonard Lausen wrote: > Due to References: header the prior email was still sorted in the > discussion thread. Cancelling this and resending without that header. > > Leonard Lausen writes: > > > Marco de Abreu writes: > >> 1. Which Python version to support. 3.5 vs 3.6 is currently in the > >> discussion due to Ubuntu 16.04 being shipped with 3.5 while the biggest > >> market share being 3.6 as of now. > > > > We could drop Python 2 even before deciding when to drop 3.5. > > > >> 2. When to do the deprecation. EOY to match with official Python 2 > >> deprecation, in 1.5 years to be in line with Ubuntu 16.04 LTS or with > the > >> next major release (2.0) to adhere to semantic versioning. > > > > From a Semantic Versioning standepoint, "Given a version number > > MAJOR.MINOR.PATCH, increment the: MAJOR version when you make > > incompatible API changes, MINOR version when you add functionality in a > > backwards compatible manner, [...]" [1]. > > > > Based on Semantic Versioning, the question is if we consider Python 2 > > support to be part of our API, or rather independent. In the latter > > case, dropping for 1.6 is fine. > > > > From a user-experience perspective, users that want to continue using > > Python 2 for the next 127 days (until EOL date) currently have bigger > > worries than needing to upgrade to the next upcoming MXNet release. They > > must transition their codebase to Py3 within 127 days. For those days, > > they may just stay on MXNet 1.5? > > > > [1]: https://semver.org/ > > > >> Once these points (and any future ones) have been properly discussed and > >> the community came to an agreement, we can formalize it with a voting > >> thread. Until then, I'd recommend to refrain from any actions or > >> user-facing communication regarding this topic. > > > > Thus, let's start a vote on dropping Python 2 for MXNet 1.6. > > It's fine if this vote fails, but we need to get a clear understanding > > how we want to move forward. > > > > For better visibility, I'm removing the In-Reply-To: header, which was > > pointing to > cahtwjdorqsrbau0a89xjwasawgbvgz7bojsu6tkmxdl+ruh...@mail.gmail.com > > > >> On Tue, Aug 27, 2019 at 1:29 AM Pedro Larroy < > pedro.larroy.li...@gmail.com> > >> wrote: > >> > >>> I have sent a PR that removes Python2 from CI. But was closed. I > thought > >>> everyone was +1 on this one. This would remove quite a bit of load on > CI: > >>> > >>> https://github.com/apache/incubator-mxnet/pull/15990 > >>> > >>> If it's not the right time to do this, what steps do we need to take? > >>> > >>> Pedro. >
Re: [Discussion] MXNet 1.5.1 release
Ok. I was just asking if we want this fix in 1.5.1 since it addresses crashes using multiprocessing. The problem with cherry picking is that the patch contains the dynamic load change which shouldn't impact anything else but is not supposed to go in a release branch. On Tue, Aug 27, 2019 at 1:19 PM Lin Yuan wrote: > https://github.com/apache/incubator-mxnet/pull/15762 contains some > unrelated changes which is being reverted. Please do not cherry pick it > yet. > > On Mon, Aug 26, 2019 at 4:25 PM Pedro Larroy > > wrote: > > > There's a fix that I did which seems to still produce crashes in 1.5 for > > some users, which I got notice today and is fixed in master. > > > > Might be useful to put in 1.5.1: > > https://github.com/apache/incubator-mxnet/pull/15762 ? > > > > Pedro. > > > > On Tue, Aug 20, 2019 at 7:49 AM Tao Lv wrote: > > > > > Hi dev, > > > > > > Here is an update for the 1.5.1 patch release. > > > > > > 1. Thanks for the effort from whole community, we have cherry picked a > > > bunch of fixes to v1.5.x branch. So far, the branch looks healthy: > > > > > > > > > http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/NightlyTestsForBinaries/activity/ > > > 2. https://github.com/apache/incubator-mxnet/pull/15803 cannot pass > the > > > CI; > > > 3. I hope julia folks can take a look at the back porting for > > > https://github.com/apache/incubator-mxnet/pull/15609 and > > > https://github.com/apache/incubator-mxnet/pull/15608 - do we still > need > > > them? > > > 4. License issue of cub and pybind is still not fixed. We also has a > > > license issue of a cat image in julia examples. > > > https://github.com/apache/incubator-mxnet/issues/15542 > > > 5. Still no progress for the sidebar issue: > > > https://github.com/apache/incubator-mxnet/issues/15200 > > > 6. There is a GPU OOM issue in 1.5.0 release and already root caused by > > > Lin: > > > > > > > > > https://github.com/apache/incubator-mxnet/issues/15703#issuecomment-522780492 > > > . > > > We need decide whether we want to get it fixed in the 1.5.1 patch > > release. > > > > > > Please find details in > > > > > > > > > https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Plan+and+Status > > > . > > > > > > Thanks, > > > -tao > > > > > > On Mon, Aug 12, 2019 at 9:57 PM Zhao, Patric > > > wrote: > > > > > > > Thanks for the explanation, Marco & Tao. Sounds great! > > > > > > > > > -Original Message- > > > > > From: Tao Lv > > > > > Sent: Monday, August 12, 2019 9:54 PM > > > > > To: dev@mxnet.incubator.apache.org > > > > > Subject: Re: [Discussion] MXNet 1.5.1 release > > > > > > > > > > > Regarding the open issue, is there default code owner/maintainer? > > If > > > > > > so, he/she will be the right people to look into the issue. > > > > > > https://github.com/apache/incubator-mxnet/blob/master/CODEOWNERS > > > > > > > > > > > > > > > > I have no idea. But the CODEOWNERS is used to receive change > > > > notificaitons, > > > > > not actually indicates the maintainer of a piece of code. > > > > > > > > > > Do we have regularly build, run, functionality and performance > > testing > > > > for > > > > > > this release? > > > > > > > > > > > > > > > As Marco mentioned, build, run and functionality of v1.5.x branch > are > > > > tracked > > > > > automatically by the CI for each cherry pick pull request and the > > > > nightly tests > > > > > here: > > > > > http://jenkins.mxnet-ci.amazon- > > > > > ml.com/blue/organizations/jenkins/NightlyTestsForBinaries/activity > . > > > > > I see it's healthy so far. > > > > > > > > > > For performance, Shufan will track CPU performance with his test > > suite > > > > and > > > > > send out the report once the branch is frozen. I'm not sure if > there > > > are > > > > any > > > > > other performance tests. > > > > > > > > > > On Mon, Aug 12, 2019 at 9:36 PM Marco de Abreu > > > > > > > > > > wrote: > > > > > > > > > > > Hi Patric, > > > > > > > > > > > > CI should automatically pick up the branch and validate it as > > usual. > > > > > > > > > > > > Best regards, > > > > > > Marco > > > > > > > > > > > > Zhao, Patric schrieb am Mo., 12. Aug. > > 2019, > > > > 15:22: > > > > > > > > > > > > > It's great works, Tao > > > > > > > > > > > > > > Regarding the open issue, is there default code > owner/maintainer? > > > If > > > > > > > so, he/she will be the right people to look into the issue. > > > > > > > https://github.com/apache/incubator- > > > > > mxnet/blob/master/CODEOWNERS > > > > > > > > > > > > > > Do we have regularly build, run, functionality and performance > > > > > > > testing > > > > > > for > > > > > > > this release? > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > --Patric > > > > > > > > > > > > > > > -Original Message- > > > > > > > > From: Tao Lv > > > > > > > > Sent: Monday, August 12, 2019 8:59 PM > > > > > > > > To: dev@mxnet.incubator.apache.org > > > > > > > > Subject: Re: [Discussion] MXNet 1.5.1 release > > > > >
Re: [Discussion] MXNet 1.5.1 release
https://github.com/apache/incubator-mxnet/pull/15762 contains some unrelated changes which is being reverted. Please do not cherry pick it yet. On Mon, Aug 26, 2019 at 4:25 PM Pedro Larroy wrote: > There's a fix that I did which seems to still produce crashes in 1.5 for > some users, which I got notice today and is fixed in master. > > Might be useful to put in 1.5.1: > https://github.com/apache/incubator-mxnet/pull/15762 ? > > Pedro. > > On Tue, Aug 20, 2019 at 7:49 AM Tao Lv wrote: > > > Hi dev, > > > > Here is an update for the 1.5.1 patch release. > > > > 1. Thanks for the effort from whole community, we have cherry picked a > > bunch of fixes to v1.5.x branch. So far, the branch looks healthy: > > > > > http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/NightlyTestsForBinaries/activity/ > > 2. https://github.com/apache/incubator-mxnet/pull/15803 cannot pass the > > CI; > > 3. I hope julia folks can take a look at the back porting for > > https://github.com/apache/incubator-mxnet/pull/15609 and > > https://github.com/apache/incubator-mxnet/pull/15608 - do we still need > > them? > > 4. License issue of cub and pybind is still not fixed. We also has a > > license issue of a cat image in julia examples. > > https://github.com/apache/incubator-mxnet/issues/15542 > > 5. Still no progress for the sidebar issue: > > https://github.com/apache/incubator-mxnet/issues/15200 > > 6. There is a GPU OOM issue in 1.5.0 release and already root caused by > > Lin: > > > > > https://github.com/apache/incubator-mxnet/issues/15703#issuecomment-522780492 > > . > > We need decide whether we want to get it fixed in the 1.5.1 patch > release. > > > > Please find details in > > > > > https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Plan+and+Status > > . > > > > Thanks, > > -tao > > > > On Mon, Aug 12, 2019 at 9:57 PM Zhao, Patric > > wrote: > > > > > Thanks for the explanation, Marco & Tao. Sounds great! > > > > > > > -Original Message- > > > > From: Tao Lv > > > > Sent: Monday, August 12, 2019 9:54 PM > > > > To: dev@mxnet.incubator.apache.org > > > > Subject: Re: [Discussion] MXNet 1.5.1 release > > > > > > > > > Regarding the open issue, is there default code owner/maintainer? > If > > > > > so, he/she will be the right people to look into the issue. > > > > > https://github.com/apache/incubator-mxnet/blob/master/CODEOWNERS > > > > > > > > > > > > > I have no idea. But the CODEOWNERS is used to receive change > > > notificaitons, > > > > not actually indicates the maintainer of a piece of code. > > > > > > > > Do we have regularly build, run, functionality and performance > testing > > > for > > > > > this release? > > > > > > > > > > > > As Marco mentioned, build, run and functionality of v1.5.x branch are > > > tracked > > > > automatically by the CI for each cherry pick pull request and the > > > nightly tests > > > > here: > > > > http://jenkins.mxnet-ci.amazon- > > > > ml.com/blue/organizations/jenkins/NightlyTestsForBinaries/activity. > > > > I see it's healthy so far. > > > > > > > > For performance, Shufan will track CPU performance with his test > suite > > > and > > > > send out the report once the branch is frozen. I'm not sure if there > > are > > > any > > > > other performance tests. > > > > > > > > On Mon, Aug 12, 2019 at 9:36 PM Marco de Abreu > > > > > > > > wrote: > > > > > > > > > Hi Patric, > > > > > > > > > > CI should automatically pick up the branch and validate it as > usual. > > > > > > > > > > Best regards, > > > > > Marco > > > > > > > > > > Zhao, Patric schrieb am Mo., 12. Aug. > 2019, > > > 15:22: > > > > > > > > > > > It's great works, Tao > > > > > > > > > > > > Regarding the open issue, is there default code owner/maintainer? > > If > > > > > > so, he/she will be the right people to look into the issue. > > > > > > https://github.com/apache/incubator- > > > > mxnet/blob/master/CODEOWNERS > > > > > > > > > > > > Do we have regularly build, run, functionality and performance > > > > > > testing > > > > > for > > > > > > this release? > > > > > > > > > > > > Thanks, > > > > > > > > > > > > --Patric > > > > > > > > > > > > > -Original Message- > > > > > > > From: Tao Lv > > > > > > > Sent: Monday, August 12, 2019 8:59 PM > > > > > > > To: dev@mxnet.incubator.apache.org > > > > > > > Subject: Re: [Discussion] MXNet 1.5.1 release > > > > > > > > > > > > > > Update: > > > > > > > > > > > > > > We're cherry picking fixes from the master to the v1.5.x > branch. > > > > > > > Some > > > > > of > > > > > > > them are already merged. Please find details on the cwiki page: > > > > > > > > > https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Pl > > > > > > > an+a > > > > > > > nd+Status > > > > > > > > > > > > > > > > > > > > > There are still 3 opens: > > > > > > > 1. Nightly test failure on CI ( > > > > > > > https://github.com/apache/incubator-mxnet/issues/15374): The > > issue > > > > > > > is > > > > > > still > > > > > > >
Re: [DISCUSS] PPMC member: Tao Lv
I sincerely apologize for the mistake that I sent this email to the dev@. This discussion thread is not intended for dev@ and I withdraw the discussion from here. -sz On 2019/08/27 18:07:09, Sheng Zha wrote: > Dear PPMC members, > > I'd like to start a discussion on inviting Tao Lv as a PPMC member. > He has been a full committer of our project since Nov. 2018, and has remained > very active > in not only maintaining MKLDNN backend, but many other areas. Over time he > has developed > good knowledge of MXNet and has been helping the project on CPU performance. > He has also been > quite responsive in any issues related to CPU/MKLDNN backend in MXNet. > Recently he has been driving the 1.5.1 release. > > PRs authored - > https://github.com/apache/incubator-mxnet/pulls?utf8=%E2%9C%93=is%3Apr+author%3Ataolv > > PR engagement - > https://github.com/apache/incubator-mxnet/pulls?utf8=%E2%9C%93=is%3Apr+involves%3Ataolv+ > > Issue engagement - > https://github.com/apache/incubator-mxnet/issues?utf8=%E2%9C%93=is%3Aissue+involves%3ATaoLv+ > > dev mailing list - > https://lists.apache.org/list.html?d...@mxnet.apache.org:lte=11M:Tao%20Lv > > I think he would be a great addition to the PPMC and it would be great to > hear what you think. > > Best, > -sz > >
[DISCUSS] PPMC member: Tao Lv
Dear PPMC members, I'd like to start a discussion on inviting Tao Lv as a PPMC member. He has been a full committer of our project since Nov. 2018, and has remained very active in not only maintaining MKLDNN backend, but many other areas. Over time he has developed good knowledge of MXNet and has been helping the project on CPU performance. He has also been quite responsive in any issues related to CPU/MKLDNN backend in MXNet. Recently he has been driving the 1.5.1 release. PRs authored - https://github.com/apache/incubator-mxnet/pulls?utf8=%E2%9C%93=is%3Apr+author%3Ataolv PR engagement - https://github.com/apache/incubator-mxnet/pulls?utf8=%E2%9C%93=is%3Apr+involves%3Ataolv+ Issue engagement - https://github.com/apache/incubator-mxnet/issues?utf8=%E2%9C%93=is%3Aissue+involves%3ATaoLv+ dev mailing list - https://lists.apache.org/list.html?d...@mxnet.apache.org:lte=11M:Tao%20Lv I think he would be a great addition to the PPMC and it would be great to hear what you think. Best, -sz
Re: [Discuss] MXNet Python 3.6 Support Deprecation
Good summary. At the start the discussion thread my ask is to announce the intention of py2 deprecation in the next release, and then actually deprecate py2 in the next major release. Thus, the appropriate timing for dropping py2 support in CI should be the start of the next major release. The py35 vs py36 discussion will not affect the outcome of py2 deprecation. BTW, one alternative option to a formal voting in the Apache way is to through lazy consensus [1], which could apply more in our project. Given the positive feedback in this discussion thread, I will assume lazy consensus in 72hrs on py2 deprecation as defined above. [1] https://community.apache.org/committers/lazyConsensus.html On 2019/08/27 00:19:14, Marco de Abreu wrote: > Pedro, > > thanks for already starting these efforts, but it might be too early for > that. Right now, this is a discussion thread where we try to gather > different opinions in order to lay a good base for a future voting thread. > In there, we would define the detailed timeline, versions etc. Until the > vote has passed, I'd say that it's too early to draw any conclusions. So > far, there are two open discussion points: > > 1. Which Python version to support. 3.5 vs 3.6 is currently in the > discussion due to Ubuntu 16.04 being shipped with 3.5 while the biggest > market share being 3.6 as of now. > 2. When to do the deprecation. EOY to match with official Python 2 > deprecation, in 1.5 years to be in line with Ubuntu 16.04 LTS or with the > next major release (2.0) to adhere to semantic versioning. > > Once these points (and any future ones) have been properly discussed and > the community came to an agreement, we can formalize it with a voting > thread. Until then, I'd recommend to refrain from any actions or > user-facing communication regarding this topic. > > Best regards, > Marco > > On Tue, Aug 27, 2019 at 1:29 AM Pedro Larroy > wrote: > > > I have sent a PR that removes Python2 from CI. But was closed. I thought > > everyone was +1 on this one. This would remove quite a bit of load on CI: > > > > https://github.com/apache/incubator-mxnet/pull/15990 > > > > If it's not the right time to do this, what steps do we need to take? > > > > Pedro. > > > > > > On Mon, Aug 26, 2019 at 1:27 AM Leonard Lausen wrote: > > > > > Lieven Govaerts writes: > > > > Hi, > > > > > > > > On Thu, 22 Aug 2019 at 17:01, Leonard Lausen > > wrote: > > > > > > > >> Hi, > > > >> > > > >> Pedro stated "Seems 3.6 is a reasonable choice." and there have been a > > > >> few +1 after Chaitanya's reply to Pedro. I would like to check if > > these > > > >> only refer to Chaitanya's mail about a dedicated "improvement" effort > > or > > > >> about dropping 3.5. > > > >> > > > >> Thus two questions: > > > >> > > > >> 1) Are there any concerns about dropping Python 3.5? Now is your > > chance > > > to > > > >> speak up if you think so. > > > >> > > > >> > > > > Ubuntu 16.04 LTS defaults to Python 3.5.x . The LTS releases are > > > supported > > > > for 5 years, so for 16.04 LTS it ends in 1.5 years. > > > > > > > > I'm not saying you should wait for 1.5 more years, people can upgrade > > to > > > > 18.04 LTS after all, but may I suggest you make this switch in a major > > > > release only? More specifically, ensure that Python 3.6-only code > > doesn't > > > > accidentally gets merged into a 1.5.X patch release. > > > > > > > > thanks, > > > > > > > > Lieven > > > > > > Hi Lieven, > > > > > > thanks. I believe the Python version compatibility falls under the > > > semantic versioning umbrella of things not to break within any 1.x > > > release. Thus above suggestion would be with respect to a 2.x release or > > > experimental / preview / new features added to 1.x, without affecting > > > existing 1.x features. It would not affect 1.5.x patch releases. > > > > > > Best regards, > > > Leonard > > > > > > > > > >> 2) Should new MXNet 1.x (experimental?) functionality (for example > > numpy > > > >> compatible interface) only target the Python versions to be supported > > in > > > >> MXNet 2? The current plan is to make many MXNet 2 features available > > as > > > >> "opt-in" in MXNet 1.x. Supporting older Python versions on MXNet 1 for > > > >> these features may impact design and functionality and create > > > >> unnecessary technical debt. > > > > > >
Re: [DISCUSS] Apache MXNet: Path to graduation
Who is the gluon "brand owner"? On Tue, Aug 27, 2019 at 10:13 AM Qing Lan wrote: > Hi Lieven, > > Thanks for your comments. After the discussion with several committers and > contributors offline, we agreed that there are space for improvement. > > > 1. About the Gluon naming > > As we know, Gluon is born with the unique API design pattern. It gradually > became the dominant Python front end for MXNet. I would suggest to discuss > more with the Brand owner and see if there could be a further integration > with MXNet. To MXNet itself, it becomes more popular with this frontend. We > lean on the strong community and improve our product better by consuming > the feedback from it. > > 2. Diversity of the PMC > Currently, we have 40 PMC numbers from different companies, like Amazon, > Uber, NVIDIA, ByteDance and a lot more. We are trying to grow the number > and invite indivials from different companies as well as research institute. > > 3. Release rotation > In the history, most of the releases were done by the Amazon side. > Currently, we are moving on to rotate this responsibility with > contributors/committers not from Amazon to start working on them. > > 4. Committers from different firm/institution should have real work on > MXNet > I can tell from the issues/PRs/rfcs they submitted and indeed and indeed > we should encourage the committers who is less active to be involved into > MXNet contribution. > > Thanks, > Qing > > > From: Lieven Govaerts > Sent: Saturday, August 10, 2019 5:59 > To: dev@mxnet.incubator.apache.org > Cc: d...@mxnet.apache.org > Subject: Re: [DISCUSS] Apache MXNet: Path to graduation > > Hi Qing, > > as a user and ASF member observing this project: > > On Sat, 10 Aug 2019 at 01:44, Qing Lan wrote: > > > Hi All, > > > > I would like to start a thread to discuss about the graduation for Apache > > MXNet. From my time working in the community, I saw a great improvement > in > > most of the area that we do to make MXNet a better place. We keep > tracking > > on all of the issues user raised and reviewing PRs. We follow the Apache > > Way to release the package in official repository. > > > > > in terms of code, documentation, visibility this project is certainly in a > healthy state, I see a lot of interest of companies and people, the > community is growing... As a user that gives me confidence my time invested > in this product is well spent. > > > > In 2017, Apache MXNet joined the Apache incubation project. I think now > is > > a good time to review the path to graduate MXNet and move forward to it. > > Please feel free to share your thoughts on graduation and space for > > improvement. > > > > > If I may share one observation: I don't see the community working a lot on > non-code topics. One example that I personally find important is the > discussion of the Gluon brand. People have expressed confusion about how > the name is used by multiple non-ASF projects, the MXNet team finds the > Gluon name very valuable yet the discussion on how to protect the name and > decide on acceptable use by other projects has stalled [1]. I suggest you > make a decision on this topic before you go for graduation. > > regards, > > Lieven > > [1] > > https://mail-archives.apache.org/mod_mbox/mxnet-dev/201903.mbox/%3ccac_cu1gi+3s6ob48kt0x5wta4oxdum8uq9tmnyku2ujyaya...@mail.gmail.com%3e > > > > > > You can find more about graduation policy in here: > > https://incubator.apache.org/guides/graduation.html > > > > Thanks, > > Qing > > >
Re: Update GCC 4.8 dependency?
Just for the sake of completeness, another factor is the python platform tag manylinux2010 compliance [1]. Since it required GLIBC2.12 and GCC4.3, unfortunately even the existing minimum version standard wouldn't allow us to be compliant. [1] https://www.python.org/dev/peps/pep-0571/ On 2019/08/27 15:17:43, "Sunderland, Kellen" wrote: > We could think about moving to a newer version and updating the standard. > I'm using gcc 4.9 with my work builds, but more modern compilers everywhere > else (and is be willing to update the work compiler). > > One of the cons is that it makes our code less portable. When we update the > minimum required compiler (in Linux) then we use a toolchains with a new libc > version, meaning MXNet could not be used on older platforms without using > docker or virtualization. In our case updating to a cpp17 compiler might > mean dropping centos5 or Ubuntu 14.04 support. > > If you look at CUDA releases as an example the continually releases binaries > compiled against older toolchains to support libc on most platforms. > > What are the features you'd like to see in C++17? I'd recommend we call out > interesting features and then see what compiler support we would need to use > the feature. It could be the case that the feature is supported in a fairly > old compiler version. > > If we want to immediately modernize the codebase, I've noticed that their are > actually a few C++14/11 features we could be using but aren't. (Clang-tidy > lists a number of them in each build). We could start with those. > > On Aug 27, 2019 2:53 AM, Leonard Lausen wrote: > Hi, > > "Currently, we only support gcc-4.8 build." [1] > > Do we ever want to change this? gcc-4.8 is now available since more than > 6 years and a lot has happened during that time. Also platforms have > upgraded their default compiler versions, and gcc-7 is now commonly > available (eg. Ubuntu 18.04 LTS, Amazon Linux 2). With gcc-7 we could > for example rely on C++17. > > Wikipedia says: > - GCC since version 7 has complete support for C++17. > - Clang 5 and later implement all the features of C++17. > - Visual Studio 2017 15.7 (MSVC 19.14) supports almost all of C++17. > > As Mu mentioned "Conservatism is not an option" if we want to bring > MXNet forward. The benefits of 6 years of work on compilers as well as > C++ ISO committee work may help us with that. > > Should we adapt a newer compiler toolchain and perhaps C++17 standard? > > Best regards > Leonard > > [1]: > https://github.com/apache/incubator-mxnet/blob/681cfc4/tools/dependencies/README.md > > > On Aug 27, 2019 2:53 AM, Leonard Lausen wrote: > Hi, > > "Currently, we only support gcc-4.8 build." [1] > > Do we ever want to change this? gcc-4.8 is now available since more than > 6 years and a lot has happened during that time. Also platforms have > upgraded their default compiler versions, and gcc-7 is now commonly > available (eg. Ubuntu 18.04 LTS, Amazon Linux 2). With gcc-7 we could > for example rely on C++17. > > Wikipedia says: > - GCC since version 7 has complete support for C++17. > - Clang 5 and later implement all the features of C++17. > - Visual Studio 2017 15.7 (MSVC 19.14) supports almost all of C++17. > > As Mu mentioned "Conservatism is not an option" if we want to bring > MXNet forward. The benefits of 6 years of work on compilers as well as > C++ ISO committee work may help us with that. > > Should we adapt a newer compiler toolchain and perhaps C++17 standard? > > Best regards > Leonard > > [1]: > https://github.com/apache/incubator-mxnet/blob/681cfc4/tools/dependencies/README.md >
Re: [DISCUSS] Apache MXNet: Path to graduation
Hi Lieven, Thanks for your comments. After the discussion with several committers and contributors offline, we agreed that there are space for improvement. 1. About the Gluon naming As we know, Gluon is born with the unique API design pattern. It gradually became the dominant Python front end for MXNet. I would suggest to discuss more with the Brand owner and see if there could be a further integration with MXNet. To MXNet itself, it becomes more popular with this frontend. We lean on the strong community and improve our product better by consuming the feedback from it. 2. Diversity of the PMC Currently, we have 40 PMC numbers from different companies, like Amazon, Uber, NVIDIA, ByteDance and a lot more. We are trying to grow the number and invite indivials from different companies as well as research institute. 3. Release rotation In the history, most of the releases were done by the Amazon side. Currently, we are moving on to rotate this responsibility with contributors/committers not from Amazon to start working on them. 4. Committers from different firm/institution should have real work on MXNet I can tell from the issues/PRs/rfcs they submitted and indeed and indeed we should encourage the committers who is less active to be involved into MXNet contribution. Thanks, Qing From: Lieven Govaerts Sent: Saturday, August 10, 2019 5:59 To: dev@mxnet.incubator.apache.org Cc: d...@mxnet.apache.org Subject: Re: [DISCUSS] Apache MXNet: Path to graduation Hi Qing, as a user and ASF member observing this project: On Sat, 10 Aug 2019 at 01:44, Qing Lan wrote: > Hi All, > > I would like to start a thread to discuss about the graduation for Apache > MXNet. From my time working in the community, I saw a great improvement in > most of the area that we do to make MXNet a better place. We keep tracking > on all of the issues user raised and reviewing PRs. We follow the Apache > Way to release the package in official repository. > > in terms of code, documentation, visibility this project is certainly in a healthy state, I see a lot of interest of companies and people, the community is growing... As a user that gives me confidence my time invested in this product is well spent. > In 2017, Apache MXNet joined the Apache incubation project. I think now is > a good time to review the path to graduate MXNet and move forward to it. > Please feel free to share your thoughts on graduation and space for > improvement. > > If I may share one observation: I don't see the community working a lot on non-code topics. One example that I personally find important is the discussion of the Gluon brand. People have expressed confusion about how the name is used by multiple non-ASF projects, the MXNet team finds the Gluon name very valuable yet the discussion on how to protect the name and decide on acceptable use by other projects has stalled [1]. I suggest you make a decision on this topic before you go for graduation. regards, Lieven [1] https://mail-archives.apache.org/mod_mbox/mxnet-dev/201903.mbox/%3ccac_cu1gi+3s6ob48kt0x5wta4oxdum8uq9tmnyku2ujyaya...@mail.gmail.com%3e > You can find more about graduation policy in here: > https://incubator.apache.org/guides/graduation.html > > Thanks, > Qing >
Re: Update GCC 4.8 dependency?
We could think about moving to a newer version and updating the standard. I'm using gcc 4.9 with my work builds, but more modern compilers everywhere else (and is be willing to update the work compiler). One of the cons is that it makes our code less portable. When we update the minimum required compiler (in Linux) then we use a toolchains with a new libc version, meaning MXNet could not be used on older platforms without using docker or virtualization. In our case updating to a cpp17 compiler might mean dropping centos5 or Ubuntu 14.04 support. If you look at CUDA releases as an example the continually releases binaries compiled against older toolchains to support libc on most platforms. What are the features you'd like to see in C++17? I'd recommend we call out interesting features and then see what compiler support we would need to use the feature. It could be the case that the feature is supported in a fairly old compiler version. If we want to immediately modernize the codebase, I've noticed that their are actually a few C++14/11 features we could be using but aren't. (Clang-tidy lists a number of them in each build). We could start with those. On Aug 27, 2019 2:53 AM, Leonard Lausen wrote: Hi, "Currently, we only support gcc-4.8 build." [1] Do we ever want to change this? gcc-4.8 is now available since more than 6 years and a lot has happened during that time. Also platforms have upgraded their default compiler versions, and gcc-7 is now commonly available (eg. Ubuntu 18.04 LTS, Amazon Linux 2). With gcc-7 we could for example rely on C++17. Wikipedia says: - GCC since version 7 has complete support for C++17. - Clang 5 and later implement all the features of C++17. - Visual Studio 2017 15.7 (MSVC 19.14) supports almost all of C++17. As Mu mentioned "Conservatism is not an option" if we want to bring MXNet forward. The benefits of 6 years of work on compilers as well as C++ ISO committee work may help us with that. Should we adapt a newer compiler toolchain and perhaps C++17 standard? Best regards Leonard [1]: https://github.com/apache/incubator-mxnet/blob/681cfc4/tools/dependencies/README.md On Aug 27, 2019 2:53 AM, Leonard Lausen wrote: Hi, "Currently, we only support gcc-4.8 build." [1] Do we ever want to change this? gcc-4.8 is now available since more than 6 years and a lot has happened during that time. Also platforms have upgraded their default compiler versions, and gcc-7 is now commonly available (eg. Ubuntu 18.04 LTS, Amazon Linux 2). With gcc-7 we could for example rely on C++17. Wikipedia says: - GCC since version 7 has complete support for C++17. - Clang 5 and later implement all the features of C++17. - Visual Studio 2017 15.7 (MSVC 19.14) supports almost all of C++17. As Mu mentioned "Conservatism is not an option" if we want to bring MXNet forward. The benefits of 6 years of work on compilers as well as C++ ISO committee work may help us with that. Should we adapt a newer compiler toolchain and perhaps C++17 standard? Best regards Leonard [1]: https://github.com/apache/incubator-mxnet/blob/681cfc4/tools/dependencies/README.md
Re: [VOTE] Python 2 Removal for MXNet 1.6
Due to References: header the prior email was still sorted in the discussion thread. Cancelling this and resending without that header. Leonard Lausen writes: > Marco de Abreu writes: >> 1. Which Python version to support. 3.5 vs 3.6 is currently in the >> discussion due to Ubuntu 16.04 being shipped with 3.5 while the biggest >> market share being 3.6 as of now. > > We could drop Python 2 even before deciding when to drop 3.5. > >> 2. When to do the deprecation. EOY to match with official Python 2 >> deprecation, in 1.5 years to be in line with Ubuntu 16.04 LTS or with the >> next major release (2.0) to adhere to semantic versioning. > > From a Semantic Versioning standepoint, "Given a version number > MAJOR.MINOR.PATCH, increment the: MAJOR version when you make > incompatible API changes, MINOR version when you add functionality in a > backwards compatible manner, [...]" [1]. > > Based on Semantic Versioning, the question is if we consider Python 2 > support to be part of our API, or rather independent. In the latter > case, dropping for 1.6 is fine. > > From a user-experience perspective, users that want to continue using > Python 2 for the next 127 days (until EOL date) currently have bigger > worries than needing to upgrade to the next upcoming MXNet release. They > must transition their codebase to Py3 within 127 days. For those days, > they may just stay on MXNet 1.5? > > [1]: https://semver.org/ > >> Once these points (and any future ones) have been properly discussed and >> the community came to an agreement, we can formalize it with a voting >> thread. Until then, I'd recommend to refrain from any actions or >> user-facing communication regarding this topic. > > Thus, let's start a vote on dropping Python 2 for MXNet 1.6. > It's fine if this vote fails, but we need to get a clear understanding > how we want to move forward. > > For better visibility, I'm removing the In-Reply-To: header, which was > pointing to cahtwjdorqsrbau0a89xjwasawgbvgz7bojsu6tkmxdl+ruh...@mail.gmail.com > >> On Tue, Aug 27, 2019 at 1:29 AM Pedro Larroy >> wrote: >> >>> I have sent a PR that removes Python2 from CI. But was closed. I thought >>> everyone was +1 on this one. This would remove quite a bit of load on CI: >>> >>> https://github.com/apache/incubator-mxnet/pull/15990 >>> >>> If it's not the right time to do this, what steps do we need to take? >>> >>> Pedro.
[VOTE] Python 2 Removal for MXNet 1.6
Marco de Abreu writes: > 1. Which Python version to support. 3.5 vs 3.6 is currently in the > discussion due to Ubuntu 16.04 being shipped with 3.5 while the biggest > market share being 3.6 as of now. We could drop Python 2 even before deciding when to drop 3.5. > 2. When to do the deprecation. EOY to match with official Python 2 > deprecation, in 1.5 years to be in line with Ubuntu 16.04 LTS or with the > next major release (2.0) to adhere to semantic versioning. >From a Semantic Versioning standepoint, "Given a version number MAJOR.MINOR.PATCH, increment the: MAJOR version when you make incompatible API changes, MINOR version when you add functionality in a backwards compatible manner, [...]" [1]. Based on Semantic Versioning, the question is if we consider Python 2 support to be part of our API, or rather independent. In the latter case, dropping for 1.6 is fine. >From a user-experience perspective, users that want to continue using Python 2 for the next 127 days (until EOL date) currently have bigger worries than needing to upgrade to the next upcoming MXNet release. They must transition their codebase to Py3 within 127 days. For those days, they may just stay on MXNet 1.5? [1]: https://semver.org/ > Once these points (and any future ones) have been properly discussed and > the community came to an agreement, we can formalize it with a voting > thread. Until then, I'd recommend to refrain from any actions or > user-facing communication regarding this topic. Thus, let's start a vote on dropping Python 2 for MXNet 1.6. It's fine if this vote fails, but we need to get a clear understanding how we want to move forward. > On Tue, Aug 27, 2019 at 1:29 AM Pedro Larroy > wrote: > >> I have sent a PR that removes Python2 from CI. But was closed. I thought >> everyone was +1 on this one. This would remove quite a bit of load on CI: >> >> https://github.com/apache/incubator-mxnet/pull/15990 >> >> If it's not the right time to do this, what steps do we need to take? >> >> Pedro.
[VOTE] Python 2 Removal for MXNet 1.6
Marco de Abreu writes: > 1. Which Python version to support. 3.5 vs 3.6 is currently in the > discussion due to Ubuntu 16.04 being shipped with 3.5 while the biggest > market share being 3.6 as of now. We could drop Python 2 even before deciding when to drop 3.5. > 2. When to do the deprecation. EOY to match with official Python 2 > deprecation, in 1.5 years to be in line with Ubuntu 16.04 LTS or with the > next major release (2.0) to adhere to semantic versioning. >From a Semantic Versioning standepoint, "Given a version number MAJOR.MINOR.PATCH, increment the: MAJOR version when you make incompatible API changes, MINOR version when you add functionality in a backwards compatible manner, [...]" [1]. Based on Semantic Versioning, the question is if we consider Python 2 support to be part of our API, or rather independent. In the latter case, dropping for 1.6 is fine. >From a user-experience perspective, users that want to continue using Python 2 for the next 127 days (until EOL date) currently have bigger worries than needing to upgrade to the next upcoming MXNet release. They must transition their codebase to Py3 within 127 days. For those days, they may just stay on MXNet 1.5? [1]: https://semver.org/ > Once these points (and any future ones) have been properly discussed and > the community came to an agreement, we can formalize it with a voting > thread. Until then, I'd recommend to refrain from any actions or > user-facing communication regarding this topic. Thus, let's start a vote on dropping Python 2 for MXNet 1.6. It's fine if this vote fails, but we need to get a clear understanding how we want to move forward. For better visibility, I'm removing the In-Reply-To: header, which was pointing to cahtwjdorqsrbau0a89xjwasawgbvgz7bojsu6tkmxdl+ruh...@mail.gmail.com > On Tue, Aug 27, 2019 at 1:29 AM Pedro Larroy > wrote: > >> I have sent a PR that removes Python2 from CI. But was closed. I thought >> everyone was +1 on this one. This would remove quite a bit of load on CI: >> >> https://github.com/apache/incubator-mxnet/pull/15990 >> >> If it's not the right time to do this, what steps do we need to take? >> >> Pedro.
Update GCC 4.8 dependency?
Hi, "Currently, we only support gcc-4.8 build." [1] Do we ever want to change this? gcc-4.8 is now available since more than 6 years and a lot has happened during that time. Also platforms have upgraded their default compiler versions, and gcc-7 is now commonly available (eg. Ubuntu 18.04 LTS, Amazon Linux 2). With gcc-7 we could for example rely on C++17. Wikipedia says: - GCC since version 7 has complete support for C++17. - Clang 5 and later implement all the features of C++17. - Visual Studio 2017 15.7 (MSVC 19.14) supports almost all of C++17. As Mu mentioned "Conservatism is not an option" if we want to bring MXNet forward. The benefits of 6 years of work on compilers as well as C++ ISO committee work may help us with that. Should we adapt a newer compiler toolchain and perhaps C++17 standard? Best regards Leonard [1]: https://github.com/apache/incubator-mxnet/blob/681cfc4/tools/dependencies/README.md
