Re: CI and PRs
Hi Chris, you are reading some confrontational or negative things where there is no bad intention and just diverse opinions and ways to express them. We went with Marco for a beer and dinner together and talked about this and we had a good exchange of technical ideas and opinions with mutual respect, often is much easier to talk in person than getting the wrong interpretation over email. (Isn't Apache about community over code?) You and me should do it sometime if you want. I send you an initial beer emoji as a friendly token of good intention. Marco's role in the project and in CI, his PMC role or contributions were never put into question. The question is how can we have a more diverse contributions and making it easier to do so to grow the community and help people contribute. Giving credit, acknowledging that many activities are a team effort and supporting them are some ideas that I think might be useful looking forward. My proposal is to acknowledge those contributions and be more inclusive now that the remaining infrastructure is open sourced. Pedro. On Fri, Aug 23, 2019 at 7:43 PM Chris Olivier wrote: > Pedro, > > I don’t see where Marco says that he “designed and implemented all aspects > of CI by himself”. I do think, however, that it’s fair to say that Marco > was in charge of the design and most likely made the majority of design > decisions as the CI was being built, especially around those tenents that > he mentioned. I know this because before I submitted Marco as a committer, > I asked some his teammates whether Marco was really responsible for CI and > the answer by all I asked were that CI was Marco's baby and he did most of > it by some large margin (I am paraphrasing). Taking other design inputs > and examples (i.e. Apache CI) is all part of any responsible design > process. > > In addition, I am not understanding the obfuscation of “people who > contributed to CI”, “person/people who designed CI”, or even > "person who oversees CI" as it is weaponized in your email. Again, nowhere > did Marco say that he did everything back then or since then. I don't > think it's fair to try to modify what Marco wrote and then try to turn it > against him. Reminds me of the techniques of network news these days, > quite frankly (whichever side you're "on" doesn't matter, because both > sides do it). > > -Chris > > > > > > On Fri, Aug 23, 2019 at 3:56 PM Pedro Larroy > > wrote: > > > Thanks for your response Marco, I think you have totally missed my > original > > point which was basically that someone volunteering effort on the CI is > as > > important as someone contributing a feature. From my perspective this > > hasn't been the case, and we had to rely a lot on you and Sheng to submit > > fixes which required access, also to relay communication with Apache > infra. > > Also in many cases we had to rely on you to channel fixes, PRs, disable > > tests etc. If the community is fine having this kind of bottleneck, fine > > with me. From my point of view and the feedback from myself and other > > people which contributed to CI this was not always a good experience. > > Having a welcoming and inclusive community is very important. I don't > want > > to start a discussion on this, but invite the community to do a bit of > soul > > searching on this topic, now that the infrastructure is open source. > > > > Also I find surprising that you claim that you designed the CI yourself, > > when this was a joint work of many individuals, including the old Apache > CI > > and additional contributions and code reviewers, people who were oncall > for > > this service or the autoscaling approach which if I remember correctly > came > > from a humble servant. Kellen did a lot of pair programming and code > > reviews. Obviously you have a done a lot of work on CI which has had a > huge > > positive impact on the project and your recognition is well deserved. The > > technical details you mention on your email are perfectly true and valid. > > > > Below is a rough list of individuals who contributed to CI, I would like > to > > thank all of them since without this work, we wouldn't be able to deliver > > with the quality that we have done in the past. > > > > > > pllarroy@mac:0: ~/d/m/ci [fc_higher_order_grad_2]> git log > > --pretty=format:%aN . | sort | uniq -c | sort -n | tail -n 10 > >6 Zach Kimberg > >6 stu1130 > >7 Jake Lee > >8 Aaron Markham > > 11 Lanking > > 12 Anton Chernov > > 13 perdasilva > > 26 Kellen Sunderland > > 34 Marco de Abreu > > 46 Pedro Larroy > > > > pllarroy@mac:0: ~/d/mxnet_ci_general [master]> git log > --pretty=format:%aN > > | sort | uniq -c | sort -n > >1 Gavin M. Bell > >1 de Abreu > >6 Bair > >7 Kellen Sunderland > >8 Jose Luis Contreras > > 14 perdasilva > > 20 Per Goncalves da Silva > > 29 Anton Chernov > > 39 Chance Bair > > 96 Pedro Larroy > > 209 Marco de Abreu > > > > > > > > Pedro. > > > > On Fri,
Re: [Discuss] MXNet Python < 3.6 Support Deprecation
I have sent a PR that removes Python2 from CI. But was closed. I thought everyone was +1 on this one. This would remove quite a bit of load on CI: https://github.com/apache/incubator-mxnet/pull/15990 If it's not the right time to do this, what steps do we need to take? Pedro. On Mon, Aug 26, 2019 at 1:27 AM Leonard Lausen wrote: > Lieven Govaerts writes: > > Hi, > > > > On Thu, 22 Aug 2019 at 17:01, Leonard Lausen wrote: > > > >> Hi, > >> > >> Pedro stated "Seems 3.6 is a reasonable choice." and there have been a > >> few +1 after Chaitanya's reply to Pedro. I would like to check if these > >> only refer to Chaitanya's mail about a dedicated "improvement" effort or > >> about dropping 3.5. > >> > >> Thus two questions: > >> > >> 1) Are there any concerns about dropping Python 3.5? Now is your chance > to > >> speak up if you think so. > >> > >> > > Ubuntu 16.04 LTS defaults to Python 3.5.x . The LTS releases are > supported > > for 5 years, so for 16.04 LTS it ends in 1.5 years. > > > > I'm not saying you should wait for 1.5 more years, people can upgrade to > > 18.04 LTS after all, but may I suggest you make this switch in a major > > release only? More specifically, ensure that Python 3.6-only code doesn't > > accidentally gets merged into a 1.5.X patch release. > > > > thanks, > > > > Lieven > > Hi Lieven, > > thanks. I believe the Python version compatibility falls under the > semantic versioning umbrella of things not to break within any 1.x > release. Thus above suggestion would be with respect to a 2.x release or > experimental / preview / new features added to 1.x, without affecting > existing 1.x features. It would not affect 1.5.x patch releases. > > Best regards, > Leonard > > > >> 2) Should new MXNet 1.x (experimental?) functionality (for example numpy > >> compatible interface) only target the Python versions to be supported in > >> MXNet 2? The current plan is to make many MXNet 2 features available as > >> "opt-in" in MXNet 1.x. Supporting older Python versions on MXNet 1 for > >> these features may impact design and functionality and create > >> unnecessary technical debt. >
Re: [Discuss] MXNet Python < 3.6 Support Deprecation
Pedro, thanks for already starting these efforts, but it might be too early for that. Right now, this is a discussion thread where we try to gather different opinions in order to lay a good base for a future voting thread. In there, we would define the detailed timeline, versions etc. Until the vote has passed, I'd say that it's too early to draw any conclusions. So far, there are two open discussion points: 1. Which Python version to support. 3.5 vs 3.6 is currently in the discussion due to Ubuntu 16.04 being shipped with 3.5 while the biggest market share being 3.6 as of now. 2. When to do the deprecation. EOY to match with official Python 2 deprecation, in 1.5 years to be in line with Ubuntu 16.04 LTS or with the next major release (2.0) to adhere to semantic versioning. Once these points (and any future ones) have been properly discussed and the community came to an agreement, we can formalize it with a voting thread. Until then, I'd recommend to refrain from any actions or user-facing communication regarding this topic. Best regards, Marco On Tue, Aug 27, 2019 at 1:29 AM Pedro Larroy wrote: > I have sent a PR that removes Python2 from CI. But was closed. I thought > everyone was +1 on this one. This would remove quite a bit of load on CI: > > https://github.com/apache/incubator-mxnet/pull/15990 > > If it's not the right time to do this, what steps do we need to take? > > Pedro. > > > On Mon, Aug 26, 2019 at 1:27 AM Leonard Lausen wrote: > > > Lieven Govaerts writes: > > > Hi, > > > > > > On Thu, 22 Aug 2019 at 17:01, Leonard Lausen > wrote: > > > > > >> Hi, > > >> > > >> Pedro stated "Seems 3.6 is a reasonable choice." and there have been a > > >> few +1 after Chaitanya's reply to Pedro. I would like to check if > these > > >> only refer to Chaitanya's mail about a dedicated "improvement" effort > or > > >> about dropping 3.5. > > >> > > >> Thus two questions: > > >> > > >> 1) Are there any concerns about dropping Python 3.5? Now is your > chance > > to > > >> speak up if you think so. > > >> > > >> > > > Ubuntu 16.04 LTS defaults to Python 3.5.x . The LTS releases are > > supported > > > for 5 years, so for 16.04 LTS it ends in 1.5 years. > > > > > > I'm not saying you should wait for 1.5 more years, people can upgrade > to > > > 18.04 LTS after all, but may I suggest you make this switch in a major > > > release only? More specifically, ensure that Python 3.6-only code > doesn't > > > accidentally gets merged into a 1.5.X patch release. > > > > > > thanks, > > > > > > Lieven > > > > Hi Lieven, > > > > thanks. I believe the Python version compatibility falls under the > > semantic versioning umbrella of things not to break within any 1.x > > release. Thus above suggestion would be with respect to a 2.x release or > > experimental / preview / new features added to 1.x, without affecting > > existing 1.x features. It would not affect 1.5.x patch releases. > > > > Best regards, > > Leonard > > > > > > >> 2) Should new MXNet 1.x (experimental?) functionality (for example > numpy > > >> compatible interface) only target the Python versions to be supported > in > > >> MXNet 2? The current plan is to make many MXNet 2 features available > as > > >> "opt-in" in MXNet 1.x. Supporting older Python versions on MXNet 1 for > > >> these features may impact design and functionality and create > > >> unnecessary technical debt. > > >
Re: [Discussion] MXNet 1.5.1 release
There's a fix that I did which seems to still produce crashes in 1.5 for some users, which I got notice today and is fixed in master. Might be useful to put in 1.5.1: https://github.com/apache/incubator-mxnet/pull/15762 ? Pedro. On Tue, Aug 20, 2019 at 7:49 AM Tao Lv wrote: > Hi dev, > > Here is an update for the 1.5.1 patch release. > > 1. Thanks for the effort from whole community, we have cherry picked a > bunch of fixes to v1.5.x branch. So far, the branch looks healthy: > > http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/NightlyTestsForBinaries/activity/ > 2. https://github.com/apache/incubator-mxnet/pull/15803 cannot pass the > CI; > 3. I hope julia folks can take a look at the back porting for > https://github.com/apache/incubator-mxnet/pull/15609 and > https://github.com/apache/incubator-mxnet/pull/15608 - do we still need > them? > 4. License issue of cub and pybind is still not fixed. We also has a > license issue of a cat image in julia examples. > https://github.com/apache/incubator-mxnet/issues/15542 > 5. Still no progress for the sidebar issue: > https://github.com/apache/incubator-mxnet/issues/15200 > 6. There is a GPU OOM issue in 1.5.0 release and already root caused by > Lin: > > https://github.com/apache/incubator-mxnet/issues/15703#issuecomment-522780492 > . > We need decide whether we want to get it fixed in the 1.5.1 patch release. > > Please find details in > > https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Plan+and+Status > . > > Thanks, > -tao > > On Mon, Aug 12, 2019 at 9:57 PM Zhao, Patric > wrote: > > > Thanks for the explanation, Marco & Tao. Sounds great! > > > > > -Original Message- > > > From: Tao Lv > > > Sent: Monday, August 12, 2019 9:54 PM > > > To: dev@mxnet.incubator.apache.org > > > Subject: Re: [Discussion] MXNet 1.5.1 release > > > > > > > Regarding the open issue, is there default code owner/maintainer? If > > > > so, he/she will be the right people to look into the issue. > > > > https://github.com/apache/incubator-mxnet/blob/master/CODEOWNERS > > > > > > > > > > I have no idea. But the CODEOWNERS is used to receive change > > notificaitons, > > > not actually indicates the maintainer of a piece of code. > > > > > > Do we have regularly build, run, functionality and performance testing > > for > > > > this release? > > > > > > > > > As Marco mentioned, build, run and functionality of v1.5.x branch are > > tracked > > > automatically by the CI for each cherry pick pull request and the > > nightly tests > > > here: > > > http://jenkins.mxnet-ci.amazon- > > > ml.com/blue/organizations/jenkins/NightlyTestsForBinaries/activity. > > > I see it's healthy so far. > > > > > > For performance, Shufan will track CPU performance with his test suite > > and > > > send out the report once the branch is frozen. I'm not sure if there > are > > any > > > other performance tests. > > > > > > On Mon, Aug 12, 2019 at 9:36 PM Marco de Abreu > > > > > > wrote: > > > > > > > Hi Patric, > > > > > > > > CI should automatically pick up the branch and validate it as usual. > > > > > > > > Best regards, > > > > Marco > > > > > > > > Zhao, Patric schrieb am Mo., 12. Aug. 2019, > > 15:22: > > > > > > > > > It's great works, Tao > > > > > > > > > > Regarding the open issue, is there default code owner/maintainer? > If > > > > > so, he/she will be the right people to look into the issue. > > > > > https://github.com/apache/incubator- > > > mxnet/blob/master/CODEOWNERS > > > > > > > > > > Do we have regularly build, run, functionality and performance > > > > > testing > > > > for > > > > > this release? > > > > > > > > > > Thanks, > > > > > > > > > > --Patric > > > > > > > > > > > -Original Message- > > > > > > From: Tao Lv > > > > > > Sent: Monday, August 12, 2019 8:59 PM > > > > > > To: dev@mxnet.incubator.apache.org > > > > > > Subject: Re: [Discussion] MXNet 1.5.1 release > > > > > > > > > > > > Update: > > > > > > > > > > > > We're cherry picking fixes from the master to the v1.5.x branch. > > > > > > Some > > > > of > > > > > > them are already merged. Please find details on the cwiki page: > > > > > > > https://cwiki.apache.org/confluence/display/MXNET/1.5.1+Release+Pl > > > > > > an+a > > > > > > nd+Status > > > > > > > > > > > > > > > > > > There are still 3 opens: > > > > > > 1. Nightly test failure on CI ( > > > > > > https://github.com/apache/incubator-mxnet/issues/15374): The > issue > > > > > > is > > > > > still > > > > > > open. I'm wondering if it has been fixed or not. If not, is there > > > > anyone > > > > > > working on it? > > > > > > 2. Broken Sidebar on website API for master and 1.5.0 ( > > > > > > https://github.com/apache/incubator-mxnet/issues/15200): I don't > > > > > > see > > > > any > > > > > > progress on this issue? Do we still want to include it into 1.5.1 > > > > > > patch > > > > > release? > > > > > > 3. License issues need to be fixed before 1.6 release ( > > > > > >
Re: [Discuss] MXNet Python < 3.6 Support Deprecation
Lieven Govaerts writes: > Hi, > > On Thu, 22 Aug 2019 at 17:01, Leonard Lausen wrote: > >> Hi, >> >> Pedro stated "Seems 3.6 is a reasonable choice." and there have been a >> few +1 after Chaitanya's reply to Pedro. I would like to check if these >> only refer to Chaitanya's mail about a dedicated "improvement" effort or >> about dropping 3.5. >> >> Thus two questions: >> >> 1) Are there any concerns about dropping Python 3.5? Now is your chance to >> speak up if you think so. >> >> > Ubuntu 16.04 LTS defaults to Python 3.5.x . The LTS releases are supported > for 5 years, so for 16.04 LTS it ends in 1.5 years. > > I'm not saying you should wait for 1.5 more years, people can upgrade to > 18.04 LTS after all, but may I suggest you make this switch in a major > release only? More specifically, ensure that Python 3.6-only code doesn't > accidentally gets merged into a 1.5.X patch release. > > thanks, > > Lieven Hi Lieven, thanks. I believe the Python version compatibility falls under the semantic versioning umbrella of things not to break within any 1.x release. Thus above suggestion would be with respect to a 2.x release or experimental / preview / new features added to 1.x, without affecting existing 1.x features. It would not affect 1.5.x patch releases. Best regards, Leonard >> 2) Should new MXNet 1.x (experimental?) functionality (for example numpy >> compatible interface) only target the Python versions to be supported in >> MXNet 2? The current plan is to make many MXNet 2 features available as >> "opt-in" in MXNet 1.x. Supporting older Python versions on MXNet 1 for >> these features may impact design and functionality and create >> unnecessary technical debt.
