Re: [Discuss] MXNet Python < 3.6 Support Deprecation
+1 On Thu, Aug 22, 2019 at 11:22 PM Junru Shao wrote: > +1 for 3.6+ > > On Thu, Aug 22, 2019 at 8:54 AM Marco de Abreu > wrote: > > > +1 for 3.6+ > > > > Yuan Tang schrieb am Do., 22. Aug. 2019, > 08:08: > > > > > +1 to target 3.6+ > > > > > > On Thu, Aug 22, 2019 at 11:01 AM Leonard Lausen > > wrote: > > > > > > > Hi, > > > > > > > > Pedro stated "Seems 3.6 is a reasonable choice." and there have been > a > > > > few +1 after Chaitanya's reply to Pedro. I would like to check if > these > > > > only refer to Chaitanya's mail about a dedicated "improvement" effort > > or > > > > about dropping 3.5. > > > > > > > > Thus two questions: > > > > > > > > 1) Are there any concerns about dropping Python 3.5? Now is your > chance > > > to > > > > speak up if you think so. > > > > > > > > 2) Should new MXNet 1.x (experimental?) functionality (for example > > numpy > > > > compatible interface) only target the Python versions to be supported > > in > > > > MXNet 2? The current plan is to make many MXNet 2 features available > as > > > > "opt-in" in MXNet 1.x. Supporting older Python versions on MXNet 1 > for > > > > these features may impact design and functionality and create > > > > unnecessary technical debt. > > > > > > > > > > > > Personally I argue for targeting only 3.6+ as > > > > - 3.5 will go EOL in 388 days and a potential MXNet 2 release > together > > > > with our Semantic Versioning backwards compatibility guarantees > would > > > > keep us "stuck" on 3.5 for the years to come. JetBrains 2018 survey > > > > showed only 11% of users used 3.5. > > > > - 3.6 introduced a number of fundamental and relevant changes that we > > > > may want to build on and for which we can expect user adoption to > > > > increase over the years (thus MXNet should try to be compatible). > > > > - "PEP 526: Syntax for variable annotations" which we may even be > > able > > > > to use for shape typing along the lines of numpy > > > > > > > > > > > > > > https://docs.google.com/document/d/1vpMse4c6DrWH5rq2tQSx3qwP_m_0lyn-Ij4WHqQqRHY/ > > > > - asyncio module is stable with 3.6 and associated 3.7 language > > > > features such as contextvars only have backports for 3.6. Some > > parts > > > > of Gluon currently rely on thread-local state, which is not > correct > > > > if users call MXNet from within asyncio code. > > > > Locking ourselves to 3.5 means we can't support these and may > provide > > > > a bad user-experience in coming years. > > > > - Part of the Ecosystem (GluonNLP) only support 3.6+ anyways. > > > > > > > > I would also like to cite James MacGlashan to point out how targeting > > > > 3.6+ could help usability and attract more users: > > > > > > > > Pipe dream: I'd love it if Mxnet not only dropped Python 2 support > > for > > > > a more consistent design, but also went all in on Python 3.6 for > type > > > > hint integration. There are enough different types involved in > MXNet > > > > that types can help clarify usage, particularly for disambiguating > > > > symbol vs ndarray vs list vs tuple; tuple of ints rather than tuple > > of > > > > floats; etc. > > > > > > > > > > > > > > https://github.com/apache/incubator-mxnet/issues/8703#issuecomment-520881450 > > > > > > > > Thus we can see targeting 3.6+ as a great opportunity for the MXNet > > > > project! > > > > > > > > Best regards > > > > Leonard > > > > > > > > "Srivastava, Rohit Kumar" > writes: > > > > > +1 > > > > > > > > > > On 7/19/19, 12:59 PM, "Zhu Zhaoqi" wrote: > > > > > > > > > > +1 > > > > > > > > > > Lin Yuan 于2019年7月19日周五 上午12:06写道： > > > > > > > > > > > +1 > > > > > > > > > > > > On Fri, Jul 19, 2019 at 12:03 AM Chaitanya Bapat < > > > > chai.ba...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > +1 definitely. > > > > > > > > > > > > > > Going forward, > > > > > > > MXNet repo as it stands has ~95,000+ lines of Python code > [1] > > > > > > > OpenEdx has a million (10x) LOC and this mammoth effort of > > > > porting from > > > > > > > Python 2 to 3 is treated as a separate project named > > > Incremental > > > > > > > Improvement. [2] > > > > > > > We can take inspiration from them and have a similar effort > > by > > > > calling > > > > > > > action from the community. Issues can be maintained in a > > > > separate JIRA > > > > > > > board to track high priority tasks. > > > > > > > > > > > > > > Also, I can see gluon-nlp adding themselves to the Python3 > > > > statement. > > > > > > Once > > > > > > > the vote passes, one of us could submit a PR to add MXNet > as > > > > well. > > > > > > > > > > > > > > [1] https://codeclimate.com/ > > > > > > > [2] > > > > > > > > > > > > > > > > > > > > > > > https://open.edx.org/blog/python-2-is-ending-we-need-to-move-to-python-3/ > > > > > > > > > > > > > > > > > > > > > On Thu, 18 Jul 2019 at 21:39, Kshitij Kalambarkar < > > >
Fwd: ApacheCon Europe 2019: Join our Hackathon!
-- Forwarded message - From: Sally Khudairi Date: Wed, Aug 21, 2019 at 9:23 AM Subject: ApacheCon Europe 2019: Join our Hackathon! To: Dear Apache Committers, There will be a hackathon space at ApacheCon Europe 2019 in Berlin. It will be available on 23rd/24th October from the start of each day’s schedule until 7:00 PM, with the possible exceptions e.g. during keynotes. We want to invite everybody to participate on the hackathon. Collaborative development on project source code, improvements to project documentation, and development of example apps or tools built upon one or more Apache projects are all encouraged. Furthermore, it's a nice opportunity to get in touch with other contributors. More details will be available leading up to the event, but here’s generally what to expect: - Dedicated space with chairs, power, wifi, snacks, and caffeine. - Tables dedicated to specific participating projects. - ‘Getting Started’ discussions for new and aspiring committers - (industrial) IoT Corner with industrial equipment to code against With most logistical concerns now attended to, we are identifying interested participants, promoters, and coordinators. Are you a PMC member or committer willing to do the following on behalf of your project: - Operate a dedicated hackathon table for some time - Designate collaborative work for your project’s table to help hackers focus - Encourage your community to hack at your project’s table - Spread the word throughout your community! Interested? Email plann...@apachecon.com to reserve a table for some time for your project that we can advertise to your community. We are currently working out a Hackathon schedule that will be posted, letting attendees know when they should expect activities related to their project and various open sessions to take place. If you will take on the lead role for your project, we will work with you to plan, prepare, and generate interest in the wider community. If you aren’t interested in hacking yourself but want to help others, we could also use persons who are willing to do one or more of the following: - Operate the primary information table in the hackathon space for several hours - Give a short introductory tools or skills presentation / answer questions Thanks! ACEU Hackathon Organization Team P.S. We invite anyone who has ideas to share with the planning committee, or is considering participating in any way, to let us know via d...@community.apache.org
Re: Disabling, circumventing and altering CI checks
Hi Marco, Thank you for helping CI all the time. You did an incredible job on it. Please let me explain why it’s urgent we need to update our CI to allow fast developing. In this summer, we managed to hire a large amount of interns to help, they did great to contribute to MXNet. But CI is one of the blocking issues. You can see that just for numpy 70+ PRs are open. VP in your org Our engineering VP expressed his concern about the numpy project progress. I heard you are switching to new role in another org, so it may not matter you too much. But it does affect the resource I can have for this project. Conservatism is not an option. MXNet has 10x less PR merged per day compared Pytorch and Tensorflow. MXNet’s downloads are also 10x less. We need to address it quickly. A large amount of contributors are paid to work on MXNet. These companies may withdraw their resources if MXNet is not so successful. We need to continue the PR boom for the numpy project. I saw you are already work on it. But we should be open for other changes. We should optimize our test. I wrote most of the test before 2016 and the original Jenkinsfile. I knew there are a large amount of redundant tests. We can resume some of them. Also some operators are tested with a large amount of random inputs, we can also reduce them. Eg if an op is tested by 1m iterations, we can reduce it to 100k with a changing random seed, so we can still test 1m times with 10 tests. There are a large amount of best practices in software testing we can follow. I hope the community is open for changes, for both short term fixes and long term improvements. Best Mu > On Aug 24, 2019, at 1:01 AM, Marco de Abreu wrote: > > Hello, > > recently I have noticed a trend of CI checks being disabled, circumvented > or altered as part of a Pull Request. While I understand that these systems > might decline your PR, they execute checks and enforce standards that the > community has agreed on. The reason for these standards is not to give > contributors a hard time but instead to improve the maintainability, > compatibility and stability of our code base. > > I acknowledge that there might have been a few issues recently that > resulted in an increased rate of false errors and I agree that this is not > ideal. But none the less, I'd like to encourage everybody to think about > the long-term impact of tampering with these systems. > > Since I prefer to not call out names, I will keep this email generic. But a > few examples of what such a problematic change could look like include: > - Changing Jenkinsfiles > - Changing Dockerfiles > - Changing runtime functions > - Disabling linting or adding exclusions > - Adding additional preprocessor statements that avoid that codepath from > being checked by CI > - Disabling tests or making them conditional to avoid execution by CI > > While I understand that new contributors might not be that used to the dos > and donts of a software project, I'd like to remind the committers of this > project to think in the best interest of the project and adhere to > engineering best practices. While I understand that there might be external > factors that create the urge to take these shortcuts, it's the committers > responsibility to ensure that they don't make their way into the codebase > or get remedied as soon as possible. > > Thanks for your understanding. > > Best regards, > Marco
Disabling, circumventing and altering CI checks
Hello, recently I have noticed a trend of CI checks being disabled, circumvented or altered as part of a Pull Request. While I understand that these systems might decline your PR, they execute checks and enforce standards that the community has agreed on. The reason for these standards is not to give contributors a hard time but instead to improve the maintainability, compatibility and stability of our code base. I acknowledge that there might have been a few issues recently that resulted in an increased rate of false errors and I agree that this is not ideal. But none the less, I'd like to encourage everybody to think about the long-term impact of tampering with these systems. Since I prefer to not call out names, I will keep this email generic. But a few examples of what such a problematic change could look like include: - Changing Jenkinsfiles - Changing Dockerfiles - Changing runtime functions - Disabling linting or adding exclusions - Adding additional preprocessor statements that avoid that codepath from being checked by CI - Disabling tests or making them conditional to avoid execution by CI While I understand that new contributors might not be that used to the dos and donts of a software project, I'd like to remind the committers of this project to think in the best interest of the project and adhere to engineering best practices. While I understand that there might be external factors that create the urge to take these shortcuts, it's the committers responsibility to ensure that they don't make their way into the codebase or get remedied as soon as possible. Thanks for your understanding. Best regards, Marco
