Typo, Should of read: Python 3.9 will lose bugfix support approx. 6 months later on ~2022-04-05.
Damian (he/him) From: Shaw, Damian P. (WTIZ 53) Sent: Friday, November 20, 2020 08:58 To: dev@airflow.apache.org Subject: RE: Default/supported Python versions for Airlfow 2.0 Hi Jarek, Took me a moment to understand what you meant by “latest supported”, you mean the oldest version that is still receiving bugfix support? If I’m reading that correctly then for future versions of Python that will mean that we will need to have the default Airflow moved when the Python release is “only” 6 months old. E.g. On the current schedule Python 3.10 will be released ~2021-10-05, and Python 3.9 will lose bugfix support approx. 6 months later on ~2021-04-05. Of course the whole point of the 1 year release cycle is that the transition to each Python release should be smaller and less troublesome, so perhaps 6 months to migrate to Python 3.10 as the default is enough time. But the comment from Halo earlier in the thread made me think that at least some of the Airflow community might be a slightly slower crowd when it comes to transitions. Damian P Shaw (he/him) From: Jarek Potiuk <jarek.pot...@polidea.com<mailto:jarek.pot...@polidea.com>> Sent: Friday, November 20, 2020 04:10 To: dev@airflow.apache.org<mailto:dev@airflow.apache.org> Subject: Re: Default/supported Python versions for Airlfow 2.0 I do not yet want to cast a formal vote on it, because maybe others have other ideas, but let me formulate what I see as a good proposal now. The "default" in our case means mainly about the one that is used by "non-approved yet PR" (we will continue running tests for all supported versions in master and similarly as yesterday we wil be able to fix them quickly). So I think we can be a bit more aggressive on the "default" version. Just to show an example of what can happen - we had master failing yesterday for Python 3.8 (fixed here - https://github.com/apache/airflow/pull/12481) and I guess we might have more cases like that. The proposal from Damian was to keep up with the schedule of Python releases. I would like to - slightly - modify it to use the latest supported version as "default" to prevent the kind of failures. " I think about such set of rules: * we use the "latest" supported version of Python as the default (3.8 today). * we officially support all Python releases till the end of their EOL We can always decide to support versions past EOL if we have a good reason - but it would have to be justified and voted. How does it sound? Any other proposals ? Just to remind the Python schedule: Branch Schedule Status First release End-of-life 3.9 PEP 596 bugfix 2020-10-05 TBD 3.8 PEP 569 bugfix 2019-10-14 2024-10 3.7 PEP 537 security 2018-06-27 2023-06-27 3.6 PEP 494 security 2016-12-23 2021-12-23 J. -- [https://s3.eu-central-1.amazonaws.com/corgi-mail/23-05-2019/jarek.potiuk/jarpot.jpg] Jarek Potiuk Polidea<https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129<tel:+48660796129> [Polidea]<https://www.polidea.com/> On Tue, Nov 17, 2020 at 8:19 PM Leah Cole <colel...@google.com.invalid<mailto:colel...@google.com.invalid>> wrote: I really like Damian's proposal. With my google devrel work we had been doing something similar to what Jarek had done above - looking at PyPI downloads and at what is most popular, but doing that makes it harder to encourage folks to move towards the future. I like that Python provides this consistent schedule - it will make it easier for us to set expectations with our users, and will align with other Python-y things. :) And, tbh, probably going to propose this in future meetings related to my devrel work so thank you Damien! On Thu, Nov 12, 2020 at 6:46 AM Jarek Potiuk <jarek.pot...@polidea.com<mailto:jarek.pot...@polidea.com>> wrote: I love the idea to have clear rules and tie it to the official schedule of Python releases - at least as a target, because we might find some issues that might prevent us from doing so. Also we should officially support all versions that are not yet reached end-of-life. I like the proposed schedule and yearly cadence. I wonder if others have similar thoughts. Such agreement/policy would require formal voting though I think? WDYT everyone? J. On Thu, Nov 12, 2020 at 3:37 PM Shaw, Damian P. <damian.sha...@credit-suisse.com<mailto:damian.sha...@credit-suisse.com>> wrote: I just wanted to add that if people are not aware PEP 0602<https://www.python.org/dev/peps/pep-0602> has been accepted and implemented for Python 3.9. This means 3 things for the Python release cycle: 1. A new version every 12 months 2. Each version receives 18 months of full support (bug fixes and security fixes) 3. After full support has ended each version receives an additional 42 months of security updates Going forward I think it makes sense to bump up the default version of Python every 1 year in cadence with the Python release cycle. Assuming people agreed the question would be how far behind should Airflow be from the new release? Personally I feel like no more than 18 months is a good, in the new Python release cadence that version of Python will no longer be receiving bug fixes and therefore will be very stable, and 18 months is a good enough time for any libraries and providers to be available (if they’re not available after 18 months maybe they have given up support?) If we retroactively apply this to the previous releases of Python that would put us at Python 3.7 default now and Python 3.8 default ~April 14, 2021. My 2 cents, Damian From: Jarek Potiuk <jarek.pot...@polidea.com<mailto:jarek.pot...@polidea.com>> Sent: Thursday, November 12, 2020 09:03 To: dev@airflow.apache.org<mailto:dev@airflow.apache.org> Subject: Re: Default/supported Python versions for Airlfow 2.0 Should we make Python 3.7 default then and leave all others as-is ? J. On Thu, Nov 5, 2020 at 1:48 PM Kaxil Naik <kaxiln...@gmail.com<mailto:kaxiln...@gmail.com>> wrote: We should definitely support Python 3.6 to make the Upgrades to Airflow 2.0 a bit easier. As of yesterday, checks these stats from PyPI downloads: Py3.7: 12,578 Py3.6: 9,806 Py3.8: 1,815 On Thu, Nov 5, 2020 at 11:40 AM Halo Ku <hal...@mail.com<mailto:hal...@mail.com>> wrote: If I may point that Airflow is a wokrflow managment system and as such the power of the tool is in direct extention to the levrage providers. This should also be checked from how many of the providers are compatible with 3.8 / 3.9 Sent: Thursday, November 05, 2020 at 1:16 PM From: "Ash Berlin-Taylor" <a...@apache.org<mailto:a...@apache.org>> To: "dev@airflow.apache.org<mailto:dev@airflow.apache.org>" <dev@airflow.apache.org<mailto:dev@airflow.apache.org>> Cc: "dev@airflow.apache.org<mailto:dev@airflow.apache.org>" <dev@airflow.apache.org<mailto:dev@airflow.apache.org>> Subject: Re: Default/supported Python versions for Airlfow 2.0 Debian stable ships python 3.7(.3) CentOS 8 has two packages - python36 and python38 Ubuntu 18.04 (LTS) has 3.6.5 Ubuntu 20.04 (LTS) has 3.8.2 (https://pkgs.org/search/?q=python3&on=files) RHEL is harder to find out about . RHEL8 has python 3.6 as python3, and RHEL 8.2 has Py3.8 as a separate package https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/configuring_basic_system_settings/index#using-python3_configuring-basic-system-settings So for default version 3.8 or 3.9 gets my vote. I think the cost/burden of supporting back to 3.6 is not very great, so we should continue to support (and I guess test) that. -ash On Nov 5 2020, at 8:49 am, Jarek Potiuk <jarek.pot...@polidea.com<mailto:jarek.pot...@polidea.com>> wrote: Hello Everyone, I have a question. What do people think about default version of Pyhon for Airflow 2.0 (and set of supported versions)? Currently, we have python 3.6 as default, but all the version up to 3.8 are officially supported and tested and PR for python 3.9 is in Draft: https://github.com/apache/airflow/pull/11950 This is the release schedule for python versions. We have a year till the end of 3.6 Branch Schedule Status First release End-of-life 3.9 PEP 596 bugfix 2020-10-05 TBD 3.8 PEP 569 bugfix 2019-10-14 2024-10 3.7 PEP 537 security 2018-06-27 2023-06-27 3.6 PEP 494 security 2016-12-23 2021-12-23 WDYT? J. -- [https://s3.eu-central-1.amazonaws.com/corgi-mail/23-05-2019/jarek.potiuk/jarpot.jpg] Jarek Potiuk Polidea<https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129<tel:+48%20660%20796%20129> [Polidea]<https://www.polidea.com/> -- [https://s3.eu-central-1.amazonaws.com/corgi-mail/23-05-2019/jarek.potiuk/jarpot.jpg] Jarek Potiuk Polidea<https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129<tel:+48660796129> [Polidea]<https://www.polidea.com/> ============================================================================== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html ============================================================================== -- [https://s3.eu-central-1.amazonaws.com/corgi-mail/23-05-2019/jarek.potiuk/jarpot.jpg] Jarek Potiuk Polidea<https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129<tel:+48660796129> [Polidea]<https://www.polidea.com/> -- Leah Cole (she/her) | Developer Programs Engineer | colel...@google.com<mailto:colel...@google.com> | (925) 257-2112 I'm working weird hours during this pandemic and am sometimes a bit slower to respond to PRs/CLs than normal. Please feel free to send me a gentle ping for a status update if my slowness is blocking you and I'll do my best to give you an ETA. -- [https://s3.eu-central-1.amazonaws.com/corgi-mail/23-05-2019/jarek.potiuk/jarpot.jpg] Jarek Potiuk Polidea<https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129<tel:+48660796129> [Polidea]<https://www.polidea.com/> =============================================================================== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html ===============================================================================
