Re: Plan for dropping python 2 support
I'd like to help with this Python 2 -> 3 migration if possible. We're nearly halfway through 2020 now -- is there currently anything stopping us from doing this migration at the moment? Is this the right time to do so? On 2019/10/05 02:11:24, Valentyn Tymofieiev wrote: > On Fri, Oct 4, 2019 at 11:02 AM Robert Bradshaw wrote: > > > Thanks for holding this vote. Note that this is a pledge to remove > > support sometime in 2020, but no promises as to whether that will be > > January or December (though I hope sooner rather than later) > > > Right. > > > > > Valentyn, did you want to go ahead and make a PR adding Apache Beam to > > the python3statement page? > > > > Yes, I sent > https://github.com/python3statement/python3statement.github.io/pull/265. > > > > > On Mon, Sep 30, 2019 at 5:10 PM Valentyn Tymofieiev > > wrote: > > > > > > As suggested and enthusiastically supported by several folks in this > > thread, I will send a vote to sign a pledge on http://python3statement.org > > on behalf of Apache Beam to discontinue Python 2 support in or before 2020. > > > > > > The motivation for signing the pledge is: > > > - to provide another signal to Beam users, and projects that depend on > > Beam that Beam Python 2 offering will soon sunset; > > > - to facilitate adoption of Python 3 by Beam users, developers, and > > runner maintainers; > > > - to facilitate adoption of Python 3 in wider Python ecosystem. > > > > > > See also http://python3stament.org for background behind this pledge > > and the list of projects which have already signed it. > > > > > > On Mon, Sep 23, 2019 at 4:45 PM Kyle Weaver wrote: > > >> > > >> Re feedback collection, we already print a message: > > >> "You are using Apache Beam with Python 2. New releases of Apache Beam > > will soon support Python 3 only." > > >> When users run Python 2 pipelines. This might be a good place to > > provide additional info along with a place to send feedback (probably > > user@). > > While I'm sure not everyone out there reads their logs, I imagine this is a > > sure and easy way of reaching at least some Python 2 users. > > >> > > >> Kyle Weaver | Software Engineer | github.com/ibzib | > > kcwea...@google.com > > >> > > >> > > >> On Fri, Sep 20, 2019 at 10:28 AM Valentyn Tymofieiev < > > valen...@google.com> wrote: > > >>> > > >>> Thank you, Chad, for refreshing this conversation and adding the > > perspective of Python 2 users of Beam who have not(yet) completed the > > migration. My thoughts below. > > >>> > > >>> - It is in the best interest of everyone to ensure a smooth migration > > for Beam users. However a migration needs to happen since Python ecosystem > > is moving off of Python 2. > > >>> - Beam has a couple of dozen dependencies, and we cannot have an > > expectation that Python 2 versions of these dependencies will be maintained > > in 2020. > > >>> - BEAM-1251 should be closed, since it may communicate a signal that > > Beam does not support Python 3, while it does. Beam has first announced > > support of Python 3 in Beam 2.11.0, admittedly later than many mainstream > > libraries in Python ecosystem. > > >>> - I think Python 2 LTS release (if we continue them) may have critical > > bug fixes, but not new features, so we won't be backporting new features. > > >>> - Beam portability allows users to customize usercode runtime > > environment, and it should be possible for users to supply a Python 2 SDK > > harness container, should they have no other option. This would require a > > backported user-supplied version of Beam SDK that works on Python 2, > > although such SDK may become difficult/impractical to maintain for most > > users. > > >>> - There are several open issues related to Python 3, but they are > > improvements in nature, and we are steadily closing them off. I am not > > aware of any adoption blockers for Beam Python 3, specific to Beam. > > >>> - I have not heard of users reports who attempted but were not able to > > use Beam on Python 3. > > >>> - This does not mean that our offering is perfect, there may be errors > > and omissions that are yet to be discovered. However, it would be in the > > best interest of the Beam community to discover these issues earlier. A > > message that Beam will discontinue Python 2 support will encourage users to > > migrate, therefore I also support Beam signing > > https://python3statement.org. > > >>> - Having more usage statistics and feedback closer to 2020 can help us > > be more confident in deciding when to stop Python 2 support. > > >>> > > >>> On Thu, Sep 19, 2019 at 6:05 PM Ahmet Altay wrote: > > > > Thanks a lot for sharing your thoughts, I completely agree that we > > need to minimize the burden on our users as much as possible. Especially in > > this case when we are offering a robust python 3 solution just now. However > > I do share the same concerns related to dependencies and tool chains, It > > will be increasingly difficult for us to keep our
Re: Plan for dropping python 2 support
On Fri, Oct 4, 2019 at 11:02 AM Robert Bradshaw wrote: > Thanks for holding this vote. Note that this is a pledge to remove > support sometime in 2020, but no promises as to whether that will be > January or December (though I hope sooner rather than later) Right. > Valentyn, did you want to go ahead and make a PR adding Apache Beam to > the python3statement page? > Yes, I sent https://github.com/python3statement/python3statement.github.io/pull/265. > > On Mon, Sep 30, 2019 at 5:10 PM Valentyn Tymofieiev > wrote: > > > > As suggested and enthusiastically supported by several folks in this > thread, I will send a vote to sign a pledge on http://python3statement.org > on behalf of Apache Beam to discontinue Python 2 support in or before 2020. > > > > The motivation for signing the pledge is: > > - to provide another signal to Beam users, and projects that depend on > Beam that Beam Python 2 offering will soon sunset; > > - to facilitate adoption of Python 3 by Beam users, developers, and > runner maintainers; > > - to facilitate adoption of Python 3 in wider Python ecosystem. > > > > See also http://python3stament.org for background behind this pledge > and the list of projects which have already signed it. > > > > On Mon, Sep 23, 2019 at 4:45 PM Kyle Weaver wrote: > >> > >> Re feedback collection, we already print a message: > >> "You are using Apache Beam with Python 2. New releases of Apache Beam > will soon support Python 3 only." > >> When users run Python 2 pipelines. This might be a good place to > provide additional info along with a place to send feedback (probably user@). > While I'm sure not everyone out there reads their logs, I imagine this is a > sure and easy way of reaching at least some Python 2 users. > >> > >> Kyle Weaver | Software Engineer | github.com/ibzib | > kcwea...@google.com > >> > >> > >> On Fri, Sep 20, 2019 at 10:28 AM Valentyn Tymofieiev < > valen...@google.com> wrote: > >>> > >>> Thank you, Chad, for refreshing this conversation and adding the > perspective of Python 2 users of Beam who have not(yet) completed the > migration. My thoughts below. > >>> > >>> - It is in the best interest of everyone to ensure a smooth migration > for Beam users. However a migration needs to happen since Python ecosystem > is moving off of Python 2. > >>> - Beam has a couple of dozen dependencies, and we cannot have an > expectation that Python 2 versions of these dependencies will be maintained > in 2020. > >>> - BEAM-1251 should be closed, since it may communicate a signal that > Beam does not support Python 3, while it does. Beam has first announced > support of Python 3 in Beam 2.11.0, admittedly later than many mainstream > libraries in Python ecosystem. > >>> - I think Python 2 LTS release (if we continue them) may have critical > bug fixes, but not new features, so we won't be backporting new features. > >>> - Beam portability allows users to customize usercode runtime > environment, and it should be possible for users to supply a Python 2 SDK > harness container, should they have no other option. This would require a > backported user-supplied version of Beam SDK that works on Python 2, > although such SDK may become difficult/impractical to maintain for most > users. > >>> - There are several open issues related to Python 3, but they are > improvements in nature, and we are steadily closing them off. I am not > aware of any adoption blockers for Beam Python 3, specific to Beam. > >>> - I have not heard of users reports who attempted but were not able to > use Beam on Python 3. > >>> - This does not mean that our offering is perfect, there may be errors > and omissions that are yet to be discovered. However, it would be in the > best interest of the Beam community to discover these issues earlier. A > message that Beam will discontinue Python 2 support will encourage users to > migrate, therefore I also support Beam signing > https://python3statement.org. > >>> - Having more usage statistics and feedback closer to 2020 can help us > be more confident in deciding when to stop Python 2 support. > >>> > >>> On Thu, Sep 19, 2019 at 6:05 PM Ahmet Altay wrote: > > Thanks a lot for sharing your thoughts, I completely agree that we > need to minimize the burden on our users as much as possible. Especially in > this case when we are offering a robust python 3 solution just now. However > I do share the same concerns related to dependencies and tool chains, It > will be increasingly difficult for us to keep our code base compatible with > python2 and python3 overtime. (To be very explicit, one of those > dependencies is Dataflow's python pre-portability workers.) > > On Thu, Sep 19, 2019 at 5:17 PM Maximilian Michels > wrote: > > > > Granted that we just have finalized the Python 3 support, we should > > allow time for it to mature and for users to make the switch. > > > > > Oh, and one more thing, I think it'd make sense for Apache Beam to >
Re: Plan for dropping python 2 support
Thanks for holding this vote. Note that this is a pledge to remove support sometime in 2020, but no promises as to whether that will be January or December (though I hope sooner rather than later). Valentyn, did you want to go ahead and make a PR adding Apache Beam to the python3statement page? On Mon, Sep 30, 2019 at 5:10 PM Valentyn Tymofieiev wrote: > > As suggested and enthusiastically supported by several folks in this thread, > I will send a vote to sign a pledge on http://python3statement.org on behalf > of Apache Beam to discontinue Python 2 support in or before 2020. > > The motivation for signing the pledge is: > - to provide another signal to Beam users, and projects that depend on Beam > that Beam Python 2 offering will soon sunset; > - to facilitate adoption of Python 3 by Beam users, developers, and runner > maintainers; > - to facilitate adoption of Python 3 in wider Python ecosystem. > > See also http://python3stament.org for background behind this pledge and the > list of projects which have already signed it. > > On Mon, Sep 23, 2019 at 4:45 PM Kyle Weaver wrote: >> >> Re feedback collection, we already print a message: >> "You are using Apache Beam with Python 2. New releases of Apache Beam will >> soon support Python 3 only." >> When users run Python 2 pipelines. This might be a good place to provide >> additional info along with a place to send feedback (probably user@). While >> I'm sure not everyone out there reads their logs, I imagine this is a sure >> and easy way of reaching at least some Python 2 users. >> >> Kyle Weaver | Software Engineer | github.com/ibzib | kcwea...@google.com >> >> >> On Fri, Sep 20, 2019 at 10:28 AM Valentyn Tymofieiev >> wrote: >>> >>> Thank you, Chad, for refreshing this conversation and adding the >>> perspective of Python 2 users of Beam who have not(yet) completed the >>> migration. My thoughts below. >>> >>> - It is in the best interest of everyone to ensure a smooth migration for >>> Beam users. However a migration needs to happen since Python ecosystem is >>> moving off of Python 2. >>> - Beam has a couple of dozen dependencies, and we cannot have an >>> expectation that Python 2 versions of these dependencies will be maintained >>> in 2020. >>> - BEAM-1251 should be closed, since it may communicate a signal that Beam >>> does not support Python 3, while it does. Beam has first announced support >>> of Python 3 in Beam 2.11.0, admittedly later than many mainstream libraries >>> in Python ecosystem. >>> - I think Python 2 LTS release (if we continue them) may have critical bug >>> fixes, but not new features, so we won't be backporting new features. >>> - Beam portability allows users to customize usercode runtime environment, >>> and it should be possible for users to supply a Python 2 SDK harness >>> container, should they have no other option. This would require a >>> backported user-supplied version of Beam SDK that works on Python 2, >>> although such SDK may become difficult/impractical to maintain for most >>> users. >>> - There are several open issues related to Python 3, but they are >>> improvements in nature, and we are steadily closing them off. I am not >>> aware of any adoption blockers for Beam Python 3, specific to Beam. >>> - I have not heard of users reports who attempted but were not able to use >>> Beam on Python 3. >>> - This does not mean that our offering is perfect, there may be errors and >>> omissions that are yet to be discovered. However, it would be in the best >>> interest of the Beam community to discover these issues earlier. A message >>> that Beam will discontinue Python 2 support will encourage users to >>> migrate, therefore I also support Beam signing https://python3statement.org. >>> - Having more usage statistics and feedback closer to 2020 can help us be >>> more confident in deciding when to stop Python 2 support. >>> >>> On Thu, Sep 19, 2019 at 6:05 PM Ahmet Altay wrote: Thanks a lot for sharing your thoughts, I completely agree that we need to minimize the burden on our users as much as possible. Especially in this case when we are offering a robust python 3 solution just now. However I do share the same concerns related to dependencies and tool chains, It will be increasingly difficult for us to keep our code base compatible with python2 and python3 overtime. (To be very explicit, one of those dependencies is Dataflow's python pre-portability workers.) On Thu, Sep 19, 2019 at 5:17 PM Maximilian Michels wrote: > > Granted that we just have finalized the Python 3 support, we should > allow time for it to mature and for users to make the switch. > > > Oh, and one more thing, I think it'd make sense for Apache Beam to > > sign https://python3statement.org/. The promise is that we'd > > discontinue Python 2 support *in* 2020, which is not committing us to > > January if we're not
Re: Plan for dropping python 2 support
As suggested and enthusiastically supported by several folks in this thread, I will send a vote to sign a pledge on http://python3statement.org on behalf of Apache Beam to discontinue Python 2 support in or before 2020. The motivation for signing the pledge is: - to provide another signal to Beam users, and projects that depend on Beam that Beam Python 2 offering will soon sunset; - to facilitate adoption of Python 3 by Beam users, developers, and runner maintainers; - to facilitate adoption of Python 3 in wider Python ecosystem. See also http://python3stament.org for background behind this pledge and the list of projects which have already signed it. On Mon, Sep 23, 2019 at 4:45 PM Kyle Weaver wrote: > Re feedback collection, we already print a message: > "You are using Apache Beam with Python 2. New releases of Apache Beam will > soon support Python 3 only." > When users run Python 2 pipelines. This might be a good place to provide > additional info along with a place to send feedback (probably user@). > While I'm sure not everyone out there reads their logs, I imagine this is a > sure and easy way of reaching at least some Python 2 users. > > Kyle Weaver | Software Engineer | github.com/ibzib | kcwea...@google.com > > > On Fri, Sep 20, 2019 at 10:28 AM Valentyn Tymofieiev > wrote: > >> Thank you, Chad, for refreshing this conversation and adding the >> perspective of Python 2 users of Beam who have not(yet) completed the >> migration. My thoughts below. >> >> - It is in the best interest of everyone to ensure a smooth migration for >> Beam users. However a migration needs to happen since Python ecosystem is >> moving off of Python 2. >> - Beam has a couple of dozen dependencies, and we cannot have an >> expectation that Python 2 versions of these dependencies will be maintained >> in 2020. >> - BEAM-1251 should be closed, since it may communicate a signal that Beam >> does not support Python 3, while it does. Beam has first announced support >> of Python 3 in Beam 2.11.0, admittedly later than many mainstream libraries >> in Python ecosystem. >> - I think Python 2 LTS release (if we continue them) may have critical >> bug fixes, but not new features, so we won't be backporting new features. >> - Beam portability allows users to customize usercode runtime >> environment, and it should be possible for users to supply a Python 2 SDK >> harness container, should they have no other option. This would require a >> backported user-supplied version of Beam SDK that works on Python 2, >> although such SDK may become difficult/impractical to maintain for most >> users. >> - There are several open issues related to Python 3, but they are >> improvements in nature, and we are steadily closing them off. I am not >> aware of any adoption blockers for Beam Python 3, specific to Beam. >> - I have not heard of users reports who attempted but were not able to >> use Beam on Python 3. >> - This does not mean that our offering is perfect, there may be errors >> and omissions that are yet to be discovered. However, it would be in the >> best interest of the Beam community to discover these issues earlier. A >> message that Beam will discontinue Python 2 support will encourage users to >> migrate, therefore I also support Beam signing >> https://python3statement.org. >> - Having more usage statistics and feedback closer to 2020 can help us be >> more confident in deciding when to stop Python 2 support. >> >> On Thu, Sep 19, 2019 at 6:05 PM Ahmet Altay wrote: >> >>> Thanks a lot for sharing your thoughts, I completely agree that we need >>> to minimize the burden on our users as much as possible. Especially in this >>> case when we are offering a robust python 3 solution just now. However I do >>> share the same concerns related to dependencies and tool chains, It will be >>> increasingly difficult for us to keep our code base compatible with python2 >>> and python3 overtime. (To be very explicit, one of those dependencies is >>> Dataflow's python pre-portability workers.) >>> >>> On Thu, Sep 19, 2019 at 5:17 PM Maximilian Michels >>> wrote: >>> Granted that we just have finalized the Python 3 support, we should allow time for it to mature and for users to make the switch. > Oh, and one more thing, I think it'd make sense for Apache Beam to > sign https://python3statement.org/. The promise is that we'd > discontinue Python 2 support *in* 2020, which is not committing us to > January if we're not ready. Worth a vote? +1 >>> >>> +1 >>> >>> On 19.09.19 15:59, Robert Bradshaw wrote: > Oh, and one more thing, I think it'd make sense for Apache Beam to > sign https://python3statement.org/. The promise is that we'd > discontinue Python 2 support *in* 2020, which is not committing us to > January if we're not ready. Worth a vote? > > > On Thu, Sep 19, 2019 at 3:58 PM Robert Bradshaw wrote: >> >> Exactly how long we
Re: Plan for dropping python 2 support
Re feedback collection, we already print a message: "You are using Apache Beam with Python 2. New releases of Apache Beam will soon support Python 3 only." When users run Python 2 pipelines. This might be a good place to provide additional info along with a place to send feedback (probably user@). While I'm sure not everyone out there reads their logs, I imagine this is a sure and easy way of reaching at least some Python 2 users. Kyle Weaver | Software Engineer | github.com/ibzib | kcwea...@google.com On Fri, Sep 20, 2019 at 10:28 AM Valentyn Tymofieiev wrote: > Thank you, Chad, for refreshing this conversation and adding the > perspective of Python 2 users of Beam who have not(yet) completed the > migration. My thoughts below. > > - It is in the best interest of everyone to ensure a smooth migration for > Beam users. However a migration needs to happen since Python ecosystem is > moving off of Python 2. > - Beam has a couple of dozen dependencies, and we cannot have an > expectation that Python 2 versions of these dependencies will be maintained > in 2020. > - BEAM-1251 should be closed, since it may communicate a signal that Beam > does not support Python 3, while it does. Beam has first announced support > of Python 3 in Beam 2.11.0, admittedly later than many mainstream libraries > in Python ecosystem. > - I think Python 2 LTS release (if we continue them) may have critical bug > fixes, but not new features, so we won't be backporting new features. > - Beam portability allows users to customize usercode runtime environment, > and it should be possible for users to supply a Python 2 SDK harness > container, should they have no other option. This would require a > backported user-supplied version of Beam SDK that works on Python 2, > although such SDK may become difficult/impractical to maintain for most > users. > - There are several open issues related to Python 3, but they are > improvements in nature, and we are steadily closing them off. I am not > aware of any adoption blockers for Beam Python 3, specific to Beam. > - I have not heard of users reports who attempted but were not able to use > Beam on Python 3. > - This does not mean that our offering is perfect, there may be errors and > omissions that are yet to be discovered. However, it would be in the best > interest of the Beam community to discover these issues earlier. A message > that Beam will discontinue Python 2 support will encourage users to > migrate, therefore I also support Beam signing > https://python3statement.org. > - Having more usage statistics and feedback closer to 2020 can help us be > more confident in deciding when to stop Python 2 support. > > On Thu, Sep 19, 2019 at 6:05 PM Ahmet Altay wrote: > >> Thanks a lot for sharing your thoughts, I completely agree that we need >> to minimize the burden on our users as much as possible. Especially in this >> case when we are offering a robust python 3 solution just now. However I do >> share the same concerns related to dependencies and tool chains, It will be >> increasingly difficult for us to keep our code base compatible with python2 >> and python3 overtime. (To be very explicit, one of those dependencies is >> Dataflow's python pre-portability workers.) >> >> On Thu, Sep 19, 2019 at 5:17 PM Maximilian Michels >> wrote: >> >>> Granted that we just have finalized the Python 3 support, we should >>> allow time for it to mature and for users to make the switch. >>> >>> > Oh, and one more thing, I think it'd make sense for Apache Beam to >>> > sign https://python3statement.org/. The promise is that we'd >>> > discontinue Python 2 support *in* 2020, which is not committing us to >>> > January if we're not ready. Worth a vote? >>> >>> +1 >>> >> >> +1 >> >> >>> >>> On 19.09.19 15:59, Robert Bradshaw wrote: >>> > Oh, and one more thing, I think it'd make sense for Apache Beam to >>> > sign https://python3statement.org/. The promise is that we'd >>> > discontinue Python 2 support *in* 2020, which is not committing us to >>> > January if we're not ready. Worth a vote? >>> > >>> > >>> > On Thu, Sep 19, 2019 at 3:58 PM Robert Bradshaw >>> wrote: >>> >> >>> >> Exactly how long we support Python 2 depends on our users. Other than >>> >> those that speak up (such as yourself, thanks!), it's hard to get a >>> >> handle on how many need Python 2 and for how long. (Should we send out >>> >> a survey? Maybe after some experience with 2.16?) >>> >> >> +1, we had some success with collecting information from users using >> Twitter surveys. >> >> >>> >> >>> >> On the one hand, the whole ecosystem is finally moving on, and even if >>> >> Beam continues to support Python 2 our dependencies, or other projects >>> >> that are being used in conjunction with Beam, will also be going >>> >> Python 3 only. On the other hand, Beam is, admittedly, quite late to >>> >> the party and could be the one holding people back, and looking at how >>> >> long it took us, if we just barely make it by the end
Re: Plan for dropping python 2 support
Thanks a lot for sharing your thoughts, I completely agree that we need to minimize the burden on our users as much as possible. Especially in this case when we are offering a robust python 3 solution just now. However I do share the same concerns related to dependencies and tool chains, It will be increasingly difficult for us to keep our code base compatible with python2 and python3 overtime. (To be very explicit, one of those dependencies is Dataflow's python pre-portability workers.) On Thu, Sep 19, 2019 at 5:17 PM Maximilian Michels wrote: > Granted that we just have finalized the Python 3 support, we should > allow time for it to mature and for users to make the switch. > > > Oh, and one more thing, I think it'd make sense for Apache Beam to > > sign https://python3statement.org/. The promise is that we'd > > discontinue Python 2 support *in* 2020, which is not committing us to > > January if we're not ready. Worth a vote? > > +1 > +1 > > On 19.09.19 15:59, Robert Bradshaw wrote: > > Oh, and one more thing, I think it'd make sense for Apache Beam to > > sign https://python3statement.org/. The promise is that we'd > > discontinue Python 2 support *in* 2020, which is not committing us to > > January if we're not ready. Worth a vote? > > > > > > On Thu, Sep 19, 2019 at 3:58 PM Robert Bradshaw > wrote: > >> > >> Exactly how long we support Python 2 depends on our users. Other than > >> those that speak up (such as yourself, thanks!), it's hard to get a > >> handle on how many need Python 2 and for how long. (Should we send out > >> a survey? Maybe after some experience with 2.16?) > +1, we had some success with collecting information from users using Twitter surveys. > >> > >> On the one hand, the whole ecosystem is finally moving on, and even if > >> Beam continues to support Python 2 our dependencies, or other projects > >> that are being used in conjunction with Beam, will also be going > >> Python 3 only. On the other hand, Beam is, admittedly, quite late to > >> the party and could be the one holding people back, and looking at how > >> long it took us, if we just barely make it by the end of the year it's > >> unreasonable to say at that point "oh, and we're dropping 2.7 at the > >> same time." > >> > >> The good news is that 2.16 is shaping up to be a release I would > >> recommend everyone migrate to Python 3 on. The remaining issues are > >> things like some issues with main sessions (which already has issues > >> in Python 2) and not supporting keyword-only arguments (a new feature, > >> not a regression). I would guess that even 2.15 is already good enough > >> for most people, at least to kick the tires and running tests to start > >> the effort. > I share the same sentiment. Beam 2.16 will offer a strong python 3 offering. Yes, there are known issues but this is not much different than the known issues for rest of the python offering. > >> > >> (I also agree with the sentiment that once we go 3.x only, it'll be > >> likely harder to maintain a 2.x LTS... but the whole LTS thing is > >> being discussed in another thread.) >> > >> On Thu, Sep 19, 2019 at 2:44 PM Chad Dombrova > wrote: > >>> > >>> Hi all, > >>> I had a read through this thread in the archives. It occurred before I > joined the mailing list, so I hope that this email connects up with the > thread properly for everyone. > >>> > >>> I'd like to respond to the following points: > >>> > I believe we are referring to two separate things with support: > - Supporting existing releases for patches - I agree that we need to > give > users a long enough window to upgrade. Great if it happens with an LTS > release. Even if it does not, I think it will be fair to offer > patches on > the last python 2 supporting release during some part of 2020 if that > becomes necessary. > - Making new releases with python 2 support - Each new Beam release > with > python 2 support will implicitly extend the lifetime of beam's python > 2 > support. I do not think we need to extend this to beyond 2019. 2 > releases > (~ 3 months) after solid python 3 support will very likely put the > last > python 2 supporting release to last quarter of 2019 already. > >>> > >>> > >>> With so many important features still under active development > (portability, expansion, external IO transforms, schema coders) and new > versions of executors tied to the Beam source, staying behind is not really > an option for many of us, and with python3 support not yet fully completed, > the window in which Beam is fully working for both python versions is > rapidly approaching 2 months, and could ultimately be even less, depending > on how long it takes to complete the dozen remaining issues in Jira, and > whatever pops up thereafter. > >>> > The cost of maintaining Python 2.7 support is higher than 0. Some > issues > that come to mind: > - Maintaining Py2.7 / Py 3+ compatibility of Beam codebase
Re: Plan for dropping python 2 support
Granted that we just have finalized the Python 3 support, we should allow time for it to mature and for users to make the switch. Oh, and one more thing, I think it'd make sense for Apache Beam to sign https://python3statement.org/. The promise is that we'd discontinue Python 2 support *in* 2020, which is not committing us to January if we're not ready. Worth a vote? +1 On 19.09.19 15:59, Robert Bradshaw wrote: Oh, and one more thing, I think it'd make sense for Apache Beam to sign https://python3statement.org/. The promise is that we'd discontinue Python 2 support *in* 2020, which is not committing us to January if we're not ready. Worth a vote? On Thu, Sep 19, 2019 at 3:58 PM Robert Bradshaw wrote: Exactly how long we support Python 2 depends on our users. Other than those that speak up (such as yourself, thanks!), it's hard to get a handle on how many need Python 2 and for how long. (Should we send out a survey? Maybe after some experience with 2.16?) On the one hand, the whole ecosystem is finally moving on, and even if Beam continues to support Python 2 our dependencies, or other projects that are being used in conjunction with Beam, will also be going Python 3 only. On the other hand, Beam is, admittedly, quite late to the party and could be the one holding people back, and looking at how long it took us, if we just barely make it by the end of the year it's unreasonable to say at that point "oh, and we're dropping 2.7 at the same time." The good news is that 2.16 is shaping up to be a release I would recommend everyone migrate to Python 3 on. The remaining issues are things like some issues with main sessions (which already has issues in Python 2) and not supporting keyword-only arguments (a new feature, not a regression). I would guess that even 2.15 is already good enough for most people, at least to kick the tires and running tests to start the effort. (I also agree with the sentiment that once we go 3.x only, it'll be likely harder to maintain a 2.x LTS... but the whole LTS thing is being discussed in another thread.) On Thu, Sep 19, 2019 at 2:44 PM Chad Dombrova wrote: Hi all, I had a read through this thread in the archives. It occurred before I joined the mailing list, so I hope that this email connects up with the thread properly for everyone. I'd like to respond to the following points: I believe we are referring to two separate things with support: - Supporting existing releases for patches - I agree that we need to give users a long enough window to upgrade. Great if it happens with an LTS release. Even if it does not, I think it will be fair to offer patches on the last python 2 supporting release during some part of 2020 if that becomes necessary. - Making new releases with python 2 support - Each new Beam release with python 2 support will implicitly extend the lifetime of beam's python 2 support. I do not think we need to extend this to beyond 2019. 2 releases (~ 3 months) after solid python 3 support will very likely put the last python 2 supporting release to last quarter of 2019 already. With so many important features still under active development (portability, expansion, external IO transforms, schema coders) and new versions of executors tied to the Beam source, staying behind is not really an option for many of us, and with python3 support not yet fully completed, the window in which Beam is fully working for both python versions is rapidly approaching 2 months, and could ultimately be even less, depending on how long it takes to complete the dozen remaining issues in Jira, and whatever pops up thereafter. The cost of maintaining Python 2.7 support is higher than 0. Some issues that come to mind: - Maintaining Py2.7 / Py 3+ compatibility of Beam codebase makes it difficult to use Python 3 syntax in Beam which may be necessary to support and test syntactic constructs introduced in Python 3. - Running additional test suites increases the load on test infrastructure and increases flakiness. I would argue that the cost of maintaining a python2-only LTS version will be far greater than maintaining python2 support for a little while longer. Dropping support for python2 could mean a number of things from simply disabling the python2 tests, to removing 2-to-3 idioms in favor of python3-only constructs. If what you have in mind is anything like the latter then the master branch will become quite divergent from the LTS release, and backporting changes will be not be as simple as cherry-picking commits. All-in-all, I think it's a lose/lose for everyone -- users and developers, of which I am both -- to drop python2 support on such a short timeline. I'm an active contributor to this project and it will put me and the company that I work for in a very bad position if you force us onto an LTS release in early 2020. I understand the appeal of moving to python3-only code and I want to get there too, but I would hope that you give your
Re: Plan for dropping python 2 support
Oh, and one more thing, I think it'd make sense for Apache Beam to sign https://python3statement.org/. The promise is that we'd discontinue Python 2 support *in* 2020, which is not committing us to January if we're not ready. Worth a vote? On Thu, Sep 19, 2019 at 3:58 PM Robert Bradshaw wrote: > > Exactly how long we support Python 2 depends on our users. Other than > those that speak up (such as yourself, thanks!), it's hard to get a > handle on how many need Python 2 and for how long. (Should we send out > a survey? Maybe after some experience with 2.16?) > > On the one hand, the whole ecosystem is finally moving on, and even if > Beam continues to support Python 2 our dependencies, or other projects > that are being used in conjunction with Beam, will also be going > Python 3 only. On the other hand, Beam is, admittedly, quite late to > the party and could be the one holding people back, and looking at how > long it took us, if we just barely make it by the end of the year it's > unreasonable to say at that point "oh, and we're dropping 2.7 at the > same time." > > The good news is that 2.16 is shaping up to be a release I would > recommend everyone migrate to Python 3 on. The remaining issues are > things like some issues with main sessions (which already has issues > in Python 2) and not supporting keyword-only arguments (a new feature, > not a regression). I would guess that even 2.15 is already good enough > for most people, at least to kick the tires and running tests to start > the effort. > > (I also agree with the sentiment that once we go 3.x only, it'll be > likely harder to maintain a 2.x LTS... but the whole LTS thing is > being discussed in another thread.) > > On Thu, Sep 19, 2019 at 2:44 PM Chad Dombrova wrote: > > > > Hi all, > > I had a read through this thread in the archives. It occurred before I > > joined the mailing list, so I hope that this email connects up with the > > thread properly for everyone. > > > > I'd like to respond to the following points: > > > >> I believe we are referring to two separate things with support: > >> - Supporting existing releases for patches - I agree that we need to give > >> users a long enough window to upgrade. Great if it happens with an LTS > >> release. Even if it does not, I think it will be fair to offer patches on > >> the last python 2 supporting release during some part of 2020 if that > >> becomes necessary. > >> - Making new releases with python 2 support - Each new Beam release with > >> python 2 support will implicitly extend the lifetime of beam's python 2 > >> support. I do not think we need to extend this to beyond 2019. 2 releases > >> (~ 3 months) after solid python 3 support will very likely put the last > >> python 2 supporting release to last quarter of 2019 already. > > > > > > With so many important features still under active development > > (portability, expansion, external IO transforms, schema coders) and new > > versions of executors tied to the Beam source, staying behind is not really > > an option for many of us, and with python3 support not yet fully completed, > > the window in which Beam is fully working for both python versions is > > rapidly approaching 2 months, and could ultimately be even less, depending > > on how long it takes to complete the dozen remaining issues in Jira, and > > whatever pops up thereafter. > > > >> The cost of maintaining Python 2.7 support is higher than 0. Some issues > >> that come to mind: > >> - Maintaining Py2.7 / Py 3+ compatibility of Beam codebase makes it > >> difficult to use Python 3 syntax in Beam which may be necessary to support > >> and test syntactic constructs introduced in Python 3. > >> - Running additional test suites increases the load on test infrastructure > >> and increases flakiness. > > > > > > I would argue that the cost of maintaining a python2-only LTS version will > > be far greater than maintaining python2 support for a little while longer. > > Dropping support for python2 could mean a number of things from simply > > disabling the python2 tests, to removing 2-to-3 idioms in favor of > > python3-only constructs. If what you have in mind is anything like the > > latter then the master branch will become quite divergent from the LTS > > release, and backporting changes will be not be as simple as cherry-picking > > commits. All-in-all, I think it's a lose/lose for everyone -- users and > > developers, of which I am both -- to drop python2 support on such a short > > timeline. > > > > I'm an active contributor to this project and it will put me and the > > company that I work for in a very bad position if you force us onto an LTS > > release in early 2020. I understand the appeal of moving to python3-only > > code and I want to get there too, but I would hope that you give your users > > are much time to transition their own code as the Beam project itself has > > taken. I'm not asking for a full 12 months to transition, but
Re: Plan for dropping python 2 support
Exactly how long we support Python 2 depends on our users. Other than those that speak up (such as yourself, thanks!), it's hard to get a handle on how many need Python 2 and for how long. (Should we send out a survey? Maybe after some experience with 2.16?) On the one hand, the whole ecosystem is finally moving on, and even if Beam continues to support Python 2 our dependencies, or other projects that are being used in conjunction with Beam, will also be going Python 3 only. On the other hand, Beam is, admittedly, quite late to the party and could be the one holding people back, and looking at how long it took us, if we just barely make it by the end of the year it's unreasonable to say at that point "oh, and we're dropping 2.7 at the same time." The good news is that 2.16 is shaping up to be a release I would recommend everyone migrate to Python 3 on. The remaining issues are things like some issues with main sessions (which already has issues in Python 2) and not supporting keyword-only arguments (a new feature, not a regression). I would guess that even 2.15 is already good enough for most people, at least to kick the tires and running tests to start the effort. (I also agree with the sentiment that once we go 3.x only, it'll be likely harder to maintain a 2.x LTS... but the whole LTS thing is being discussed in another thread.) On Thu, Sep 19, 2019 at 2:44 PM Chad Dombrova wrote: > > Hi all, > I had a read through this thread in the archives. It occurred before I joined > the mailing list, so I hope that this email connects up with the thread > properly for everyone. > > I'd like to respond to the following points: > >> I believe we are referring to two separate things with support: >> - Supporting existing releases for patches - I agree that we need to give >> users a long enough window to upgrade. Great if it happens with an LTS >> release. Even if it does not, I think it will be fair to offer patches on >> the last python 2 supporting release during some part of 2020 if that >> becomes necessary. >> - Making new releases with python 2 support - Each new Beam release with >> python 2 support will implicitly extend the lifetime of beam's python 2 >> support. I do not think we need to extend this to beyond 2019. 2 releases >> (~ 3 months) after solid python 3 support will very likely put the last >> python 2 supporting release to last quarter of 2019 already. > > > With so many important features still under active development (portability, > expansion, external IO transforms, schema coders) and new versions of > executors tied to the Beam source, staying behind is not really an option for > many of us, and with python3 support not yet fully completed, the window in > which Beam is fully working for both python versions is rapidly approaching 2 > months, and could ultimately be even less, depending on how long it takes to > complete the dozen remaining issues in Jira, and whatever pops up thereafter. > >> The cost of maintaining Python 2.7 support is higher than 0. Some issues >> that come to mind: >> - Maintaining Py2.7 / Py 3+ compatibility of Beam codebase makes it >> difficult to use Python 3 syntax in Beam which may be necessary to support >> and test syntactic constructs introduced in Python 3. >> - Running additional test suites increases the load on test infrastructure >> and increases flakiness. > > > I would argue that the cost of maintaining a python2-only LTS version will be > far greater than maintaining python2 support for a little while longer. > Dropping support for python2 could mean a number of things from simply > disabling the python2 tests, to removing 2-to-3 idioms in favor of > python3-only constructs. If what you have in mind is anything like the > latter then the master branch will become quite divergent from the LTS > release, and backporting changes will be not be as simple as cherry-picking > commits. All-in-all, I think it's a lose/lose for everyone -- users and > developers, of which I am both -- to drop python2 support on such a short > timeline. > > I'm an active contributor to this project and it will put me and the company > that I work for in a very bad position if you force us onto an LTS release in > early 2020. I understand the appeal of moving to python3-only code and I > want to get there too, but I would hope that you give your users are much > time to transition their own code as the Beam project itself has taken. I'm > not asking for a full 12 months to transition, but more than a couple will be > required. > > thanks, > -chad > > > >
Re: Plan for dropping python 2 support
Hi all, I had a read through this thread in the archives. It occurred before I joined the mailing list, so I hope that this email connects up with the thread properly for everyone. I'd like to respond to the following points: I believe we are referring to two separate things with support: > - Supporting existing releases for patches - I agree that we need to give > users a long enough window to upgrade. Great if it happens with an LTS > release. Even if it does not, I think it will be fair to offer patches on > the last python 2 supporting release during some part of 2020 if that > becomes necessary. > - Making new releases with python 2 support - Each new Beam release with > python 2 support will implicitly extend the lifetime of beam's python 2 > support. I do not think we need to extend this to beyond 2019. 2 releases > (~ 3 months) after solid python 3 support will very likely put the last > python 2 supporting release to last quarter of 2019 already. With so many important features still under active development (portability, expansion, external IO transforms, schema coders) and new versions of executors tied to the Beam source, staying behind is not really an option for many of us, and with python3 support not yet fully completed, the window in which Beam is fully working for both python versions is rapidly approaching 2 months, and could ultimately be even less, depending on how long it takes to complete the dozen remaining issues in Jira, and whatever pops up thereafter. The cost of maintaining Python 2.7 support is higher than 0. Some issues > that come to mind: > - Maintaining Py2.7 / Py 3+ compatibility of Beam codebase makes it > difficult to use Python 3 syntax in Beam which may be necessary to support > and test syntactic constructs introduced in Python 3. > - Running additional test suites increases the load on test infrastructure > and increases flakiness. I would argue that the cost of maintaining a python2-only LTS version will be far greater than maintaining python2 support for a little while longer. Dropping support for python2 could mean a number of things from simply disabling the python2 tests, to removing 2-to-3 idioms in favor of python3-only constructs. If what you have in mind is anything like the latter then the master branch will become quite divergent from the LTS release, and backporting changes will be not be as simple as cherry-picking commits. All-in-all, I think it's a lose/lose for everyone -- users and developers, of which I am both -- to drop python2 support on such a short timeline. I'm an active contributor to this project and it will put me and the company that I work for in a very bad position if you force us onto an LTS release in early 2020. I understand the appeal of moving to python3-only code and I want to get there too, but I would hope that you give your users are much time to transition their own code as the Beam project itself has taken. I'm not asking for a full 12 months to transition, but more than a couple will be required. thanks, -chad
Re: Plan for dropping python 2 support
On Sat, Jun 22, 2019 at 1:09 AM Valentyn Tymofieiev wrote: > > On Tue, Jun 18, 2019 at 2:01 PM Ahmet Altay wrote: >> >> Thank you for the update, very helpful. It might be worthwhile to share a >> version of this with user mailing list after 2.14. > > > I think so too, we can send an update to user list when 2.14.0 is released. > >> >> Remaining question for me is: There is no plan for an LTS release currently. >> Would it make sense for us to target one after known remaining issues are >> mostly fixed. What release would that be? > > > From Python 3 standpoint, 2.16.0 could be a good target for an LTS release, > that would give us two more releases in 2019 where we can mark Python 2 > support as deprecated and remove it with the first release in 2020. Our LTS policy (such as it is) is to designate them after they've proven themselves, not ahead of time. That being said, support for 3.x is a good criteria for a candidate to be declared as the LTS one. >> On Tue, Jun 18, 2019 at 12:08 AM Valentyn Tymofieiev >> wrote: >>> >>> To give a better understanding where we are w.r.t. Python 3, I'd like to >>> give a quick overview of the recent work that has been happening in Beam >>> community to support Python 3, and to summarize the current status of this >>> effort. >>> >>> Current status: >>> >>> Beam 2.11.0 was the first release that offered Python 3 support, >>> specifically Python 3.5 support. Due to several limitations that have been >>> fixed since 2.11.0, Beam 2.13.0 (or newer version) is recommended for >>> Python 3 pipelines. >>> >>> Pipelines running on Portable Flink / Spark runners may have to use Beam >>> 2.14.0 once it becomes available. >>> >>> Python 3.5 or newer version of the interpreter is required to install Beam >>> and run Python 3 pipelines. >>> >>> >>> Known remaining limitations of current Python 3 offering: >>> >>> Several syntactic constructs introduced in Python 3 (keyword-only >>> arguments, dataclasses), are not yet supported. See: BEAM-5878, BEAM-7284. >>> >>> Pickling errors occasionally prevent usage of --save_main_session flag, but >>> changes to the pipeline code may help to overcome this limitation. See: >>> BEAM-6158, BEAM-7540 >>> >>> Beam has limited type inference capabilities support in Python 3.6+, and >>> type checking of Beam typehints is not always enforced, see: BEAM-2713, >>> BEAM-7377. >>> >>> >>> The cause of limitations 1-2 largely lies in Beam dependency 'dill' that >>> supports pickling. In the immediate future we will be working on evaluating >>> replacements or/and fixes to address this. We are also working on an >>> improved typehints support in Python 3, see: BEAM-2713. >>> >>> >>> The efforts to make Beam codebase Python3-compatible started back in 2017. >>> Most of this work is visible in BEAM-1251[1] and in Kanban Board [2]. >>> >>> >>> 2017: >>> >>> BEAM-1251 is opened, and first efforts to make Beam codebase >>> Python3-compatible followed shortly. >>> >>> >>> Q3-Q4 2019: >>> >>> Active work on "futurizing" Beam codebase piece-by-piece while preventing >>> regressions in performance in existing Python 2 offering. >>> >>> Building test infrastructure to incorporate Python 3 test scenarios. >>> >>> >>> Apache Beam 2.11.0 (Q1 2019): >>> >>> "Futurization" of Beam Python codebase completed. >>> >>> Apache Beam 2.11.0 is released with Python 3 support, with limitations. >>> >>> Continuous pre-commit and post-commit test suites added for Python 3.5. >>> >>> Gaps in Python 3 support in Datastore IO, Avro IO, Bigquery IO identified >>> and scoped. >>> >>> Continuous testing mostly limited to Python 3.5. >>> >>> >>> Apache Beam 2.12.0 (Q2 2019): >>> >>> Pre and Post-commit test coverage expanded to Python 3.5, 3.6, 3.7. >>> >>> Direct and Dataflow runners added support for Python 3.6 - 3.7. >>> >>> >>> Apache Beam 2.13.0 (Q2 2019) >>> >>> Avro IO support enabled on Python 3. >>> >>> Datastore IO support enabled on Python 3. >>> >>> Bigquery IO support for BYTES datatype enabled on Python 3. >>> >>> >>> Apache Beam 2.14.0 (to be released in Q3 2019) >>> >>> Python 3 bug fixes for Bigquery IO and Portable Runner >>> >>> Every Python SDK commit exercises Direct, Dataflow, and Portable Flink >>> runners on Python 3 in various test suites. >>> >>> Beam 2.14.0 will declare Python 3.5, 3.6, 3.7 support in PyPi. >>> >>> >>> Next steps: >>> >>> Address known limitations and user feedback. >>> >>> Increase Python 3 test coverage in portable runner. >>> >>> Assist Beam users in Python 2 -> Python 3 migration. >>> >>> Deprecate of Python 2 support in Beam, cleanup the codebase. >>> >>> >>> I'd like to thank all Beam contributors who have been helping to push this >>> effort so far. >>> >>> >>> [1] https://issues.apache.org/jira/browse/BEAM-1251 >>> >>> [2] >>> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=245=detail >>> >>> >>> On Tue, Jun 18, 2019 at 12:03 AM Valentyn Tymofieiev >>> wrote: I like the
Re: Plan for dropping python 2 support
On Tue, Jun 18, 2019 at 2:01 PM Ahmet Altay wrote: > Thank you for the update, very helpful. It might be worthwhile to share a > version of this with user mailing list after 2.14. > I think so too, we can send an update to user list when 2.14.0 is released. > Remaining question for me is: There is no plan for an LTS release > currently. Would it make sense for us to target one after known remaining > issues are mostly fixed. What release would that be? > >From Python 3 standpoint, 2.16.0 could be a good target for an LTS release, that would give us two more releases in 2019 where we can mark Python 2 support as deprecated and remove it with the first release in 2020. > > On Tue, Jun 18, 2019 at 12:08 AM Valentyn Tymofieiev > wrote: > >> To give a better understanding where we are w.r.t. Python 3, I'd like to >> give a quick overview of the recent work that has been happening in Beam >> community to support Python 3, and to summarize the current status of this >> effort. >> >> Current status: >> >>1. >> >>Beam 2.11.0 was the first release that offered Python 3 support, >>specifically Python 3.5 support. Due to several limitations that have been >>fixed since 2.11.0, Beam 2.13.0 (or newer version) is recommended for >>Python 3 pipelines. >>2. >> >>Pipelines running on Portable Flink / Spark runners may have to use >>Beam 2.14.0 once it becomes available. >>3. >> >>Python 3.5 or newer version of the interpreter is required to install >>Beam and run Python 3 pipelines. >> >> >> Known remaining limitations of current Python 3 offering: >> >> >>1. >> >>Several syntactic constructs introduced in Python 3 (keyword-only >>arguments, dataclasses), are not yet supported. See: BEAM-5878, BEAM-7284. >>2. >> >>Pickling errors occasionally prevent usage of --save_main_session >>flag, but changes to the pipeline code may help to overcome this >>limitation. See: BEAM-6158, BEAM-7540 >>3. >> >>Beam has limited type inference capabilities support in Python 3.6+, >>and type checking of Beam typehints is not always enforced, see: >> BEAM-2713, >>BEAM-7377. >> >> >> The cause of limitations 1-2 largely lies in Beam dependency 'dill' that >> supports pickling. In the immediate future we will be working on evaluating >> replacements or/and fixes to address this. We are also working on an >> improved typehints support in Python 3, see: BEAM-2713. >> >> The efforts to make Beam codebase Python3-compatible started back in >> 2017. Most of this work is visible in BEAM-1251[1] and in Kanban Board [2]. >> >> >> 2017: >> >>- >> >>BEAM-1251 is opened, and first efforts to make Beam codebase >>Python3-compatible followed shortly. >> >> >> Q3-Q4 2019: >> >>- >> >>Active work on "futurizing" Beam codebase piece-by-piece while >>preventing regressions in performance in existing Python 2 offering. >>- >> >>Building test infrastructure to incorporate Python 3 test scenarios. >> >> >> Apache Beam 2.11.0 (Q1 2019): >> >>- >> >>"Futurization" of Beam Python codebase completed. >>- >> >>Apache Beam 2.11.0 is released with Python 3 support, with >>limitations. >>- >> >>Continuous pre-commit and post-commit test suites added for Python >>3.5. >>- >> >>Gaps in Python 3 support in Datastore IO, Avro IO, Bigquery IO >>identified and scoped. >>- >> >>Continuous testing mostly limited to Python 3.5. >> >> >> Apache Beam 2.12.0 (Q2 2019): >> >>- >> >>Pre and Post-commit test coverage expanded to Python 3.5, 3.6, 3.7. >>- >> >>Direct and Dataflow runners added support for Python 3.6 - 3.7. >> >> >> Apache Beam 2.13.0 (Q2 2019) >> >>- >> >>Avro IO support enabled on Python 3. >>- >> >>Datastore IO support enabled on Python 3. >>- >> >>Bigquery IO support for BYTES datatype enabled on Python 3. >> >> >> Apache Beam 2.14.0 (to be released in Q3 2019) >> >>- >> >>Python 3 bug fixes for Bigquery IO and Portable Runner >>- >> >>Every Python SDK commit exercises Direct, Dataflow, and Portable >>Flink runners on Python 3 in various test suites. >>- >> >>Beam 2.14.0 will declare Python 3.5, 3.6, 3.7 support in PyPi. >> >> >> Next steps: >> >>- >> >>Address known limitations and user feedback. >>- >> >>Increase Python 3 test coverage in portable runner. >>- >> >>Assist Beam users in Python 2 -> Python 3 migration. >>- >> >>Deprecate of Python 2 support in Beam, cleanup the codebase. >> >> >> I'd like to thank all Beam contributors who have been helping to push >> this effort so far. >> >> >> [1] https://issues.apache.org/jira/browse/BEAM-1251 >> >> [2] >> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=245=detail >> >> On Tue, Jun 18, 2019 at 12:03 AM Valentyn Tymofieiev >> wrote: >> >>> I like the update Ismaël referenced [1], I think we should prepare a >>> similar
Re: Plan for dropping python 2 support
Thank you for the update, very helpful. It might be worthwhile to share a version of this with user mailing list after 2.14. Remaining question for me is: There is no plan for an LTS release currently. Would it make sense for us to target one after known remaining issues are mostly fixed. What release would that be? On Tue, Jun 18, 2019 at 12:08 AM Valentyn Tymofieiev wrote: > To give a better understanding where we are w.r.t. Python 3, I'd like to > give a quick overview of the recent work that has been happening in Beam > community to support Python 3, and to summarize the current status of this > effort. > > Current status: > >1. > >Beam 2.11.0 was the first release that offered Python 3 support, >specifically Python 3.5 support. Due to several limitations that have been >fixed since 2.11.0, Beam 2.13.0 (or newer version) is recommended for >Python 3 pipelines. >2. > >Pipelines running on Portable Flink / Spark runners may have to use >Beam 2.14.0 once it becomes available. >3. > >Python 3.5 or newer version of the interpreter is required to install >Beam and run Python 3 pipelines. > > > Known remaining limitations of current Python 3 offering: > > >1. > >Several syntactic constructs introduced in Python 3 (keyword-only >arguments, dataclasses), are not yet supported. See: BEAM-5878, BEAM-7284. >2. > >Pickling errors occasionally prevent usage of --save_main_session >flag, but changes to the pipeline code may help to overcome this >limitation. See: BEAM-6158, BEAM-7540 >3. > >Beam has limited type inference capabilities support in Python 3.6+, >and type checking of Beam typehints is not always enforced, see: BEAM-2713, >BEAM-7377. > > > The cause of limitations 1-2 largely lies in Beam dependency 'dill' that > supports pickling. In the immediate future we will be working on evaluating > replacements or/and fixes to address this. We are also working on an > improved typehints support in Python 3, see: BEAM-2713. > > The efforts to make Beam codebase Python3-compatible started back in 2017. > Most of this work is visible in BEAM-1251[1] and in Kanban Board [2]. > > > 2017: > >- > >BEAM-1251 is opened, and first efforts to make Beam codebase >Python3-compatible followed shortly. > > > Q3-Q4 2019: > >- > >Active work on "futurizing" Beam codebase piece-by-piece while >preventing regressions in performance in existing Python 2 offering. >- > >Building test infrastructure to incorporate Python 3 test scenarios. > > > Apache Beam 2.11.0 (Q1 2019): > >- > >"Futurization" of Beam Python codebase completed. >- > >Apache Beam 2.11.0 is released with Python 3 support, with limitations. >- > >Continuous pre-commit and post-commit test suites added for Python >3.5. >- > >Gaps in Python 3 support in Datastore IO, Avro IO, Bigquery IO >identified and scoped. >- > >Continuous testing mostly limited to Python 3.5. > > > Apache Beam 2.12.0 (Q2 2019): > >- > >Pre and Post-commit test coverage expanded to Python 3.5, 3.6, 3.7. >- > >Direct and Dataflow runners added support for Python 3.6 - 3.7. > > > Apache Beam 2.13.0 (Q2 2019) > >- > >Avro IO support enabled on Python 3. >- > >Datastore IO support enabled on Python 3. >- > >Bigquery IO support for BYTES datatype enabled on Python 3. > > > Apache Beam 2.14.0 (to be released in Q3 2019) > >- > >Python 3 bug fixes for Bigquery IO and Portable Runner >- > >Every Python SDK commit exercises Direct, Dataflow, and Portable Flink >runners on Python 3 in various test suites. >- > >Beam 2.14.0 will declare Python 3.5, 3.6, 3.7 support in PyPi. > > > Next steps: > >- > >Address known limitations and user feedback. >- > >Increase Python 3 test coverage in portable runner. >- > >Assist Beam users in Python 2 -> Python 3 migration. >- > >Deprecate of Python 2 support in Beam, cleanup the codebase. > > > I'd like to thank all Beam contributors who have been helping to push this > effort so far. > > > [1] https://issues.apache.org/jira/browse/BEAM-1251 > > [2] > https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=245=detail > > On Tue, Jun 18, 2019 at 12:03 AM Valentyn Tymofieiev > wrote: > >> I like the update Ismaël referenced [1], I think we should prepare a >> similar update for Beam users. I would propose the following: >> - Designate last LTS release that we will have in 2019 to be the last LTS >> release with Python 2 support. >> - Add a Beam-specific deprecation warning on Python 2 starting from the >> last LTS release, or last 2 releases of Beam in 2019, whichever happens >> earlier. >> - Remove Python 2 support starting from the first release in 2020. >> >> The cost of maintaining Python 2.7 support is higher than 0. Some issues >> that come to mind: >> - Maintaining Py2.7 / Py 3+ compatibility
Re: Plan for dropping python 2 support
To give a better understanding where we are w.r.t. Python 3, I'd like to give a quick overview of the recent work that has been happening in Beam community to support Python 3, and to summarize the current status of this effort. Current status: 1. Beam 2.11.0 was the first release that offered Python 3 support, specifically Python 3.5 support. Due to several limitations that have been fixed since 2.11.0, Beam 2.13.0 (or newer version) is recommended for Python 3 pipelines. 2. Pipelines running on Portable Flink / Spark runners may have to use Beam 2.14.0 once it becomes available. 3. Python 3.5 or newer version of the interpreter is required to install Beam and run Python 3 pipelines. Known remaining limitations of current Python 3 offering: 1. Several syntactic constructs introduced in Python 3 (keyword-only arguments, dataclasses), are not yet supported. See: BEAM-5878, BEAM-7284. 2. Pickling errors occasionally prevent usage of --save_main_session flag, but changes to the pipeline code may help to overcome this limitation. See: BEAM-6158, BEAM-7540 3. Beam has limited type inference capabilities support in Python 3.6+, and type checking of Beam typehints is not always enforced, see: BEAM-2713, BEAM-7377. The cause of limitations 1-2 largely lies in Beam dependency 'dill' that supports pickling. In the immediate future we will be working on evaluating replacements or/and fixes to address this. We are also working on an improved typehints support in Python 3, see: BEAM-2713. The efforts to make Beam codebase Python3-compatible started back in 2017. Most of this work is visible in BEAM-1251[1] and in Kanban Board [2]. 2017: - BEAM-1251 is opened, and first efforts to make Beam codebase Python3-compatible followed shortly. Q3-Q4 2019: - Active work on "futurizing" Beam codebase piece-by-piece while preventing regressions in performance in existing Python 2 offering. - Building test infrastructure to incorporate Python 3 test scenarios. Apache Beam 2.11.0 (Q1 2019): - "Futurization" of Beam Python codebase completed. - Apache Beam 2.11.0 is released with Python 3 support, with limitations. - Continuous pre-commit and post-commit test suites added for Python 3.5. - Gaps in Python 3 support in Datastore IO, Avro IO, Bigquery IO identified and scoped. - Continuous testing mostly limited to Python 3.5. Apache Beam 2.12.0 (Q2 2019): - Pre and Post-commit test coverage expanded to Python 3.5, 3.6, 3.7. - Direct and Dataflow runners added support for Python 3.6 - 3.7. Apache Beam 2.13.0 (Q2 2019) - Avro IO support enabled on Python 3. - Datastore IO support enabled on Python 3. - Bigquery IO support for BYTES datatype enabled on Python 3. Apache Beam 2.14.0 (to be released in Q3 2019) - Python 3 bug fixes for Bigquery IO and Portable Runner - Every Python SDK commit exercises Direct, Dataflow, and Portable Flink runners on Python 3 in various test suites. - Beam 2.14.0 will declare Python 3.5, 3.6, 3.7 support in PyPi. Next steps: - Address known limitations and user feedback. - Increase Python 3 test coverage in portable runner. - Assist Beam users in Python 2 -> Python 3 migration. - Deprecate of Python 2 support in Beam, cleanup the codebase. I'd like to thank all Beam contributors who have been helping to push this effort so far. [1] https://issues.apache.org/jira/browse/BEAM-1251 [2] https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=245=detail On Tue, Jun 18, 2019 at 12:03 AM Valentyn Tymofieiev wrote: > I like the update Ismaël referenced [1], I think we should prepare a > similar update for Beam users. I would propose the following: > - Designate last LTS release that we will have in 2019 to be the last LTS > release with Python 2 support. > - Add a Beam-specific deprecation warning on Python 2 starting from the > last LTS release, or last 2 releases of Beam in 2019, whichever happens > earlier. > - Remove Python 2 support starting from the first release in 2020. > > The cost of maintaining Python 2.7 support is higher than 0. Some issues > that come to mind: > - Maintaining Py2.7 / Py 3+ compatibility of Beam codebase makes it > difficult to use Python 3 syntax in Beam which may be necessary to support > and test syntactic constructs introduced in Python 3. > - Running additional test suites increases the load on test infrastructure > and increases flakiness. > > [1] https://spark.apache.org/news/plan-for-dropping-python-2-support.html > > On Tue, Jun 11, 2019 at 7:57 AM Robert Bradshaw > wrote: > >> Sounds good. >> >> On Fri, Jun 7, 2019 at 8:28 PM Ahmet Altay wrote: >> >>> I agree with you. A more recent LTS release with python 2 support will >>> be good. Cost of maintaining python 2 support is also fairly low (maybe >>> zero
Re: Plan for dropping python 2 support
I like the update Ismaël referenced [1], I think we should prepare a similar update for Beam users. I would propose the following: - Designate last LTS release that we will have in 2019 to be the last LTS release with Python 2 support. - Add a Beam-specific deprecation warning on Python 2 starting from the last LTS release, or last 2 releases of Beam in 2019, whichever happens earlier. - Remove Python 2 support starting from the first release in 2020. The cost of maintaining Python 2.7 support is higher than 0. Some issues that come to mind: - Maintaining Py2.7 / Py 3+ compatibility of Beam codebase makes it difficult to use Python 3 syntax in Beam which may be necessary to support and test syntactic constructs introduced in Python 3. - Running additional test suites increases the load on test infrastructure and increases flakiness. [1] https://spark.apache.org/news/plan-for-dropping-python-2-support.html On Tue, Jun 11, 2019 at 7:57 AM Robert Bradshaw wrote: > Sounds good. > > On Fri, Jun 7, 2019 at 8:28 PM Ahmet Altay wrote: > >> I agree with you. A more recent LTS release with python 2 support will be >> good. Cost of maintaining python 2 support is also fairly low (maybe zero >> actually besides keeping some pre-existing compatibility code). >> >> I believe we are referring to two separate things with support: >> - Supporting existing releases for patches - I agree that we need to give >> users a long enough window to upgrade. Great if it happens with an LTS >> release. Even if it does not, I think it will be fair to offer patches on >> the last python 2 supporting release during some part of 2020 if that >> becomes necessary. >> - Making new releases with python 2 support - Each new Beam release with >> python 2 support will implicitly extend the lifetime of beam's python 2 >> support. I do not think we need to extend this to beyond 2019. 2 releases >> (~ 3 months) after solid python 3 support will very likely put the last >> python 2 supporting release to last quarter of 2019 already. >> >> On Fri, Jun 7, 2019 at 2:15 AM Robert Bradshaw >> wrote: >> >>> I don't think the second release with robust/recommended Python 3 >>> support should be the last release with Python 2 support--that is >>> simply not enough time for people to migrate. (Look at how long it >>> took us...) It does make a lot of sense to at least have one LTS >>> release with support for both. >>> >>> Regarding timeline, I think we could safely say we expect to support >>> Python 2 through 2019, likely for some of 2020 (possibly only via an >>> LTS release), and (very) unlikely beyond 2020. >>> >>> On Wed, Jun 5, 2019 at 6:34 PM Ahmet Altay wrote: >>> > >>> > I agree with the sentiment on this thread. Our priority needs to be >>> offering good python 3 support that we can comfortably recommend users to >>> switch. Progress on that so far has been promising and I do anticipate that >>> we will reach there in the near future. >>> > >>> > My proposal would be, once we reach to that state, we can mark the >>> first subsequent Beam release as the last Beam release that supports Python >>> 2. (Alternatively: in line with the previous experimental/deprecated >>> discussion we can make 2 more release with python 2 support rather than >>> just 1 more.) With the current state, we would not give users plenty of >>> time to upgrade python 3. So in addition, I would suggest we can consider >>> and upgrade relief by offering something like a 6-month support on the last >>> python 2 compatible release. We might do that in the context of an LTS >>> release. >>> > >>> > I do not believe we have a timeline we can share with users at this >>> point. However if we go with this suggestion, we will probably support >>> python 2 approximately until mid-2020. >>> > >>> > Ahmet >>> > >>> > On Wed, Jun 5, 2019 at 4:53 AM Tanay Tummalapalli >>> wrote: >>> >> >>> >> We can support Python 2 for some time in 2020, but, we should target >>> a date no later than 2020 to drop support. >>> >> If we do plan to drop support for Python 2 in 2020, we should sign >>> the Python 3 statement[1], declaring that we will "drop support for Python >>> 2.7 no later than 2020". >>> >> >>> >> In addition to the statement, keeping a target release and date(if >>> possible) or timeline to drop support would also help users to decide when >>> they need to work on migrating to Python 3. >>> >> >>> >> Regards, >>> >> - TT >>> >> >>> >> [1] https://python3statement.org/ >>> >> >>> >> On Wed, Jun 5, 2019 at 4:37 PM Robert Bradshaw >>> wrote: >>> >>> >>> >>> Until Python 3 support for Beam is officially out of beta and >>> >>> recommended, I don't think we can tell people to stop using Python 2. >>> >>> Given that 2020 is just over 6 months away, that seems a short >>> >>> transition time, so I would guess we'll have to continue supporting >>> >>> Python 2 sometime into 2020. >>> >>> >>> >>> A quick survey of users would be valuable here. But first priority is >>> >>> making
Re: Plan for dropping python 2 support
Sounds good. On Fri, Jun 7, 2019 at 8:28 PM Ahmet Altay wrote: > I agree with you. A more recent LTS release with python 2 support will be > good. Cost of maintaining python 2 support is also fairly low (maybe zero > actually besides keeping some pre-existing compatibility code). > > I believe we are referring to two separate things with support: > - Supporting existing releases for patches - I agree that we need to give > users a long enough window to upgrade. Great if it happens with an LTS > release. Even if it does not, I think it will be fair to offer patches on > the last python 2 supporting release during some part of 2020 if that > becomes necessary. > - Making new releases with python 2 support - Each new Beam release with > python 2 support will implicitly extend the lifetime of beam's python 2 > support. I do not think we need to extend this to beyond 2019. 2 releases > (~ 3 months) after solid python 3 support will very likely put the last > python 2 supporting release to last quarter of 2019 already. > > On Fri, Jun 7, 2019 at 2:15 AM Robert Bradshaw > wrote: > >> I don't think the second release with robust/recommended Python 3 >> support should be the last release with Python 2 support--that is >> simply not enough time for people to migrate. (Look at how long it >> took us...) It does make a lot of sense to at least have one LTS >> release with support for both. >> >> Regarding timeline, I think we could safely say we expect to support >> Python 2 through 2019, likely for some of 2020 (possibly only via an >> LTS release), and (very) unlikely beyond 2020. >> >> On Wed, Jun 5, 2019 at 6:34 PM Ahmet Altay wrote: >> > >> > I agree with the sentiment on this thread. Our priority needs to be >> offering good python 3 support that we can comfortably recommend users to >> switch. Progress on that so far has been promising and I do anticipate that >> we will reach there in the near future. >> > >> > My proposal would be, once we reach to that state, we can mark the >> first subsequent Beam release as the last Beam release that supports Python >> 2. (Alternatively: in line with the previous experimental/deprecated >> discussion we can make 2 more release with python 2 support rather than >> just 1 more.) With the current state, we would not give users plenty of >> time to upgrade python 3. So in addition, I would suggest we can consider >> and upgrade relief by offering something like a 6-month support on the last >> python 2 compatible release. We might do that in the context of an LTS >> release. >> > >> > I do not believe we have a timeline we can share with users at this >> point. However if we go with this suggestion, we will probably support >> python 2 approximately until mid-2020. >> > >> > Ahmet >> > >> > On Wed, Jun 5, 2019 at 4:53 AM Tanay Tummalapalli >> wrote: >> >> >> >> We can support Python 2 for some time in 2020, but, we should target a >> date no later than 2020 to drop support. >> >> If we do plan to drop support for Python 2 in 2020, we should sign the >> Python 3 statement[1], declaring that we will "drop support for Python 2.7 >> no later than 2020". >> >> >> >> In addition to the statement, keeping a target release and date(if >> possible) or timeline to drop support would also help users to decide when >> they need to work on migrating to Python 3. >> >> >> >> Regards, >> >> - TT >> >> >> >> [1] https://python3statement.org/ >> >> >> >> On Wed, Jun 5, 2019 at 4:37 PM Robert Bradshaw >> wrote: >> >>> >> >>> Until Python 3 support for Beam is officially out of beta and >> >>> recommended, I don't think we can tell people to stop using Python 2. >> >>> Given that 2020 is just over 6 months away, that seems a short >> >>> transition time, so I would guess we'll have to continue supporting >> >>> Python 2 sometime into 2020. >> >>> >> >>> A quick survey of users would be valuable here. But first priority is >> >>> making Python 3 rock solid so we can unconditionally recommend it over >> >>> Python 2. >> >>> >> >>> On Wed, Jun 5, 2019 at 12:27 PM Ismaël Mejía >> wrote: >> >>> > >> >>> > Python 2 won't be maintained after 2020 [1]. I was wondering what >> will >> >>> > be our (Beam) plan for this. Other projects [2] have started to >> alert >> >>> > users that support will be removed so maybe we should decide or >> policy >> >>> > for this too. >> >>> > >> >>> > [1] https://pythonclock.org/ >> >>> > [2] >> https://spark.apache.org/news/plan-for-dropping-python-2-support.html >> >
Re: Plan for dropping python 2 support
I agree with you. A more recent LTS release with python 2 support will be good. Cost of maintaining python 2 support is also fairly low (maybe zero actually besides keeping some pre-existing compatibility code). I believe we are referring to two separate things with support: - Supporting existing releases for patches - I agree that we need to give users a long enough window to upgrade. Great if it happens with an LTS release. Even if it does not, I think it will be fair to offer patches on the last python 2 supporting release during some part of 2020 if that becomes necessary. - Making new releases with python 2 support - Each new Beam release with python 2 support will implicitly extend the lifetime of beam's python 2 support. I do not think we need to extend this to beyond 2019. 2 releases (~ 3 months) after solid python 3 support will very likely put the last python 2 supporting release to last quarter of 2019 already. On Fri, Jun 7, 2019 at 2:15 AM Robert Bradshaw wrote: > I don't think the second release with robust/recommended Python 3 > support should be the last release with Python 2 support--that is > simply not enough time for people to migrate. (Look at how long it > took us...) It does make a lot of sense to at least have one LTS > release with support for both. > > Regarding timeline, I think we could safely say we expect to support > Python 2 through 2019, likely for some of 2020 (possibly only via an > LTS release), and (very) unlikely beyond 2020. > > On Wed, Jun 5, 2019 at 6:34 PM Ahmet Altay wrote: > > > > I agree with the sentiment on this thread. Our priority needs to be > offering good python 3 support that we can comfortably recommend users to > switch. Progress on that so far has been promising and I do anticipate that > we will reach there in the near future. > > > > My proposal would be, once we reach to that state, we can mark the first > subsequent Beam release as the last Beam release that supports Python 2. > (Alternatively: in line with the previous experimental/deprecated > discussion we can make 2 more release with python 2 support rather than > just 1 more.) With the current state, we would not give users plenty of > time to upgrade python 3. So in addition, I would suggest we can consider > and upgrade relief by offering something like a 6-month support on the last > python 2 compatible release. We might do that in the context of an LTS > release. > > > > I do not believe we have a timeline we can share with users at this > point. However if we go with this suggestion, we will probably support > python 2 approximately until mid-2020. > > > > Ahmet > > > > On Wed, Jun 5, 2019 at 4:53 AM Tanay Tummalapalli > wrote: > >> > >> We can support Python 2 for some time in 2020, but, we should target a > date no later than 2020 to drop support. > >> If we do plan to drop support for Python 2 in 2020, we should sign the > Python 3 statement[1], declaring that we will "drop support for Python 2.7 > no later than 2020". > >> > >> In addition to the statement, keeping a target release and date(if > possible) or timeline to drop support would also help users to decide when > they need to work on migrating to Python 3. > >> > >> Regards, > >> - TT > >> > >> [1] https://python3statement.org/ > >> > >> On Wed, Jun 5, 2019 at 4:37 PM Robert Bradshaw > wrote: > >>> > >>> Until Python 3 support for Beam is officially out of beta and > >>> recommended, I don't think we can tell people to stop using Python 2. > >>> Given that 2020 is just over 6 months away, that seems a short > >>> transition time, so I would guess we'll have to continue supporting > >>> Python 2 sometime into 2020. > >>> > >>> A quick survey of users would be valuable here. But first priority is > >>> making Python 3 rock solid so we can unconditionally recommend it over > >>> Python 2. > >>> > >>> On Wed, Jun 5, 2019 at 12:27 PM Ismaël Mejía > wrote: > >>> > > >>> > Python 2 won't be maintained after 2020 [1]. I was wondering what > will > >>> > be our (Beam) plan for this. Other projects [2] have started to alert > >>> > users that support will be removed so maybe we should decide or > policy > >>> > for this too. > >>> > > >>> > [1] https://pythonclock.org/ > >>> > [2] > https://spark.apache.org/news/plan-for-dropping-python-2-support.html >
Re: Plan for dropping python 2 support
I don't think the second release with robust/recommended Python 3 support should be the last release with Python 2 support--that is simply not enough time for people to migrate. (Look at how long it took us...) It does make a lot of sense to at least have one LTS release with support for both. Regarding timeline, I think we could safely say we expect to support Python 2 through 2019, likely for some of 2020 (possibly only via an LTS release), and (very) unlikely beyond 2020. On Wed, Jun 5, 2019 at 6:34 PM Ahmet Altay wrote: > > I agree with the sentiment on this thread. Our priority needs to be offering > good python 3 support that we can comfortably recommend users to switch. > Progress on that so far has been promising and I do anticipate that we will > reach there in the near future. > > My proposal would be, once we reach to that state, we can mark the first > subsequent Beam release as the last Beam release that supports Python 2. > (Alternatively: in line with the previous experimental/deprecated discussion > we can make 2 more release with python 2 support rather than just 1 more.) > With the current state, we would not give users plenty of time to upgrade > python 3. So in addition, I would suggest we can consider and upgrade relief > by offering something like a 6-month support on the last python 2 compatible > release. We might do that in the context of an LTS release. > > I do not believe we have a timeline we can share with users at this point. > However if we go with this suggestion, we will probably support python 2 > approximately until mid-2020. > > Ahmet > > On Wed, Jun 5, 2019 at 4:53 AM Tanay Tummalapalli wrote: >> >> We can support Python 2 for some time in 2020, but, we should target a date >> no later than 2020 to drop support. >> If we do plan to drop support for Python 2 in 2020, we should sign the >> Python 3 statement[1], declaring that we will "drop support for Python 2.7 >> no later than 2020". >> >> In addition to the statement, keeping a target release and date(if possible) >> or timeline to drop support would also help users to decide when they need >> to work on migrating to Python 3. >> >> Regards, >> - TT >> >> [1] https://python3statement.org/ >> >> On Wed, Jun 5, 2019 at 4:37 PM Robert Bradshaw wrote: >>> >>> Until Python 3 support for Beam is officially out of beta and >>> recommended, I don't think we can tell people to stop using Python 2. >>> Given that 2020 is just over 6 months away, that seems a short >>> transition time, so I would guess we'll have to continue supporting >>> Python 2 sometime into 2020. >>> >>> A quick survey of users would be valuable here. But first priority is >>> making Python 3 rock solid so we can unconditionally recommend it over >>> Python 2. >>> >>> On Wed, Jun 5, 2019 at 12:27 PM Ismaël Mejía wrote: >>> > >>> > Python 2 won't be maintained after 2020 [1]. I was wondering what will >>> > be our (Beam) plan for this. Other projects [2] have started to alert >>> > users that support will be removed so maybe we should decide or policy >>> > for this too. >>> > >>> > [1] https://pythonclock.org/ >>> > [2] https://spark.apache.org/news/plan-for-dropping-python-2-support.html
Re: Plan for dropping python 2 support
I agree with the sentiment on this thread. Our priority needs to be offering good python 3 support that we can comfortably recommend users to switch. Progress on that so far has been promising and I do anticipate that we will reach there in the near future. My proposal would be, once we reach to that state, we can mark the first subsequent Beam release as the last Beam release that supports Python 2. (Alternatively: in line with the previous experimental/deprecated discussion we can make 2 more release with python 2 support rather than just 1 more.) With the current state, we would not give users plenty of time to upgrade python 3. So in addition, I would suggest we can consider and upgrade relief by offering something like a 6-month support on the last python 2 compatible release. We might do that in the context of an LTS release. I do not believe we have a timeline we can share with users at this point. However if we go with this suggestion, we will probably support python 2 approximately until mid-2020. Ahmet On Wed, Jun 5, 2019 at 4:53 AM Tanay Tummalapalli wrote: > We can support Python 2 for some time in 2020, but, we should target a > date no later than 2020 to drop support. > If we do plan to drop support for Python 2 in 2020, we should sign the > Python 3 statement[1], declaring that we will "drop support for Python 2.7 > no later than 2020". > > In addition to the statement, keeping a target release and date(if > possible) or timeline to drop support would also help users to decide when > they need to work on migrating to Python 3. > > Regards, > - TT > > [1] https://python3statement.org/ > > On Wed, Jun 5, 2019 at 4:37 PM Robert Bradshaw > wrote: > >> Until Python 3 support for Beam is officially out of beta and >> recommended, I don't think we can tell people to stop using Python 2. >> Given that 2020 is just over 6 months away, that seems a short >> transition time, so I would guess we'll have to continue supporting >> Python 2 sometime into 2020. >> >> A quick survey of users would be valuable here. But first priority is >> making Python 3 rock solid so we can unconditionally recommend it over >> Python 2. >> >> On Wed, Jun 5, 2019 at 12:27 PM Ismaël Mejía wrote: >> > >> > Python 2 won't be maintained after 2020 [1]. I was wondering what will >> > be our (Beam) plan for this. Other projects [2] have started to alert >> > users that support will be removed so maybe we should decide or policy >> > for this too. >> > >> > [1] https://pythonclock.org/ >> > [2] >> https://spark.apache.org/news/plan-for-dropping-python-2-support.html >> >
Re: Plan for dropping python 2 support
We can support Python 2 for some time in 2020, but, we should target a date no later than 2020 to drop support. If we do plan to drop support for Python 2 in 2020, we should sign the Python 3 statement[1], declaring that we will "drop support for Python 2.7 no later than 2020". In addition to the statement, keeping a target release and date(if possible) or timeline to drop support would also help users to decide when they need to work on migrating to Python 3. Regards, - TT [1] https://python3statement.org/ On Wed, Jun 5, 2019 at 4:37 PM Robert Bradshaw wrote: > Until Python 3 support for Beam is officially out of beta and > recommended, I don't think we can tell people to stop using Python 2. > Given that 2020 is just over 6 months away, that seems a short > transition time, so I would guess we'll have to continue supporting > Python 2 sometime into 2020. > > A quick survey of users would be valuable here. But first priority is > making Python 3 rock solid so we can unconditionally recommend it over > Python 2. > > On Wed, Jun 5, 2019 at 12:27 PM Ismaël Mejía wrote: > > > > Python 2 won't be maintained after 2020 [1]. I was wondering what will > > be our (Beam) plan for this. Other projects [2] have started to alert > > users that support will be removed so maybe we should decide or policy > > for this too. > > > > [1] https://pythonclock.org/ > > [2] > https://spark.apache.org/news/plan-for-dropping-python-2-support.html >
Re: Plan for dropping python 2 support
Until Python 3 support for Beam is officially out of beta and recommended, I don't think we can tell people to stop using Python 2. Given that 2020 is just over 6 months away, that seems a short transition time, so I would guess we'll have to continue supporting Python 2 sometime into 2020. A quick survey of users would be valuable here. But first priority is making Python 3 rock solid so we can unconditionally recommend it over Python 2. On Wed, Jun 5, 2019 at 12:27 PM Ismaël Mejía wrote: > > Python 2 won't be maintained after 2020 [1]. I was wondering what will > be our (Beam) plan for this. Other projects [2] have started to alert > users that support will be removed so maybe we should decide or policy > for this too. > > [1] https://pythonclock.org/ > [2] https://spark.apache.org/news/plan-for-dropping-python-2-support.html
Plan for dropping python 2 support
Python 2 won't be maintained after 2020 [1]. I was wondering what will be our (Beam) plan for this. Other projects [2] have started to alert users that support will be removed so maybe we should decide or policy for this too. [1] https://pythonclock.org/ [2] https://spark.apache.org/news/plan-for-dropping-python-2-support.html
