Re: [DISCUSS] Migrate development scripts under dev/ from Python2 to Python 3
Yeah, we will probably drop Python 2 entirely after 3.0.0. Python 2 is already deprecated. On Thu, 15 Aug 2019, 18:25 Driesprong, Fokko, wrote: > Sorry for the late reply, was a bit busy lately, but I still would like to > share my thoughts on this. > > For Apache Airflow we're dropping support for Python 2 in the next major > release. We're now supporting Python 3.5+. Mostly because: > >- Easier to maintain and test, and less if/else constructions for the >different Python versions. Also, not having to test against Python 2.x >reduces the build matrix. >- Python 3 has support for typing. From Python 3.5 you can include >provisional type hints. An excellent presentation by Guido himself: >https://www.youtube.com/watch?v=2wDvzy6Hgxg. From Python 3.5 it is >still provisional, but it is a really good idea. From Airflow we've noticed >that using mypy is catching bugs early: > - This will put less stress on the (boring part of the) reviewing > process since a lot of this stuff is checked automatically. > - For new developers, it is easier to read the code because of the > annotations. > - Can be used as an input for generated documentation (or check if > it still in sync with the docstrings) > - Easier to extend the code since you know what kind of types you > can expect, and your IDE will also pick up the hinting. >- Python 2.x will be EOL end this year > > I have a strong preference to migrate everything to Python 3. > > Cheers, Fokko > > > Op wo 7 aug. 2019 om 12:14 schreef Weichen Xu : > >> All right we could support both Python 2 and Python 3 for spark 3.0. >> >> On Wed, Aug 7, 2019 at 6:10 PM Hyukjin Kwon wrote: >> >>> We didn't drop Python 2 yet although it's deprecated. So I think It >>> should support both Python 2 and Python 3 at the current status. >>> >>> 2019년 8월 7일 (수) 오후 6:54, Weichen Xu 님이 작성: >>> Hi all, I would like to discuss the compatibility for dev scripts. Because we already decided to deprecate python2 in spark 3.0, for development scripts under dev/ , we have two choice: 1) Migration from Python 2 to Python 3 2) Support both Python 2 and Python 3 I tend to option (2) which is more friendly to maintenance. Regards, Weichen >>>
Re: [DISCUSS] Migrate development scripts under dev/ from Python2 to Python 3
I mean python 2 _will be_ deprecated in Spark 3. On Thu, 15 Aug 2019, 18:37 Hyukjin Kwon, wrote: > Yeah, we will probably drop Python 2 entirely after 3.0.0. Python 2 is > already deprecated. > > On Thu, 15 Aug 2019, 18:25 Driesprong, Fokko, > wrote: > >> Sorry for the late reply, was a bit busy lately, but I still would like >> to share my thoughts on this. >> >> For Apache Airflow we're dropping support for Python 2 in the next major >> release. We're now supporting Python 3.5+. Mostly because: >> >>- Easier to maintain and test, and less if/else constructions for the >>different Python versions. Also, not having to test against Python 2.x >>reduces the build matrix. >>- Python 3 has support for typing. From Python 3.5 you can include >>provisional type hints. An excellent presentation by Guido himself: >>https://www.youtube.com/watch?v=2wDvzy6Hgxg. From Python 3.5 it is >>still provisional, but it is a really good idea. From Airflow we've >> noticed >>that using mypy is catching bugs early: >> - This will put less stress on the (boring part of the) reviewing >> process since a lot of this stuff is checked automatically. >> - For new developers, it is easier to read the code because of the >> annotations. >> - Can be used as an input for generated documentation (or check if >> it still in sync with the docstrings) >> - Easier to extend the code since you know what kind of types you >> can expect, and your IDE will also pick up the hinting. >>- Python 2.x will be EOL end this year >> >> I have a strong preference to migrate everything to Python 3. >> >> Cheers, Fokko >> >> >> Op wo 7 aug. 2019 om 12:14 schreef Weichen Xu > >: >> >>> All right we could support both Python 2 and Python 3 for spark 3.0. >>> >>> On Wed, Aug 7, 2019 at 6:10 PM Hyukjin Kwon wrote: >>> We didn't drop Python 2 yet although it's deprecated. So I think It should support both Python 2 and Python 3 at the current status. 2019년 8월 7일 (수) 오후 6:54, Weichen Xu 님이 작성: > Hi all, > > I would like to discuss the compatibility for dev scripts. Because we > already decided to deprecate python2 in spark 3.0, for development scripts > under dev/ , we have two choice: > 1) Migration from Python 2 to Python 3 > 2) Support both Python 2 and Python 3 > > I tend to option (2) which is more friendly to maintenance. > > Regards, > Weichen >
Re: [DISCUSS] Migrate development scripts under dev/ from Python2 to Python 3
Sorry for the late reply, was a bit busy lately, but I still would like to share my thoughts on this. For Apache Airflow we're dropping support for Python 2 in the next major release. We're now supporting Python 3.5+. Mostly because: - Easier to maintain and test, and less if/else constructions for the different Python versions. Also, not having to test against Python 2.x reduces the build matrix. - Python 3 has support for typing. From Python 3.5 you can include provisional type hints. An excellent presentation by Guido himself: https://www.youtube.com/watch?v=2wDvzy6Hgxg. From Python 3.5 it is still provisional, but it is a really good idea. From Airflow we've noticed that using mypy is catching bugs early: - This will put less stress on the (boring part of the) reviewing process since a lot of this stuff is checked automatically. - For new developers, it is easier to read the code because of the annotations. - Can be used as an input for generated documentation (or check if it still in sync with the docstrings) - Easier to extend the code since you know what kind of types you can expect, and your IDE will also pick up the hinting. - Python 2.x will be EOL end this year I have a strong preference to migrate everything to Python 3. Cheers, Fokko Op wo 7 aug. 2019 om 12:14 schreef Weichen Xu : > All right we could support both Python 2 and Python 3 for spark 3.0. > > On Wed, Aug 7, 2019 at 6:10 PM Hyukjin Kwon wrote: > >> We didn't drop Python 2 yet although it's deprecated. So I think It >> should support both Python 2 and Python 3 at the current status. >> >> 2019년 8월 7일 (수) 오후 6:54, Weichen Xu 님이 작성: >> >>> Hi all, >>> >>> I would like to discuss the compatibility for dev scripts. Because we >>> already decided to deprecate python2 in spark 3.0, for development scripts >>> under dev/ , we have two choice: >>> 1) Migration from Python 2 to Python 3 >>> 2) Support both Python 2 and Python 3 >>> >>> I tend to option (2) which is more friendly to maintenance. >>> >>> Regards, >>> Weichen >>> >>
Re: [DISCUSS] Migrate development scripts under dev/ from Python2 to Python 3
All right we could support both Python 2 and Python 3 for spark 3.0. On Wed, Aug 7, 2019 at 6:10 PM Hyukjin Kwon wrote: > We didn't drop Python 2 yet although it's deprecated. So I think It should > support both Python 2 and Python 3 at the current status. > > 2019년 8월 7일 (수) 오후 6:54, Weichen Xu 님이 작성: > >> Hi all, >> >> I would like to discuss the compatibility for dev scripts. Because we >> already decided to deprecate python2 in spark 3.0, for development scripts >> under dev/ , we have two choice: >> 1) Migration from Python 2 to Python 3 >> 2) Support both Python 2 and Python 3 >> >> I tend to option (2) which is more friendly to maintenance. >> >> Regards, >> Weichen >> >
Re: [DISCUSS] Migrate development scripts under dev/ from Python2 to Python 3
We didn't drop Python 2 yet although it's deprecated. So I think It should support both Python 2 and Python 3 at the current status. 2019년 8월 7일 (수) 오후 6:54, Weichen Xu 님이 작성: > Hi all, > > I would like to discuss the compatibility for dev scripts. Because we > already decided to deprecate python2 in spark 3.0, for development scripts > under dev/ , we have two choice: > 1) Migration from Python 2 to Python 3 > 2) Support both Python 2 and Python 3 > > I tend to option (2) which is more friendly to maintenance. > > Regards, > Weichen >
[DISCUSS] Migrate development scripts under dev/ from Python2 to Python 3
Hi all, I would like to discuss the compatibility for dev scripts. Because we already decided to deprecate python2 in spark 3.0, for development scripts under dev/ , we have two choice: 1) Migration from Python 2 to Python 3 2) Support both Python 2 and Python 3 I tend to option (2) which is more friendly to maintenance. Regards, Weichen
