+1 I think phasing out EOL of any feature or supported language is a better strategy if possible than a quick drop. With enough admonition, it can gradually be dropped in 3.x— of course, there are exceptions.
Cheers Jules Sent from my iPhone Pardon the dumb thumb typos :) > On Sep 15, 2018, at 10:49 AM, Reynold Xin <r...@databricks.com> wrote: > > we can also declare python 2 as deprecated and drop it in 3.x, not > necessarily 3.0. > > -- > excuse the brevity and lower case due to wrist injury > > >> On Sat, Sep 15, 2018 at 10:33 AM Erik Erlandson <eerla...@redhat.com> wrote: >> I am probably splitting hairs to finely, but I was considering the >> difference between improvements to the jvm-side (py4j and the scala/java >> code) that would make it easier to write the python layer ("python-friendly >> api"), and actual improvements to the python layers ("friendly python api"). >> >> They're not mutually exclusive of course, and both worth working on. But >> it's *possible* to improve either without the other. >> >> Stub files look like a great solution for type annotations, maybe even if >> only python 3 is supported. >> >> I definitely agree that any decision to drop python 2 should not be taken >> lightly. Anecdotally, I'm seeing an increase in python developers announcing >> that they are dropping support for python 2 (and loving it). As people have >> already pointed out, if we don't drop python 2 for spark 3.0, we're stuck >> with it until 4.0, which would place spark in a possibly-awkward position of >> supporting python 2 for some time after it goes EOL. >> >> Under the current release cadence, spark 3.0 will land some time in early >> 2019, which at that point will be mere months until EOL for py2. >> >>> On Fri, Sep 14, 2018 at 5:01 PM, Holden Karau <hol...@pigscanfly.ca> wrote: >>> >>> >>>> On Fri, Sep 14, 2018, 3:26 PM Erik Erlandson <eerla...@redhat.com> wrote: >>>> To be clear, is this about "python-friendly API" or "friendly python API" ? >>> >>> Well what would you consider to be different between those two statements? >>> I think it would be good to be a bit more explicit, but I don't think we >>> should necessarily limit ourselves. >>>> >>>> On the python side, it might be nice to take advantage of static typing. >>>> Requires python 3.6 but with python 2 going EOL, a spark-3.0 might be a >>>> good opportunity to jump the python-3-only train. >>> >>> I think we can make types sort of work without ditching 2 (the types only >>> would work in 3 but it would still function in 2). Ditching 2 entirely >>> would be a big thing to consider, I honestly hadn't been considering that >>> but it could be from just spending so much time maintaining a 2/3 code >>> base. I'd suggest reaching out to to user@ before making that kind of >>> change. >>>> >>>>> On Fri, Sep 14, 2018 at 12:15 PM, Holden Karau <hol...@pigscanfly.ca> >>>>> wrote: >>>>> Since we're talking about Spark 3.0 in the near future (and since some >>>>> recent conversation on a proposed change reminded me) I wanted to open up >>>>> the floor and see if folks have any ideas on how we could make a more >>>>> Python friendly API for 3.0? I'm planning on taking some time to look at >>>>> other systems in the solution space and see what we might want to learn >>>>> from them but I'd love to hear what other folks are thinking too. >>>>> >>>>> -- >>>>> Twitter: https://twitter.com/holdenkarau >>>>> Books (Learning Spark, High Performance Spark, etc.): >>>>> https://amzn.to/2MaRAG9 >>>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>>> >>