What's the release due for Apache Spark 3.0? Will it be tomorrow or somewhere at the middle of 2019 year?
I think we shouldn't care much about Python 2.x today, since quite soon it support turns into pumpkin. For today's projects I hope nobody takes into account support of 2.7 unless there is some legacy still to carry on, but do we want to take that baggage into Apache Spark 3.x era? The next time you may drop it would be only 4.0 release because of breaking change. -- ,,,^..^,,, On Sat, Sep 15, 2018 at 2:21 PM Maciej Szymkiewicz <mszymkiew...@gmail.com> wrote: > > There is no need to ditch Python 2. There are basically two options > > Use stub files and limit yourself to support only Python 3 support. Python 3 > users benefit from type hints, Python 2 users don't, but no core > functionality is affected. This is the approach I've used with > https://github.com/zero323/pyspark-stubs/. > Use comment based inline syntax or stub files and don't use backward > incompatible features (primarily typing module - > https://docs.python.org/3/library/typing.html). Both Python 2 and 3 is > supported, but more advanced components are not. Small win for Python 2 > users, moderate loss for Python 3 users. > > > > On Sat, 15 Sep 2018 at 02:38, Nicholas Chammas <nicholas.cham...@gmail.com> > wrote: >> >> Do we need to ditch Python 2 support to provide type hints? I don’t think so. >> >> Python lets you specify typing stubs that provide the same benefit without >> forcing Python 3. >> >> 2018년 9월 14일 (금) 오후 8:01, Holden Karau <hol...@pigscanfly.ca>님이 작성: >>> >>> >>> >>> On Fri, Sep 14, 2018, 3:26 PM Erik Erlandson <eerla...@redhat.com> wrote: >>>> >>>> To be clear, is this about "python-friendly API" or "friendly python API" ? >>> >>> Well what would you consider to be different between those two statements? >>> I think it would be good to be a bit more explicit, but I don't think we >>> should necessarily limit ourselves. >>>> >>>> >>>> On the python side, it might be nice to take advantage of static typing. >>>> Requires python 3.6 but with python 2 going EOL, a spark-3.0 might be a >>>> good opportunity to jump the python-3-only train. >>> >>> I think we can make types sort of work without ditching 2 (the types only >>> would work in 3 but it would still function in 2). Ditching 2 entirely >>> would be a big thing to consider, I honestly hadn't been considering that >>> but it could be from just spending so much time maintaining a 2/3 code >>> base. I'd suggest reaching out to to user@ before making that kind of >>> change. >>>> >>>> >>>> On Fri, Sep 14, 2018 at 12:15 PM, Holden Karau <hol...@pigscanfly.ca> >>>> wrote: >>>>> >>>>> Since we're talking about Spark 3.0 in the near future (and since some >>>>> recent conversation on a proposed change reminded me) I wanted to open up >>>>> the floor and see if folks have any ideas on how we could make a more >>>>> Python friendly API for 3.0? I'm planning on taking some time to look at >>>>> other systems in the solution space and see what we might want to learn >>>>> from them but I'd love to hear what other folks are thinking too. >>>>> >>>>> -- >>>>> Twitter: https://twitter.com/holdenkarau >>>>> Books (Learning Spark, High Performance Spark, etc.): >>>>> https://amzn.to/2MaRAG9 >>>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>>> >>>> > > --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
