Re: Remove non-Tungsten mode in Spark 3?

The issue with the offheap mode is it is a pretty big behavior change and does require additional setup (also for users that run with UDFs that allocate a lot of heap memory, it might not be as good). I can see us removing the legacy mode since it's been legacy for a long time and perhaps very

Re: Apache Spark 2.2.3 ?

Thank you, Sean! Bests, Dongjoon. On Thu, Jan 3, 2019 at 2:50 PM Sean Owen wrote: > Yes, that one's not going to be back-ported to 2.3. I think it's fine to > proceed with a 2.2 release with what's there now and call it done. > Note that Spark 2.3 would be EOL around September of this year. >

Remove non-Tungsten mode in Spark 3?

Just wondering if there is a good reason to keep around the pre-tungsten on-heap memory mode for Spark 3, and make spark.memory.offHeap.enabled always true? It would simplify the code somewhat, but I don't feel I'm so aware of the tradeoffs. I know we didn't deprecate it, but it's been off by

Re: Apache Spark 2.2.3 ?

Yes, that one's not going to be back-ported to 2.3. I think it's fine to proceed with a 2.2 release with what's there now and call it done. Note that Spark 2.3 would be EOL around September of this year. On Thu, Jan 3, 2019 at 2:31 PM Dongjoon Hyun wrote: > Thank you for additional support for

Re: Apache Spark 2.2.3 ?

Thank you for additional support for 2.2.3, Felix and Takeshi! The following is the update for Apache Spark 2.2.3 release. For correctness issues, two more patches landed on `branch-2.2`. SPARK-22951 fix aggregation after dropDuplicates on empty dataframes SPARK-25591 Avoid

Re:

Unsub me 2 pls. On Thu, 3 Jan 2019 at 15:22, marco rocchi < rocchi.1407...@studenti.uniroma1.it> wrote: > Unsubscribe me, please. > > Thank you so much >

[no subject]

Unsubscribe me, please. Thank you so much

Re: SPARk-25299: Updates As Of December 19, 2018

Hi Matt, i'm a developer of SparkRDMA shuffle manager: https://github.com/Mellanox/SparkRDMA Thanks for your effort on improving Spark Shuffle API. We are very interested in participating in this. Have for now several comments: 1. Went through these 4 documents:

Re: Apache Spark 2.2.3 ?

Hi, all, happy new year! +1 on the release of 2.2.3/2.3.3. I checked there is no ongoing issue targeting on 2.3.3, too. On Thu, Jan 3, 2019 at 8:50 AM Felix Cheung wrote: > +1 on 2.2.3 of course > > > -- > *From:* Dongjoon Hyun > *Sent:* Wednesday, January 2, 2019