[VOTE][RESULT] Spark 2.4.4 (RC3)

Hi, All. The vote passes. Thanks to all who helped with this release 2.4.4! It was very intensive vote with +11 (including +8 PMC votes) and no -1. I'll follow up later with a release announcement once everything is published. +1 (* = binding): Dongjoon Hyun Kazuaki Ishizaki Sean Owen* Wenchen

Re: DSv2 sync notes - 21 August 2019

Sorry all, I'm out on vacation till the week after next, I'll chime in then unless we need consensus sooner On Fri, Aug 30, 2019, 4:05 PM Ryan Blue wrote: > Sorry these notes were delayed. Here’s what we talked about in the last > DSv2 sync. > > *Attendees*: > > Ryan Blue > John Zhuge > Burak

Re: [VOTE] Release Apache Spark 2.4.4 (RC3)

+1 Xiao Felix Cheung 于2019年8月30日周五 上午2:03写道： > +1 > > Run tests, R tests, r-hub Debian, Ubuntu, mac, Windows > > -- > *From:* Hyukjin Kwon > *Sent:* Wednesday, August 28, 2019 9:14 PM > *To:* Takeshi Yamamuro > *Cc:* dev; Dongjoon Hyun > *Subject:* Re: [VOTE]

DSv2 sync notes - 21 August 2019

Sorry these notes were delayed. Here’s what we talked about in the last DSv2 sync. *Attendees*: Ryan Blue John Zhuge Burak Yavuz Gengliang Wang Terry Kim Wenchen Fan Xin Ren Srabasti Banerjee Priyanka Gomatam *Topics*: - Follow up on renaming append to insert in v2 API - Changes to

Re: Providing a namespace for third-party configurations

My 2 cents, I second with Sean, as it's not necessary for downstream projects to have a rule for config name starting with "spark". I guess I know why they want it, but then it should be clear what's the goal. To differentiate with existing spark configs? Using another prefix would help. To

Re: Providing a namespace for third-party configurations

It's possible, but pretty unlikely to have an exact namespace collision. It's probably a best practice to clearly separates settings, etc that are downstream add-ons into a separate namespace, and I don't mind a sentence in a doc somewhere suggesting a convention, but I really think it's up to

Providing a namespace for third-party configurations

I discovered today that EMR provides its own optimizations for Spark . Some of these optimizations are controlled by configuration settings with names like `spark.sql.dynamicPartitionPruning.enabled` or

Re: [VOTE] Release Apache Spark 2.4.4 (RC3)

+1 Run tests, R tests, r-hub Debian, Ubuntu, mac, Windows From: Hyukjin Kwon Sent: Wednesday, August 28, 2019 9:14 PM To: Takeshi Yamamuro Cc: dev; Dongjoon Hyun Subject: Re: [VOTE] Release Apache Spark 2.4.4 (RC3) +1 (from the last blocker PR) 2019년 8월 29일