Re: Branch 2.4 is cut

Since it's not a clean revert, I've sent a PR to revert it from 2.4, please take a look, thanks! https://github.com/apache/spark/pull/22388 On Tue, Sep 11, 2018 at 1:16 AM Ryan Blue wrote: > SPARK-24882 was committed in order to make some progress, with a note > about following up with

Re: Branch 2.4 is cut

changes in 2.4 are adding now? From: Wenchen Fan Sent: Monday, September 10, 2018 12:35 AM To: ar...@apache.org Cc: Ryan Blue; skn...@berkeley.edu; Dongjoon Hyun; joshro...@databricks.com; Sean Owen; Spark dev list Subject: Re: Branch 2.4 is cut There are a lot

Re: Branch 2.4 is cut

There are a lot of "breaking" changes we made in 2.4 for data source v2, while I agree SPARK-24882 is "breaking" most. I don't agree SPARK-24882 is half-baked. But I'm willing to revert it if we have a bunch of data source v2 users and they are not willing to update their implementation intensely

Re: Branch 2.4 is cut

Ryan's proposal makes a lot of sense. Its better not to release half-baked changes in 2.4 which not only breaks a lot of the APIs released in 2.3, but also expected to change further due redesigns before 3.0 so don't see much value releasing it in 2.4. On Sun, 9 Sep 2018 at 22:42, Wenchen Fan

Re: Branch 2.4 is cut

Strictly speaking, data source v2 is always half-finished until we mark it as stable. We need some small milestones to move forward step by step. The redesign also happens in an incremental way. SPARK-24882 mostly focus on the "RDD" part of the API: the separation of reader factory and input

Re: Branch 2.4 is cut

Wenchen, can you hold off on the first RC? The half-finished changes from the redesign of the DataSourceV2 API are in master, added in SPARK-24882 , and are now in the 2.4 branch. We've had a lot of good discussion since that PR was merged to update and

Re: Branch 2.4 is cut

++joshrosen (thanks for the help w/deploying the jenkins configs) the basic 2.4 builds are deployed and building! i haven't created (a) build(s) yet for scala 2.12... i'll be coordinating this w/the databricks folks next week. On Fri, Sep 7, 2018 at 9:53 AM, Dongjoon Hyun wrote: > Thank

Re: Branch 2.4 is cut

Thank you, Shane! :D Bests, Dongjoon. On Fri, Sep 7, 2018 at 9:51 AM shane knapp wrote: > i'll try and get to the 2.4 branch stuff today... > >

Re: Branch 2.4 is cut

>>>> >>>>> Let's try also producing a 2.12 build with this release. The machinery >>>>> should be there in the release scripts, but let me know if something fails >>>>> while running the release for 2.12. >>>>> >>>>

Re: Branch 2.4 is cut

gt;> >>>>>> https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test/job/spark-master-test-maven-hadoop-2.7-ubuntu-scala-2.12/229/ >>>>>> >>>>>> Let's try also producing a 2.12 build with this release. The >>>>>> machine

Re: Branch 2.4 is cut

jenkins/view/Spark%20QA%20Test/job/spark-master-test-maven-hadoop-2.7-ubuntu-scala-2.12/229/ >>>>> >>>>> Let's try also producing a 2.12 build with this release. The machinery >>>>> should be there in the release scripts, but let me know if something fails >>>&g

Re: Branch 2.4 is cut

aven-hadoop-2.7-ubuntu-scala-2.12/229/ >>>> >>>> Let's try also producing a 2.12 build with this release. The machinery >>>> should be there in the release scripts, but let me know if something fails >>>> while running the release for 2.12. >&g

Re: Branch 2.4 is cut

op-2.7-ubuntu-scala-2.12/229/ >>> >>> Let's try also producing a 2.12 build with this release. The machinery >>> should be there in the release scripts, but let me know if something fails >>> while running the release for 2.12. >>> >>&g

Re: Branch 2.4 is cut

2.12/229/ >> >> Let's try also producing a 2.12 build with this release. The machinery >> should be there in the release scripts, but let me know if something fails >> while running the release for 2.12. >> >> On Thu, Sep 6, 2018 at 12:32 AM Wenchen Fan wrote: >>

Re: Branch 2.4 is cut

> Let's try also producing a 2.12 build with this release. The machinery > should be there in the release scripts, but let me know if something fails > while running the release for 2.12. > > On Thu, Sep 6, 2018 at 12:32 AM Wenchen Fan wrote: > >> Hi all, >> >

Re: Branch 2.4 is cut

know if something fails while running the release for 2.12. On Thu, Sep 6, 2018 at 12:32 AM Wenchen Fan wrote: > Hi all, > > I've cut the branch-2.4 since all the major blockers are resolved. If no > objections I'll shortly followup with an RC to get the QA started in > parallel.

Re: Branch 2.4 is cut

Thanks, Wenchen. 2018년 9월 6일 (목) 오후 3:32, Wenchen Fan 님이 작성: > Hi all, > > I've cut the branch-2.4 since all the major blockers are resolved. If no > objections I'll shortly followup with an RC to get the QA started in > parallel. > > Committers, please only merge PRs to br

Branch 2.4 is cut

Hi all, I've cut the branch-2.4 since all the major blockers are resolved. If no objections I'll shortly followup with an RC to get the QA started in parallel. Committers, please only merge PRs to branch-2.4 that are bug fixes, performance regression fixes, document changes, or test suites