Re: [VOTE] Apache Spark 3.0 RC2

Hyukjin, you're right that I could have looked more closely. Sorry for that. I definitely should have been more careful. rb On Fri, May 22, 2020 at 5:19 PM Hyukjin Kwon wrote: > Ryan, > > > I'm fine with the commit, other than the fact that it violated ASF norms >

Re: [VOTE] Apache Spark 3.0 RC2

Ryan, > I'm fine with the commit, other than the fact that it violated ASF norms to commit without waiting for a review. Looks it became the different proposal as you and other people discussed and suggested there, which you didn't technically vote

Re: [VOTE] Apache Spark 3.0 RC2

Hi all, Can we help review this pr and resolve this issue before spark-3.0 RC3. This is a fault tolerance bug in spark. not as serious as a correctness issue, but pretty high up.( I just cite the comment, https://github.com/apache/spark/pull/26339#issuecomment-632707720).

Re: [VOTE] Apache Spark 3.0 RC2

Thanks for reporting these issues! Please continue to test RC2 and report more issues. Cheers, Xiao On Fri, May 22, 2020 at 7:40 AM Koert Kuipers wrote: > i would like to point out that SPARK-27194 is a fault tolerance bug that > causes jobs to fail when any single task is retried. for us

Re: [VOTE] Apache Spark 3.0 RC2

i would like to point out that SPARK-27194 is a fault tolerance bug that causes jobs to fail when any single task is retried. for us this is a major headache because we have to keep restarting jobs (and explain that spark is really fault tolerant generally, just not here).

Re: [VOTE] Apache Spark 3.0 RC2

Looks like there're new blocker issues newly figured out. * https://issues.apache.org/jira/browse/SPARK-31786 * https://issues.apache.org/jira/browse/SPARK-31761 (not yet marked as blocker but according to JIRA comment it's a regression issue as well as correctness issue IMHO) Let's collect the

Re: [VOTE] Apache Spark 3.0 RC2

Okay, I took a look at the PR and I think it should be okay. The new classes are unfortunately public, but are in catalyst which is considered private. So this is the approach we discussed. I'm fine with the commit, other than the fact that it violated ASF norms

Re: [VOTE] Apache Spark 3.0 RC2

Why was https://github.com/apache/spark/pull/28523 merged with a -1? We discussed this months ago and concluded that it was a bad idea to introduce a new v2 API that cannot have reliable behavior across sources. The last time I checked that PR, the approach I discussed with Tathagata was to not

Re: [VOTE] Apache Spark 3.0 RC2

Seems the priority of SPARK-31706 is incorrectly marked, and it's a blocker now. The fix was merged just a few hours ago. This should be a -1 for RC2. On Wed, May 20, 2020 at 2:42 PM rickestcode wrote: > +1 > > > > -- > Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ > >

Re: [VOTE] Apache Spark 3.0 RC2

+1 -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: [VOTE] Apache Spark 3.0 RC2

Looks good to me, +1. The usual, sigs and licenses and artifacts look correct, no open issues for 3.0.0, and build and test works for me with most profiles enabled. My, there are 3397 JIRAs resolved for 3.0.0! (If a committer thinks an issue should block a release, at least target it to 3.0.0 and

Re: [VOTE] Apache Spark 3.0 RC2

Looks like the priority of SPARK-31706 [1] is incorrectly marked - it sounds like a blocker, as SPARK-26785 [2] / SPARK-26956 [3] dropped the feature of "update" on streaming output mode (as a result) and SPARK-31706 restores it. SPARK-31706 is not yet resolved, which may be valid reason to roll a