Re: [VOTE] Release Apache Spark 2.4.5 (RC1)

2020-01-16 Thread Xiao Li
-1 Let us include the correctness fix: https://github.com/apache/spark/pull/27229 Thanks, Xiao On Thu, Jan 16, 2020 at 8:46 AM Dongjoon Hyun wrote: > Thank you, Jungtaek! > > Bests, > Dongjoon. > > > On Wed, Jan 15, 2020 at 8:57 PM Jungtaek Lim > wrote: > >> Once we decided to cancel the

Re: [VOTE] Release Apache Spark 2.4.5 (RC1)

2020-01-16 Thread Dongjoon Hyun
Thank you, Jungtaek! Bests, Dongjoon. On Wed, Jan 15, 2020 at 8:57 PM Jungtaek Lim wrote: > Once we decided to cancel the RC1, what about including SPARK-29450 ( > https://github.com/apache/spark/pull/27209) into RC2? > > SPARK-29450 was merged into master, and Xiao figured out it fixed a >

Re: [VOTE] Release Apache Spark 2.4.5 (RC1)

2020-01-15 Thread Jungtaek Lim
Once we decided to cancel the RC1, what about including SPARK-29450 ( https://github.com/apache/spark/pull/27209) into RC2? SPARK-29450 was merged into master, and Xiao figured out it fixed a regression, long lasting one (broken at 2.3.0). The link refers the PR for 2.4 branch. Thanks, Jungtaek

Re: [VOTE] Release Apache Spark 2.4.5 (RC1)

2020-01-15 Thread Dongjoon Hyun
Sure. Wenchen and Hyukjin. I observed all of the above reported issues and have been waiting to collect more information before cancelling RC1 vote. The other stuff I've observed is that Marcelo and Sean also requested reverting the existing commit. - https://github.com/apache/spark/pull/24732

Re: [VOTE] Release Apache Spark 2.4.5 (RC1)

2020-01-15 Thread Hyukjin Kwon
If we go for RC2, we should include both: https://github.com/apache/spark/pull/27210 https://github.com/apache/spark/pull/27184 just for the sake of being complete and making the maintenance simple. 2020년 1월 16일 (목) 오후 12:38, Wenchen Fan 님이 작성: > Recently we merged several fixes to 2.4: >

Re: [VOTE] Release Apache Spark 2.4.5 (RC1)

2020-01-15 Thread Wenchen Fan
Recently we merged several fixes to 2.4: https://issues.apache.org/jira/browse/SPARK-30325 a driver hang issue https://issues.apache.org/jira/browse/SPARK-30246 a memory leak issue https://issues.apache.org/jira/browse/SPARK-29708 a correctness issue(for a rarely used feature, so not merged

Re: [VOTE] Release Apache Spark 2.4.5 (RC1)

2020-01-15 Thread Hyukjin Kwon
+1 On Wed, 15 Jan 2020, 08:24 Takeshi Yamamuro, wrote: > +1; > > I checked the links and materials, then I run the tests with > `-Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos -Pkubernetes > -Psparkr` > on macOS (Java 8). > All the things look fine and I didn't see the error on my env >

Re: [VOTE] Release Apache Spark 2.4.5 (RC1)

2020-01-14 Thread Takeshi Yamamuro
+1; I checked the links and materials, then I run the tests with `-Pyarn -Phadoop-2.7 -Phive -Phive-thriftserver -Pmesos -Pkubernetes -Psparkr` on macOS (Java 8). All the things look fine and I didn't see the error on my env that Sean said above. Thanks, Dongjoon! Bests, Takeshi On Wed, Jan

Re: [VOTE] Release Apache Spark 2.4.5 (RC1)

2020-01-14 Thread DB Tsai
+1 Thanks. Sincerely, DB Tsai -- Web: https://www.dbtsai.com PGP Key ID: 42E5B25A8F7A82C1 On Tue, Jan 14, 2020 at 11:08 AM Sean Owen wrote: > > Yeah it's something about the env I spun up, but I don't know what. It > happens frequently

Re: [VOTE] Release Apache Spark 2.4.5 (RC1)

2020-01-14 Thread Sean Owen
Yeah it's something about the env I spun up, but I don't know what. It happens frequently when I test, but not on Jenkins. The Kafka error comes up every now and then and a clean rebuild fixes it, but not in my case. I don't know why. But if nobody else sees it, I'm pretty sure it's just an

Re: [VOTE] Release Apache Spark 2.4.5 (RC1)

2020-01-14 Thread Sean Owen
+1 from me. I checked sigs/licenses, and built/tested from source on Java 8 + Ubuntu 18.04 with " -Pyarn -Phive -Phive-thriftserver -Phadoop-2.7 -Pmesos -Pkubernetes -Psparkr -Pkinesis-asl". I do get test failures, but, these are some I have always seen on Ubuntu, and I do not know why they

Re: [VOTE] Release Apache Spark 2.4.5 (RC1)

2020-01-13 Thread Dongjoon Hyun
+1. I verified with GPG and tested RC1 with the followings. - Profile: -Pyarn -Phadoop-2.7 -Pkubernetes -Pkinesis-asl -Phive -Phive-thriftserver - Java: OpenJDK 1.8.0_232 - OS: CentOS (7.5.1804) - All Scala/Java UTs and JDBC IT passed. - Test with Amazon EKS Client Version: v1.17.0

[VOTE] Release Apache Spark 2.4.5 (RC1)

2020-01-13 Thread Dongjoon Hyun
Please vote on releasing the following candidate as Apache Spark version 2.4.5. The vote is open until January 16th 5AM PST and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 2.4.5 [ ] -1 Do not release this package because