Re: [VOTE] Release Apache Spark 1.3.1 (RC3)
+1 from myself as well On Mon, Apr 13, 2015 at 8:35 PM, GuoQiang Li wi...@qq.com wrote: +1 (non-binding) -- Original -- From: Patrick Wendell;pwend...@gmail.com; Date: Sat, Apr 11, 2015 02:05 PM To: dev@spark.apache.orgdev@spark.apache.org; Subject: [VOTE] Release Apache Spark 1.3.1 (RC3) Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc2 (commit 3e83913): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=3e8391327ba586eaf54447043bd526d919043a44 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc3/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1088/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc3-docs/ The patches on top of RC2 are: [SPARK-6851] [SQL] Create new instance for each converted parquet relation [SPARK-5969] [PySpark] Fix descending pyspark.rdd.sortByKey. [SPARK-6343] Doc driver-worker network reqs [SPARK-6767] [SQL] Fixed Query DSL error in spark sql Readme [SPARK-6781] [SQL] use sqlContext in python shell [SPARK-6753] Clone SparkConf in ShuffleSuite tests [SPARK-6506] [PySpark] Do not try to retrieve SPARK_HOME when not needed... Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Tuesday, April 14, at 07:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1 (RC3)
+1builds - checktests - checkinstalls and sample run - check Thanking you. With Regards Sree On Friday, April 10, 2015 11:07 PM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc2 (commit 3e83913): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=3e8391327ba586eaf54447043bd526d919043a44 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc3/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1088/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc3-docs/ The patches on top of RC2 are: [SPARK-6851] [SQL] Create new instance for each converted parquet relation [SPARK-5969] [PySpark] Fix descending pyspark.rdd.sortByKey. [SPARK-6343] Doc driver-worker network reqs [SPARK-6767] [SQL] Fixed Query DSL error in spark sql Readme [SPARK-6781] [SQL] use sqlContext in python shell [SPARK-6753] Clone SparkConf in ShuffleSuite tests [SPARK-6506] [PySpark] Do not try to retrieve SPARK_HOME when not needed... Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Tuesday, April 14, at 07:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1 (RC3)
+1 (non-binding) Tested 2.6 build with standalone and yarn (no external shuffle service this time, although it does come up). On Fri, Apr 10, 2015 at 11:05 PM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc2 (commit 3e83913): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=3e8391327ba586eaf54447043bd526d919043a44 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc3/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1088/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc3-docs/ The patches on top of RC2 are: [SPARK-6851] [SQL] Create new instance for each converted parquet relation [SPARK-5969] [PySpark] Fix descending pyspark.rdd.sortByKey. [SPARK-6343] Doc driver-worker network reqs [SPARK-6767] [SQL] Fixed Query DSL error in spark sql Readme [SPARK-6781] [SQL] use sqlContext in python shell [SPARK-6753] Clone SparkConf in ShuffleSuite tests [SPARK-6506] [PySpark] Do not try to retrieve SPARK_HOME when not needed... Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Tuesday, April 14, at 07:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org -- Marcelo - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1 (RC3)
+1 On Fri, Apr 10, 2015 at 11:05 PM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc2 (commit 3e83913): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=3e8391327ba586eaf54447043bd526d919043a44 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc3/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1088/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc3-docs/ The patches on top of RC2 are: [SPARK-6851] [SQL] Create new instance for each converted parquet relation [SPARK-5969] [PySpark] Fix descending pyspark.rdd.sortByKey. [SPARK-6343] Doc driver-worker network reqs [SPARK-6767] [SQL] Fixed Query DSL error in spark sql Readme [SPARK-6781] [SQL] use sqlContext in python shell [SPARK-6753] Clone SparkConf in ShuffleSuite tests [SPARK-6506] [PySpark] Do not try to retrieve SPARK_HOME when not needed... Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Tuesday, April 14, at 07:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1 (RC3)
+1 (non-binding) On Sat, Apr 11, 2015 at 11:48 AM Krishna Sankar ksanka...@gmail.com wrote: +1. All tests OK (same as RC2) Cheers k/ On Fri, Apr 10, 2015 at 11:05 PM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc2 (commit 3e83913): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=3e8391327ba586eaf54447043bd526d919043a44 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc3/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1088/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc3-docs/ The patches on top of RC2 are: [SPARK-6851] [SQL] Create new instance for each converted parquet relation [SPARK-5969] [PySpark] Fix descending pyspark.rdd.sortByKey. [SPARK-6343] Doc driver-worker network reqs [SPARK-6767] [SQL] Fixed Query DSL error in spark sql Readme [SPARK-6781] [SQL] use sqlContext in python shell [SPARK-6753] Clone SparkConf in ShuffleSuite tests [SPARK-6506] [PySpark] Do not try to retrieve SPARK_HOME when not needed... Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Tuesday, April 14, at 07:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1 (RC3)
+1 On Fri, Apr 10, 2015 at 11:07 PM -0700, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc2 (commit 3e83913): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=3e8391327ba586eaf54447043bd526d919043a44 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc3/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1088/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc3-docs/ The patches on top of RC2 are: [SPARK-6851] [SQL] Create new instance for each converted parquet relation [SPARK-5969] [PySpark] Fix descending pyspark.rdd.sortByKey. [SPARK-6343] Doc driver-worker network reqs [SPARK-6767] [SQL] Fixed Query DSL error in spark sql Readme [SPARK-6781] [SQL] use sqlContext in python shell [SPARK-6753] Clone SparkConf in ShuffleSuite tests [SPARK-6506] [PySpark] Do not try to retrieve SPARK_HOME when not needed... Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Tuesday, April 14, at 07:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1 (RC3)
+1 same result as last time. On Sat, Apr 11, 2015 at 7:05 AM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc2 (commit 3e83913): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=3e8391327ba586eaf54447043bd526d919043a44 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc3/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1088/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc3-docs/ The patches on top of RC2 are: [SPARK-6851] [SQL] Create new instance for each converted parquet relation [SPARK-5969] [PySpark] Fix descending pyspark.rdd.sortByKey. [SPARK-6343] Doc driver-worker network reqs [SPARK-6767] [SQL] Fixed Query DSL error in spark sql Readme [SPARK-6781] [SQL] use sqlContext in python shell [SPARK-6753] Clone SparkConf in ShuffleSuite tests [SPARK-6506] [PySpark] Do not try to retrieve SPARK_HOME when not needed... Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Tuesday, April 14, at 07:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1 (RC2)
-1 (binding) We just were alerted to a pretty serious regression since 1.3.0 ( https://issues.apache.org/jira/browse/SPARK-6851). Should have a fix shortly. Michael On Fri, Apr 10, 2015 at 6:10 AM, Corey Nolet cjno...@gmail.com wrote: +1 (non-binding) - Verified signatures - built on Mac OSX - built on Fedora 21 All builds were done using profiles: hive, hive-thriftserver, hadoop-2.4, yarn +1 tested ML-related items on Mac OS X On Wed, Apr 8, 2015 at 7:59 PM, Krishna Sankar ksanka...@gmail.com wrote: +1 (non-binding, of course) 1. Compiled OSX 10.10 (Yosemite) OK Total time: 14:16 min mvn clean package -Pyarn -Dyarn.version=2.6.0 -Phadoop-2.4 -Dhadoop.version=2.6.0 -Phive -DskipTests -Dscala-2.11 2. Tested pyspark, mlib - running as well as compare results with 1.3.0 pyspark works well with the new iPython 3.0.0 release 2.1. statistics (min,max,mean,Pearson,Spearman) OK 2.2. Linear/Ridge/Laso Regression OK 2.3. Decision Tree, Naive Bayes OK 2.4. KMeans OK Center And Scale OK 2.5. RDD operations OK State of the Union Texts - MapReduce, Filter,sortByKey (word count) 2.6. Recommendation (Movielens medium dataset ~1 M ratings) OK Model evaluation/optimization (rank, numIter, lambda) with itertools OK 3. Scala - MLlib 3.1. statistics (min,max,mean,Pearson,Spearman) OK 3.2. LinearRegressionWithSGD OK 3.3. Decision Tree OK 3.4. KMeans OK 3.5. Recommendation (Movielens medium dataset ~1 M ratings) OK 4.0. Spark SQL from Python OK 4.1. result = sqlContext.sql(SELECT * from people WHERE State = 'WA') OK On Tue, Apr 7, 2015 at 10:46 PM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc2 (commit 7c4473a): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=7c4473aa5a7f5de0323394aaedeefbf9738e8eb5 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1083/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2-docs/ The patches on top of RC1 are: [SPARK-6737] Fix memory leak in OutputCommitCoordinator https://github.com/apache/spark/pull/5397 [SPARK-6636] Use public DNS hostname everywhere in spark_ec2.py https://github.com/apache/spark/pull/5302 [SPARK-6205] [CORE] UISeleniumSuite fails for Hadoop 2.x test with NoClassDefFoundError https://github.com/apache/spark/pull/4933 Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Saturday, April 11, at 07:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1 (RC2)
+1 tested on OS X Sean On Apr 7, 2015, at 11:46 PM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc2 (commit 7c4473a): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=7c4473aa5a7f5de0323394aaedeefbf9738e8eb5 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1083/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2-docs/ The patches on top of RC1 are: [SPARK-6737] Fix memory leak in OutputCommitCoordinator https://github.com/apache/spark/pull/5397 [SPARK-6636] Use public DNS hostname everywhere in spark_ec2.py https://github.com/apache/spark/pull/5302 [SPARK-6205] [CORE] UISeleniumSuite fails for Hadoop 2.x test with NoClassDefFoundError https://github.com/apache/spark/pull/4933 Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Saturday, April 11, at 07:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1 (RC2)
Still a +1 from me; same result (except that now of course the UISeleniumSuite test does not fail) On Wed, Apr 8, 2015 at 1:46 AM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc2 (commit 7c4473a): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=7c4473aa5a7f5de0323394aaedeefbf9738e8eb5 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1083/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2-docs/ The patches on top of RC1 are: [SPARK-6737] Fix memory leak in OutputCommitCoordinator https://github.com/apache/spark/pull/5397 [SPARK-6636] Use public DNS hostname everywhere in spark_ec2.py https://github.com/apache/spark/pull/5302 [SPARK-6205] [CORE] UISeleniumSuite fails for Hadoop 2.x test with NoClassDefFoundError https://github.com/apache/spark/pull/4933 Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Saturday, April 11, at 07:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1 (RC2)
The RC2 bits are lacking Hadoop 2.4 and Hadoop 2.6 - was that intended (they were included in RC1)? On Wed, Apr 8, 2015 at 9:01 AM Tom Graves tgraves...@yahoo.com.invalid wrote: +1. Tested spark on yarn against hadoop 2.6. Tom On Wednesday, April 8, 2015 6:15 AM, Sean Owen so...@cloudera.com wrote: Still a +1 from me; same result (except that now of course the UISeleniumSuite test does not fail) On Wed, Apr 8, 2015 at 1:46 AM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc2 (commit 7c4473a): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h= 7c4473aa5a7f5de0323394aaedeefbf9738e8eb5 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1083/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2-docs/ The patches on top of RC1 are: [SPARK-6737] Fix memory leak in OutputCommitCoordinator https://github.com/apache/spark/pull/5397 [SPARK-6636] Use public DNS hostname everywhere in spark_ec2.py https://github.com/apache/spark/pull/5302 [SPARK-6205] [CORE] UISeleniumSuite fails for Hadoop 2.x test with NoClassDefFoundError https://github.com/apache/spark/pull/4933 Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Saturday, April 11, at 07:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1 (RC2)
+1. Tested on Mac OS X and verified that some of the bugs were fixed. Matei On Apr 8, 2015, at 7:13 AM, Sean Owen so...@cloudera.com wrote: Still a +1 from me; same result (except that now of course the UISeleniumSuite test does not fail) On Wed, Apr 8, 2015 at 1:46 AM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc2 (commit 7c4473a): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=7c4473aa5a7f5de0323394aaedeefbf9738e8eb5 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1083/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2-docs/ The patches on top of RC1 are: [SPARK-6737] Fix memory leak in OutputCommitCoordinator https://github.com/apache/spark/pull/5397 [SPARK-6636] Use public DNS hostname everywhere in spark_ec2.py https://github.com/apache/spark/pull/5302 [SPARK-6205] [CORE] UISeleniumSuite fails for Hadoop 2.x test with NoClassDefFoundError https://github.com/apache/spark/pull/4933 Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Saturday, April 11, at 07:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1 (RC2)
+1. Tested spark on yarn against hadoop 2.6. Tom On Wednesday, April 8, 2015 6:15 AM, Sean Owen so...@cloudera.com wrote: Still a +1 from me; same result (except that now of course the UISeleniumSuite test does not fail) On Wed, Apr 8, 2015 at 1:46 AM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc2 (commit 7c4473a): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=7c4473aa5a7f5de0323394aaedeefbf9738e8eb5 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1083/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2-docs/ The patches on top of RC1 are: [SPARK-6737] Fix memory leak in OutputCommitCoordinator https://github.com/apache/spark/pull/5397 [SPARK-6636] Use public DNS hostname everywhere in spark_ec2.py https://github.com/apache/spark/pull/5302 [SPARK-6205] [CORE] UISeleniumSuite fails for Hadoop 2.x test with NoClassDefFoundError https://github.com/apache/spark/pull/4933 Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Saturday, April 11, at 07:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1 (RC2)
+1 Tested on 4 nodes Mesos cluster with fine-grain and coarse-grain mode. Tim On Wed, Apr 8, 2015 at 9:32 AM, Denny Lee denny.g@gmail.com wrote: The RC2 bits are lacking Hadoop 2.4 and Hadoop 2.6 - was that intended (they were included in RC1)? On Wed, Apr 8, 2015 at 9:01 AM Tom Graves tgraves...@yahoo.com.invalid wrote: +1. Tested spark on yarn against hadoop 2.6. Tom On Wednesday, April 8, 2015 6:15 AM, Sean Owen so...@cloudera.com wrote: Still a +1 from me; same result (except that now of course the UISeleniumSuite test does not fail) On Wed, Apr 8, 2015 at 1:46 AM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc2 (commit 7c4473a): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h= 7c4473aa5a7f5de0323394aaedeefbf9738e8eb5 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1083/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2-docs/ The patches on top of RC1 are: [SPARK-6737] Fix memory leak in OutputCommitCoordinator https://github.com/apache/spark/pull/5397 [SPARK-6636] Use public DNS hostname everywhere in spark_ec2.py https://github.com/apache/spark/pull/5302 [SPARK-6205] [CORE] UISeleniumSuite fails for Hadoop 2.x test with NoClassDefFoundError https://github.com/apache/spark/pull/4933 Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Saturday, April 11, at 07:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1 (RC2)
Hey Denny, I beleive the 2.4 bits are there. The 2.6 bits I had done specially (we haven't merge that into our upstream build script). I'll do it again now for RC2. - Patrick On Wed, Apr 8, 2015 at 1:53 PM, Timothy Chen tnac...@gmail.com wrote: +1 Tested on 4 nodes Mesos cluster with fine-grain and coarse-grain mode. Tim On Wed, Apr 8, 2015 at 9:32 AM, Denny Lee denny.g@gmail.com wrote: The RC2 bits are lacking Hadoop 2.4 and Hadoop 2.6 - was that intended (they were included in RC1)? On Wed, Apr 8, 2015 at 9:01 AM Tom Graves tgraves...@yahoo.com.invalid wrote: +1. Tested spark on yarn against hadoop 2.6. Tom On Wednesday, April 8, 2015 6:15 AM, Sean Owen so...@cloudera.com wrote: Still a +1 from me; same result (except that now of course the UISeleniumSuite test does not fail) On Wed, Apr 8, 2015 at 1:46 AM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc2 (commit 7c4473a): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h= 7c4473aa5a7f5de0323394aaedeefbf9738e8eb5 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1083/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2-docs/ The patches on top of RC1 are: [SPARK-6737] Fix memory leak in OutputCommitCoordinator https://github.com/apache/spark/pull/5397 [SPARK-6636] Use public DNS hostname everywhere in spark_ec2.py https://github.com/apache/spark/pull/5302 [SPARK-6205] [CORE] UISeleniumSuite fails for Hadoop 2.x test with NoClassDefFoundError https://github.com/apache/spark/pull/4933 Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Saturday, April 11, at 07:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1 (RC2)
Oh, it appears the 2.4 bits without hive are there but not the 2.4 bits with hive. Cool stuff on the 2.6. On Wed, Apr 8, 2015 at 12:30 Patrick Wendell pwend...@gmail.com wrote: Hey Denny, I beleive the 2.4 bits are there. The 2.6 bits I had done specially (we haven't merge that into our upstream build script). I'll do it again now for RC2. - Patrick On Wed, Apr 8, 2015 at 1:53 PM, Timothy Chen tnac...@gmail.com wrote: +1 Tested on 4 nodes Mesos cluster with fine-grain and coarse-grain mode. Tim On Wed, Apr 8, 2015 at 9:32 AM, Denny Lee denny.g@gmail.com wrote: The RC2 bits are lacking Hadoop 2.4 and Hadoop 2.6 - was that intended (they were included in RC1)? On Wed, Apr 8, 2015 at 9:01 AM Tom Graves tgraves...@yahoo.com.invalid wrote: +1. Tested spark on yarn against hadoop 2.6. Tom On Wednesday, April 8, 2015 6:15 AM, Sean Owen so...@cloudera.com wrote: Still a +1 from me; same result (except that now of course the UISeleniumSuite test does not fail) On Wed, Apr 8, 2015 at 1:46 AM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc2 (commit 7c4473a): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h= 7c4473aa5a7f5de0323394aaedeefbf9738e8eb5 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/ orgapachespark-1083/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2-docs/ The patches on top of RC1 are: [SPARK-6737] Fix memory leak in OutputCommitCoordinator https://github.com/apache/spark/pull/5397 [SPARK-6636] Use public DNS hostname everywhere in spark_ec2.py https://github.com/apache/spark/pull/5302 [SPARK-6205] [CORE] UISeleniumSuite fails for Hadoop 2.x test with NoClassDefFoundError https://github.com/apache/spark/pull/4933 Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Saturday, April 11, at 07:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1 (RC2)
Oh I see - ah okay I'm guessing it was a transient build error and I'll get it posted ASAP. On Wed, Apr 8, 2015 at 3:41 PM, Denny Lee denny.g@gmail.com wrote: Oh, it appears the 2.4 bits without hive are there but not the 2.4 bits with hive. Cool stuff on the 2.6. On Wed, Apr 8, 2015 at 12:30 Patrick Wendell pwend...@gmail.com wrote: Hey Denny, I beleive the 2.4 bits are there. The 2.6 bits I had done specially (we haven't merge that into our upstream build script). I'll do it again now for RC2. - Patrick On Wed, Apr 8, 2015 at 1:53 PM, Timothy Chen tnac...@gmail.com wrote: +1 Tested on 4 nodes Mesos cluster with fine-grain and coarse-grain mode. Tim On Wed, Apr 8, 2015 at 9:32 AM, Denny Lee denny.g@gmail.com wrote: The RC2 bits are lacking Hadoop 2.4 and Hadoop 2.6 - was that intended (they were included in RC1)? On Wed, Apr 8, 2015 at 9:01 AM Tom Graves tgraves...@yahoo.com.invalid wrote: +1. Tested spark on yarn against hadoop 2.6. Tom On Wednesday, April 8, 2015 6:15 AM, Sean Owen so...@cloudera.com wrote: Still a +1 from me; same result (except that now of course the UISeleniumSuite test does not fail) On Wed, Apr 8, 2015 at 1:46 AM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc2 (commit 7c4473a): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h= 7c4473aa5a7f5de0323394aaedeefbf9738e8eb5 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1083/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2-docs/ The patches on top of RC1 are: [SPARK-6737] Fix memory leak in OutputCommitCoordinator https://github.com/apache/spark/pull/5397 [SPARK-6636] Use public DNS hostname everywhere in spark_ec2.py https://github.com/apache/spark/pull/5302 [SPARK-6205] [CORE] UISeleniumSuite fails for Hadoop 2.x test with NoClassDefFoundError https://github.com/apache/spark/pull/4933 Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Saturday, April 11, at 07:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1 (RC2)
+1 (non-binding) Tested Scala, SparkSQL, and MLLib on OSX against Hadoop 2.6 On Wed, Apr 8, 2015 at 5:35 PM Joseph Bradley jos...@databricks.com wrote: +1 tested ML-related items on Mac OS X On Wed, Apr 8, 2015 at 7:59 PM, Krishna Sankar ksanka...@gmail.com wrote: +1 (non-binding, of course) 1. Compiled OSX 10.10 (Yosemite) OK Total time: 14:16 min mvn clean package -Pyarn -Dyarn.version=2.6.0 -Phadoop-2.4 -Dhadoop.version=2.6.0 -Phive -DskipTests -Dscala-2.11 2. Tested pyspark, mlib - running as well as compare results with 1.3.0 pyspark works well with the new iPython 3.0.0 release 2.1. statistics (min,max,mean,Pearson,Spearman) OK 2.2. Linear/Ridge/Laso Regression OK 2.3. Decision Tree, Naive Bayes OK 2.4. KMeans OK Center And Scale OK 2.5. RDD operations OK State of the Union Texts - MapReduce, Filter,sortByKey (word count) 2.6. Recommendation (Movielens medium dataset ~1 M ratings) OK Model evaluation/optimization (rank, numIter, lambda) with itertools OK 3. Scala - MLlib 3.1. statistics (min,max,mean,Pearson,Spearman) OK 3.2. LinearRegressionWithSGD OK 3.3. Decision Tree OK 3.4. KMeans OK 3.5. Recommendation (Movielens medium dataset ~1 M ratings) OK 4.0. Spark SQL from Python OK 4.1. result = sqlContext.sql(SELECT * from people WHERE State = 'WA') OK On Tue, Apr 7, 2015 at 10:46 PM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc2 (commit 7c4473a): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h= 7c4473aa5a7f5de0323394aaedeefbf9738e8eb5 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/ orgapachespark-1083/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2-docs/ The patches on top of RC1 are: [SPARK-6737] Fix memory leak in OutputCommitCoordinator https://github.com/apache/spark/pull/5397 [SPARK-6636] Use public DNS hostname everywhere in spark_ec2.py https://github.com/apache/spark/pull/5302 [SPARK-6205] [CORE] UISeleniumSuite fails for Hadoop 2.x test with NoClassDefFoundError https://github.com/apache/spark/pull/4933 Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Saturday, April 11, at 07:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1 (RC2)
+1 Built against Hadoop 2.6 and ran some jobs against a pseudo-distributed YARN cluster. -Sandy On Wed, Apr 8, 2015 at 12:49 PM, Patrick Wendell pwend...@gmail.com wrote: Oh I see - ah okay I'm guessing it was a transient build error and I'll get it posted ASAP. On Wed, Apr 8, 2015 at 3:41 PM, Denny Lee denny.g@gmail.com wrote: Oh, it appears the 2.4 bits without hive are there but not the 2.4 bits with hive. Cool stuff on the 2.6. On Wed, Apr 8, 2015 at 12:30 Patrick Wendell pwend...@gmail.com wrote: Hey Denny, I beleive the 2.4 bits are there. The 2.6 bits I had done specially (we haven't merge that into our upstream build script). I'll do it again now for RC2. - Patrick On Wed, Apr 8, 2015 at 1:53 PM, Timothy Chen tnac...@gmail.com wrote: +1 Tested on 4 nodes Mesos cluster with fine-grain and coarse-grain mode. Tim On Wed, Apr 8, 2015 at 9:32 AM, Denny Lee denny.g@gmail.com wrote: The RC2 bits are lacking Hadoop 2.4 and Hadoop 2.6 - was that intended (they were included in RC1)? On Wed, Apr 8, 2015 at 9:01 AM Tom Graves tgraves...@yahoo.com.invalid wrote: +1. Tested spark on yarn against hadoop 2.6. Tom On Wednesday, April 8, 2015 6:15 AM, Sean Owen so...@cloudera.com wrote: Still a +1 from me; same result (except that now of course the UISeleniumSuite test does not fail) On Wed, Apr 8, 2015 at 1:46 AM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc2 (commit 7c4473a): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h= 7c4473aa5a7f5de0323394aaedeefbf9738e8eb5 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1083/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2-docs/ The patches on top of RC1 are: [SPARK-6737] Fix memory leak in OutputCommitCoordinator https://github.com/apache/spark/pull/5397 [SPARK-6636] Use public DNS hostname everywhere in spark_ec2.py https://github.com/apache/spark/pull/5302 [SPARK-6205] [CORE] UISeleniumSuite fails for Hadoop 2.x test with NoClassDefFoundError https://github.com/apache/spark/pull/4933 Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Saturday, April 11, at 07:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1 (RC2)
+1 tested ML-related items on Mac OS X On Wed, Apr 8, 2015 at 7:59 PM, Krishna Sankar ksanka...@gmail.com wrote: +1 (non-binding, of course) 1. Compiled OSX 10.10 (Yosemite) OK Total time: 14:16 min mvn clean package -Pyarn -Dyarn.version=2.6.0 -Phadoop-2.4 -Dhadoop.version=2.6.0 -Phive -DskipTests -Dscala-2.11 2. Tested pyspark, mlib - running as well as compare results with 1.3.0 pyspark works well with the new iPython 3.0.0 release 2.1. statistics (min,max,mean,Pearson,Spearman) OK 2.2. Linear/Ridge/Laso Regression OK 2.3. Decision Tree, Naive Bayes OK 2.4. KMeans OK Center And Scale OK 2.5. RDD operations OK State of the Union Texts - MapReduce, Filter,sortByKey (word count) 2.6. Recommendation (Movielens medium dataset ~1 M ratings) OK Model evaluation/optimization (rank, numIter, lambda) with itertools OK 3. Scala - MLlib 3.1. statistics (min,max,mean,Pearson,Spearman) OK 3.2. LinearRegressionWithSGD OK 3.3. Decision Tree OK 3.4. KMeans OK 3.5. Recommendation (Movielens medium dataset ~1 M ratings) OK 4.0. Spark SQL from Python OK 4.1. result = sqlContext.sql(SELECT * from people WHERE State = 'WA') OK On Tue, Apr 7, 2015 at 10:46 PM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc2 (commit 7c4473a): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=7c4473aa5a7f5de0323394aaedeefbf9738e8eb5 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1083/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2-docs/ The patches on top of RC1 are: [SPARK-6737] Fix memory leak in OutputCommitCoordinator https://github.com/apache/spark/pull/5397 [SPARK-6636] Use public DNS hostname everywhere in spark_ec2.py https://github.com/apache/spark/pull/5302 [SPARK-6205] [CORE] UISeleniumSuite fails for Hadoop 2.x test with NoClassDefFoundError https://github.com/apache/spark/pull/4933 Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Saturday, April 11, at 07:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1 (RC2)
+1 (non-binding, of course) 1. Compiled OSX 10.10 (Yosemite) OK Total time: 14:16 min mvn clean package -Pyarn -Dyarn.version=2.6.0 -Phadoop-2.4 -Dhadoop.version=2.6.0 -Phive -DskipTests -Dscala-2.11 2. Tested pyspark, mlib - running as well as compare results with 1.3.0 pyspark works well with the new iPython 3.0.0 release 2.1. statistics (min,max,mean,Pearson,Spearman) OK 2.2. Linear/Ridge/Laso Regression OK 2.3. Decision Tree, Naive Bayes OK 2.4. KMeans OK Center And Scale OK 2.5. RDD operations OK State of the Union Texts - MapReduce, Filter,sortByKey (word count) 2.6. Recommendation (Movielens medium dataset ~1 M ratings) OK Model evaluation/optimization (rank, numIter, lambda) with itertools OK 3. Scala - MLlib 3.1. statistics (min,max,mean,Pearson,Spearman) OK 3.2. LinearRegressionWithSGD OK 3.3. Decision Tree OK 3.4. KMeans OK 3.5. Recommendation (Movielens medium dataset ~1 M ratings) OK 4.0. Spark SQL from Python OK 4.1. result = sqlContext.sql(SELECT * from people WHERE State = 'WA') OK On Tue, Apr 7, 2015 at 10:46 PM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc2 (commit 7c4473a): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=7c4473aa5a7f5de0323394aaedeefbf9738e8eb5 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1083/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc2-docs/ The patches on top of RC1 are: [SPARK-6737] Fix memory leak in OutputCommitCoordinator https://github.com/apache/spark/pull/5397 [SPARK-6636] Use public DNS hostname everywhere in spark_ec2.py https://github.com/apache/spark/pull/5302 [SPARK-6205] [CORE] UISeleniumSuite fails for Hadoop 2.x test with NoClassDefFoundError https://github.com/apache/spark/pull/4933 Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Saturday, April 11, at 07:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1
+1 (non-binding) Ran standalone and yarn tests on the hadoop-2.6 tarball, with and without the external shuffle service in yarn mode. On Sat, Apr 4, 2015 at 5:09 PM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc1 (commit 0dcb5d9f): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=0dcb5d9f31b713ed90bcec63ebc4e530cbb69851 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc1/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1080 The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc1-docs/ Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Wednesday, April 08, at 01:10 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - Patrick - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org -- Marcelo - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1
Hey All, Today SPARK-6737 came to my attention. This is a bug that causes a memory leak for any long running program that repeatedly saves data out to a Hadoop FileSystem. For that reason, it is problematic for Spark Streaming. My sense is that this is severe enough to cut another RC once the fix is merged (which is imminent): https://issues.apache.org/jira/browse/SPARK-6737 I'll leave a bit of time for others to comment, in particular if people feel we should not wait for this fix. - Patrick On Tue, Apr 7, 2015 at 2:34 PM, Marcelo Vanzin van...@cloudera.com wrote: +1 (non-binding) Ran standalone and yarn tests on the hadoop-2.6 tarball, with and without the external shuffle service in yarn mode. On Sat, Apr 4, 2015 at 5:09 PM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc1 (commit 0dcb5d9f): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=0dcb5d9f31b713ed90bcec63ebc4e530cbb69851 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc1/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1080 The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc1-docs/ Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Wednesday, April 08, at 01:10 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - Patrick - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org -- Marcelo - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1
Ah, ok. It was missing in the list of jiras. So +1. Thanks, Hari On Mon, Apr 6, 2015 at 11:36 AM, Patrick Wendell pwend...@gmail.com wrote: I believe TD just forgot to set the fix version on the JIRA. There is a fix for this in 1.3: https://github.com/apache/spark/commit/03e263f5b527cf574f4ffcd5cd886f7723e3756e - Patrick On Mon, Apr 6, 2015 at 2:31 PM, Mark Hamstra m...@clearstorydata.com wrote: Is that correct, or is the JIRA just out of sync, since TD's PR was merged? https://github.com/apache/spark/pull/5008 On Mon, Apr 6, 2015 at 11:10 AM, Hari Shreedharan hshreedha...@cloudera.com wrote: It does not look like https://issues.apache.org/jira/browse/SPARK-6222 made it. It was targeted towards this release. Thanks, Hari On Mon, Apr 6, 2015 at 11:04 AM, York, Brennon brennon.y...@capitalone.com wrote: +1 (non-binding) Tested GraphX, build infrastructure, core test suite on OSX 10.9 w/ Java 1.7/1.8 On 4/6/15, 5:21 AM, Sean Owen so...@cloudera.com wrote: SPARK-6673 is not, in the end, relevant for 1.3.x I believe; we just resolved it for 1.4 anyway. False alarm there. I back-ported SPARK-6205 into the 1.3 branch for next time. We'll pick it up if there's another RC, but by itself is not something that needs a new RC. (I will give the same treatment to branch 1.2 if needed in light of the 1.2.2 release.) I applied the simple change in SPARK-6205 in order to continue executing tests and all was well. I still see a few failures in Hive tests: - show_create_table_serde *** FAILED *** - show_tblproperties *** FAILED *** - udf_std *** FAILED *** - udf_stddev *** FAILED *** with ... mvn -Phadoop-2.4 -Pyarn -Phive -Phive-0.13.1 -Dhadoop.version=2.6.0 -DskipTests clean package; mvn -Phadoop-2.4 -Pyarn -Phive -Phive-0.13.1 -Dhadoop.version=2.6.0 test ... but these are not regressions from 1.3.0. +1 from me at this point on the current artifacts. On Sun, Apr 5, 2015 at 9:24 AM, Sean Owen so...@cloudera.com wrote: Signatures and hashes are good. LICENSE, NOTICE still check out. Compiles for a Hadoop 2.6 + YARN + Hive profile. I still see the UISeleniumSuite test failure observed in 1.3.0, which is minor and already fixed. I don't know why I didn't back-port it: https://issues.apache.org/jira/browse/SPARK-6205 If we roll another, let's get this easy fix in, but it is only an issue with tests. On JIRA, I checked open issues with Fix Version = 1.3.0 or 1.3.1 and all look legitimate (e.g. reopened or in progress) There is 1 open Blocker for 1.3.1 per Andrew: https://issues.apache.org/jira/browse/SPARK-6673 spark-shell.cmd can't start even when spark was built in Windows I believe this can be resolved quickly but as a matter of hygiene should be fixed or demoted before release. FYI there are 16 Critical issues marked for 1.3.0 / 1.3.1; worth examining before release to see how critical they are: SPARK-6701,Flaky test: o.a.s.deploy.yarn.YarnClusterSuite Python application,,Open,4/3/15 SPARK-6484,Ganglia metrics xml reporter doesn't escape correctly,Josh Rosen,Open,3/24/15 SPARK-6270,Standalone Master hangs when streaming job completes,,Open,3/11/15 SPARK-6209,ExecutorClassLoader can leak connections after failing to load classes from the REPL class server,Josh Rosen,In Progress,4/2/15 SPARK-5113,Audit and document use of hostnames and IP addresses in Spark,,Open,3/24/15 SPARK-5098,Number of running tasks become negative after tasks lost,,Open,1/14/15 SPARK-4925,Publish Spark SQL hive-thriftserver maven artifact,Patrick Wendell,Reopened,3/23/15 SPARK-4922,Support dynamic allocation for coarse-grained Mesos,,Open,3/31/15 SPARK-4888,Spark EC2 doesn't mount local disks for i2.8xlarge instances,,Open,1/27/15 SPARK-4879,Missing output partitions after job completes with speculative execution,Josh Rosen,Open,3/5/15 SPARK-4751,Support dynamic allocation for standalone mode,Andrew Or,Open,12/22/14 SPARK-4454,Race condition in DAGScheduler,Josh Rosen,Reopened,2/18/15 SPARK-4452,Shuffle data structures can starve others on the same thread for memory,Tianshuo Deng,Open,1/24/15 SPARK-4352,Incorporate locality preferences in dynamic allocation requests,,Open,1/26/15 SPARK-4227,Document external shuffle service,,Open,3/23/15 SPARK-3650,Triangle Count handles reverse edges incorrectly,,Open,2/23/15 On Sun, Apr 5, 2015 at 1:09 AM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc1 (commit 0dcb5d9f): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=0dcb5d9f3 1b713ed90bcec63ebc4e530cbb69851 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc1/ Release
Re: [VOTE] Release Apache Spark 1.3.1
+1 On Sat, Apr 4, 2015 at 5:09 PM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc1 (commit 0dcb5d9f): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=0dcb5d9f31b713ed90bcec63ebc4e530cbb69851 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc1/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1080 The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc1-docs/ Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Wednesday, April 08, at 01:10 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - Patrick - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1
I believe TD just forgot to set the fix version on the JIRA. There is a fix for this in 1.3: https://github.com/apache/spark/commit/03e263f5b527cf574f4ffcd5cd886f7723e3756e - Patrick On Mon, Apr 6, 2015 at 2:31 PM, Mark Hamstra m...@clearstorydata.com wrote: Is that correct, or is the JIRA just out of sync, since TD's PR was merged? https://github.com/apache/spark/pull/5008 On Mon, Apr 6, 2015 at 11:10 AM, Hari Shreedharan hshreedha...@cloudera.com wrote: It does not look like https://issues.apache.org/jira/browse/SPARK-6222 made it. It was targeted towards this release. Thanks, Hari On Mon, Apr 6, 2015 at 11:04 AM, York, Brennon brennon.y...@capitalone.com wrote: +1 (non-binding) Tested GraphX, build infrastructure, core test suite on OSX 10.9 w/ Java 1.7/1.8 On 4/6/15, 5:21 AM, Sean Owen so...@cloudera.com wrote: SPARK-6673 is not, in the end, relevant for 1.3.x I believe; we just resolved it for 1.4 anyway. False alarm there. I back-ported SPARK-6205 into the 1.3 branch for next time. We'll pick it up if there's another RC, but by itself is not something that needs a new RC. (I will give the same treatment to branch 1.2 if needed in light of the 1.2.2 release.) I applied the simple change in SPARK-6205 in order to continue executing tests and all was well. I still see a few failures in Hive tests: - show_create_table_serde *** FAILED *** - show_tblproperties *** FAILED *** - udf_std *** FAILED *** - udf_stddev *** FAILED *** with ... mvn -Phadoop-2.4 -Pyarn -Phive -Phive-0.13.1 -Dhadoop.version=2.6.0 -DskipTests clean package; mvn -Phadoop-2.4 -Pyarn -Phive -Phive-0.13.1 -Dhadoop.version=2.6.0 test ... but these are not regressions from 1.3.0. +1 from me at this point on the current artifacts. On Sun, Apr 5, 2015 at 9:24 AM, Sean Owen so...@cloudera.com wrote: Signatures and hashes are good. LICENSE, NOTICE still check out. Compiles for a Hadoop 2.6 + YARN + Hive profile. I still see the UISeleniumSuite test failure observed in 1.3.0, which is minor and already fixed. I don't know why I didn't back-port it: https://issues.apache.org/jira/browse/SPARK-6205 If we roll another, let's get this easy fix in, but it is only an issue with tests. On JIRA, I checked open issues with Fix Version = 1.3.0 or 1.3.1 and all look legitimate (e.g. reopened or in progress) There is 1 open Blocker for 1.3.1 per Andrew: https://issues.apache.org/jira/browse/SPARK-6673 spark-shell.cmd can't start even when spark was built in Windows I believe this can be resolved quickly but as a matter of hygiene should be fixed or demoted before release. FYI there are 16 Critical issues marked for 1.3.0 / 1.3.1; worth examining before release to see how critical they are: SPARK-6701,Flaky test: o.a.s.deploy.yarn.YarnClusterSuite Python application,,Open,4/3/15 SPARK-6484,Ganglia metrics xml reporter doesn't escape correctly,Josh Rosen,Open,3/24/15 SPARK-6270,Standalone Master hangs when streaming job completes,,Open,3/11/15 SPARK-6209,ExecutorClassLoader can leak connections after failing to load classes from the REPL class server,Josh Rosen,In Progress,4/2/15 SPARK-5113,Audit and document use of hostnames and IP addresses in Spark,,Open,3/24/15 SPARK-5098,Number of running tasks become negative after tasks lost,,Open,1/14/15 SPARK-4925,Publish Spark SQL hive-thriftserver maven artifact,Patrick Wendell,Reopened,3/23/15 SPARK-4922,Support dynamic allocation for coarse-grained Mesos,,Open,3/31/15 SPARK-4888,Spark EC2 doesn't mount local disks for i2.8xlarge instances,,Open,1/27/15 SPARK-4879,Missing output partitions after job completes with speculative execution,Josh Rosen,Open,3/5/15 SPARK-4751,Support dynamic allocation for standalone mode,Andrew Or,Open,12/22/14 SPARK-4454,Race condition in DAGScheduler,Josh Rosen,Reopened,2/18/15 SPARK-4452,Shuffle data structures can starve others on the same thread for memory,Tianshuo Deng,Open,1/24/15 SPARK-4352,Incorporate locality preferences in dynamic allocation requests,,Open,1/26/15 SPARK-4227,Document external shuffle service,,Open,3/23/15 SPARK-3650,Triangle Count handles reverse edges incorrectly,,Open,2/23/15 On Sun, Apr 5, 2015 at 1:09 AM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc1 (commit 0dcb5d9f): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=0dcb5d9f3 1b713ed90bcec63ebc4e530cbb69851 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc1/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be
Re: [VOTE] Release Apache Spark 1.3.1
+1 On Apr 4, 2015, at 6:11 PM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc1 (commit 0dcb5d9f): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=0dcb5d9f31b713ed90bcec63ebc4e530cbb69851 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc1/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1080 The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc1-docs/ Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Wednesday, April 08, at 01:10 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - Patrick - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1
SPARK-6673 is not, in the end, relevant for 1.3.x I believe; we just resolved it for 1.4 anyway. False alarm there. I back-ported SPARK-6205 into the 1.3 branch for next time. We'll pick it up if there's another RC, but by itself is not something that needs a new RC. (I will give the same treatment to branch 1.2 if needed in light of the 1.2.2 release.) I applied the simple change in SPARK-6205 in order to continue executing tests and all was well. I still see a few failures in Hive tests: - show_create_table_serde *** FAILED *** - show_tblproperties *** FAILED *** - udf_std *** FAILED *** - udf_stddev *** FAILED *** with ... mvn -Phadoop-2.4 -Pyarn -Phive -Phive-0.13.1 -Dhadoop.version=2.6.0 -DskipTests clean package; mvn -Phadoop-2.4 -Pyarn -Phive -Phive-0.13.1 -Dhadoop.version=2.6.0 test ... but these are not regressions from 1.3.0. +1 from me at this point on the current artifacts. On Sun, Apr 5, 2015 at 9:24 AM, Sean Owen so...@cloudera.com wrote: Signatures and hashes are good. LICENSE, NOTICE still check out. Compiles for a Hadoop 2.6 + YARN + Hive profile. I still see the UISeleniumSuite test failure observed in 1.3.0, which is minor and already fixed. I don't know why I didn't back-port it: https://issues.apache.org/jira/browse/SPARK-6205 If we roll another, let's get this easy fix in, but it is only an issue with tests. On JIRA, I checked open issues with Fix Version = 1.3.0 or 1.3.1 and all look legitimate (e.g. reopened or in progress) There is 1 open Blocker for 1.3.1 per Andrew: https://issues.apache.org/jira/browse/SPARK-6673 spark-shell.cmd can't start even when spark was built in Windows I believe this can be resolved quickly but as a matter of hygiene should be fixed or demoted before release. FYI there are 16 Critical issues marked for 1.3.0 / 1.3.1; worth examining before release to see how critical they are: SPARK-6701,Flaky test: o.a.s.deploy.yarn.YarnClusterSuite Python application,,Open,4/3/15 SPARK-6484,Ganglia metrics xml reporter doesn't escape correctly,Josh Rosen,Open,3/24/15 SPARK-6270,Standalone Master hangs when streaming job completes,,Open,3/11/15 SPARK-6209,ExecutorClassLoader can leak connections after failing to load classes from the REPL class server,Josh Rosen,In Progress,4/2/15 SPARK-5113,Audit and document use of hostnames and IP addresses in Spark,,Open,3/24/15 SPARK-5098,Number of running tasks become negative after tasks lost,,Open,1/14/15 SPARK-4925,Publish Spark SQL hive-thriftserver maven artifact,Patrick Wendell,Reopened,3/23/15 SPARK-4922,Support dynamic allocation for coarse-grained Mesos,,Open,3/31/15 SPARK-4888,Spark EC2 doesn't mount local disks for i2.8xlarge instances,,Open,1/27/15 SPARK-4879,Missing output partitions after job completes with speculative execution,Josh Rosen,Open,3/5/15 SPARK-4751,Support dynamic allocation for standalone mode,Andrew Or,Open,12/22/14 SPARK-4454,Race condition in DAGScheduler,Josh Rosen,Reopened,2/18/15 SPARK-4452,Shuffle data structures can starve others on the same thread for memory,Tianshuo Deng,Open,1/24/15 SPARK-4352,Incorporate locality preferences in dynamic allocation requests,,Open,1/26/15 SPARK-4227,Document external shuffle service,,Open,3/23/15 SPARK-3650,Triangle Count handles reverse edges incorrectly,,Open,2/23/15 On Sun, Apr 5, 2015 at 1:09 AM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc1 (commit 0dcb5d9f): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=0dcb5d9f31b713ed90bcec63ebc4e530cbb69851 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc1/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1080 The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc1-docs/ Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Wednesday, April 08, at 01:10 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - Patrick - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1
+1 Verified some MLlib bug fixes on OS X. -Xiangrui On Sun, Apr 5, 2015 at 1:24 AM, Sean Owen so...@cloudera.com wrote: Signatures and hashes are good. LICENSE, NOTICE still check out. Compiles for a Hadoop 2.6 + YARN + Hive profile. I still see the UISeleniumSuite test failure observed in 1.3.0, which is minor and already fixed. I don't know why I didn't back-port it: https://issues.apache.org/jira/browse/SPARK-6205 If we roll another, let's get this easy fix in, but it is only an issue with tests. On JIRA, I checked open issues with Fix Version = 1.3.0 or 1.3.1 and all look legitimate (e.g. reopened or in progress) There is 1 open Blocker for 1.3.1 per Andrew: https://issues.apache.org/jira/browse/SPARK-6673 spark-shell.cmd can't start even when spark was built in Windows I believe this can be resolved quickly but as a matter of hygiene should be fixed or demoted before release. FYI there are 16 Critical issues marked for 1.3.0 / 1.3.1; worth examining before release to see how critical they are: SPARK-6701,Flaky test: o.a.s.deploy.yarn.YarnClusterSuite Python application,,Open,4/3/15 SPARK-6484,Ganglia metrics xml reporter doesn't escape correctly,Josh Rosen,Open,3/24/15 SPARK-6270,Standalone Master hangs when streaming job completes,,Open,3/11/15 SPARK-6209,ExecutorClassLoader can leak connections after failing to load classes from the REPL class server,Josh Rosen,In Progress,4/2/15 SPARK-5113,Audit and document use of hostnames and IP addresses in Spark,,Open,3/24/15 SPARK-5098,Number of running tasks become negative after tasks lost,,Open,1/14/15 SPARK-4925,Publish Spark SQL hive-thriftserver maven artifact,Patrick Wendell,Reopened,3/23/15 SPARK-4922,Support dynamic allocation for coarse-grained Mesos,,Open,3/31/15 SPARK-4888,Spark EC2 doesn't mount local disks for i2.8xlarge instances,,Open,1/27/15 SPARK-4879,Missing output partitions after job completes with speculative execution,Josh Rosen,Open,3/5/15 SPARK-4751,Support dynamic allocation for standalone mode,Andrew Or,Open,12/22/14 SPARK-4454,Race condition in DAGScheduler,Josh Rosen,Reopened,2/18/15 SPARK-4452,Shuffle data structures can starve others on the same thread for memory,Tianshuo Deng,Open,1/24/15 SPARK-4352,Incorporate locality preferences in dynamic allocation requests,,Open,1/26/15 SPARK-4227,Document external shuffle service,,Open,3/23/15 SPARK-3650,Triangle Count handles reverse edges incorrectly,,Open,2/23/15 On Sun, Apr 5, 2015 at 1:09 AM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc1 (commit 0dcb5d9f): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=0dcb5d9f31b713ed90bcec63ebc4e530cbb69851 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc1/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1080 The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc1-docs/ Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Wednesday, April 08, at 01:10 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - Patrick - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1
+1 (non-binding) Verified various DataFrame functions, Hive integration, MLlib, etc. on OSX. On Sun, Apr 5, 2015 at 9:16 PM Xiangrui Meng men...@gmail.com wrote: +1 Verified some MLlib bug fixes on OS X. -Xiangrui On Sun, Apr 5, 2015 at 1:24 AM, Sean Owen so...@cloudera.com wrote: Signatures and hashes are good. LICENSE, NOTICE still check out. Compiles for a Hadoop 2.6 + YARN + Hive profile. I still see the UISeleniumSuite test failure observed in 1.3.0, which is minor and already fixed. I don't know why I didn't back-port it: https://issues.apache.org/jira/browse/SPARK-6205 If we roll another, let's get this easy fix in, but it is only an issue with tests. On JIRA, I checked open issues with Fix Version = 1.3.0 or 1.3.1 and all look legitimate (e.g. reopened or in progress) There is 1 open Blocker for 1.3.1 per Andrew: https://issues.apache.org/jira/browse/SPARK-6673 spark-shell.cmd can't start even when spark was built in Windows I believe this can be resolved quickly but as a matter of hygiene should be fixed or demoted before release. FYI there are 16 Critical issues marked for 1.3.0 / 1.3.1; worth examining before release to see how critical they are: SPARK-6701,Flaky test: o.a.s.deploy.yarn.YarnClusterSuite Python application,,Open,4/3/15 SPARK-6484,Ganglia metrics xml reporter doesn't escape correctly,Josh Rosen,Open,3/24/15 SPARK-6270,Standalone Master hangs when streaming job completes,,Open,3/11/15 SPARK-6209,ExecutorClassLoader can leak connections after failing to load classes from the REPL class server,Josh Rosen,In Progress,4/2/15 SPARK-5113,Audit and document use of hostnames and IP addresses in Spark,,Open,3/24/15 SPARK-5098,Number of running tasks become negative after tasks lost,,Open,1/14/15 SPARK-4925,Publish Spark SQL hive-thriftserver maven artifact,Patrick Wendell,Reopened,3/23/15 SPARK-4922,Support dynamic allocation for coarse-grained Mesos,,Open,3/31/15 SPARK-4888,Spark EC2 doesn't mount local disks for i2.8xlarge instances,,Open,1/27/15 SPARK-4879,Missing output partitions after job completes with speculative execution,Josh Rosen,Open,3/5/15 SPARK-4751,Support dynamic allocation for standalone mode,Andrew Or,Open,12/22/14 SPARK-4454,Race condition in DAGScheduler,Josh Rosen,Reopened,2/18/15 SPARK-4452,Shuffle data structures can starve others on the same thread for memory,Tianshuo Deng,Open,1/24/15 SPARK-4352,Incorporate locality preferences in dynamic allocation requests,,Open,1/26/15 SPARK-4227,Document external shuffle service,,Open,3/23/15 SPARK-3650,Triangle Count handles reverse edges incorrectly,,Open,2/23/15 On Sun, Apr 5, 2015 at 1:09 AM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc1 (commit 0dcb5d9f): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h= 0dcb5d9f31b713ed90bcec63ebc4e530cbb69851 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc1/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1080 The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc1-docs/ Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Wednesday, April 08, at 01:10 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - Patrick - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1
Signatures and hashes are good. LICENSE, NOTICE still check out. Compiles for a Hadoop 2.6 + YARN + Hive profile. I still see the UISeleniumSuite test failure observed in 1.3.0, which is minor and already fixed. I don't know why I didn't back-port it: https://issues.apache.org/jira/browse/SPARK-6205 If we roll another, let's get this easy fix in, but it is only an issue with tests. On JIRA, I checked open issues with Fix Version = 1.3.0 or 1.3.1 and all look legitimate (e.g. reopened or in progress) There is 1 open Blocker for 1.3.1 per Andrew: https://issues.apache.org/jira/browse/SPARK-6673 spark-shell.cmd can't start even when spark was built in Windows I believe this can be resolved quickly but as a matter of hygiene should be fixed or demoted before release. FYI there are 16 Critical issues marked for 1.3.0 / 1.3.1; worth examining before release to see how critical they are: SPARK-6701,Flaky test: o.a.s.deploy.yarn.YarnClusterSuite Python application,,Open,4/3/15 SPARK-6484,Ganglia metrics xml reporter doesn't escape correctly,Josh Rosen,Open,3/24/15 SPARK-6270,Standalone Master hangs when streaming job completes,,Open,3/11/15 SPARK-6209,ExecutorClassLoader can leak connections after failing to load classes from the REPL class server,Josh Rosen,In Progress,4/2/15 SPARK-5113,Audit and document use of hostnames and IP addresses in Spark,,Open,3/24/15 SPARK-5098,Number of running tasks become negative after tasks lost,,Open,1/14/15 SPARK-4925,Publish Spark SQL hive-thriftserver maven artifact,Patrick Wendell,Reopened,3/23/15 SPARK-4922,Support dynamic allocation for coarse-grained Mesos,,Open,3/31/15 SPARK-4888,Spark EC2 doesn't mount local disks for i2.8xlarge instances,,Open,1/27/15 SPARK-4879,Missing output partitions after job completes with speculative execution,Josh Rosen,Open,3/5/15 SPARK-4751,Support dynamic allocation for standalone mode,Andrew Or,Open,12/22/14 SPARK-4454,Race condition in DAGScheduler,Josh Rosen,Reopened,2/18/15 SPARK-4452,Shuffle data structures can starve others on the same thread for memory,Tianshuo Deng,Open,1/24/15 SPARK-4352,Incorporate locality preferences in dynamic allocation requests,,Open,1/26/15 SPARK-4227,Document external shuffle service,,Open,3/23/15 SPARK-3650,Triangle Count handles reverse edges incorrectly,,Open,2/23/15 On Sun, Apr 5, 2015 at 1:09 AM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc1 (commit 0dcb5d9f): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=0dcb5d9f31b713ed90bcec63ebc4e530cbb69851 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc1/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1080 The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc1-docs/ Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Wednesday, April 08, at 01:10 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - Patrick - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
Re: [VOTE] Release Apache Spark 1.3.1
+1 (non-binding, of course) 1. Compiled OSX 10.10 (Yosemite) OK Total time: 15:04 min mvn clean package -Pyarn -Dyarn.version=2.6.0 -Phadoop-2.4 -Dhadoop.version=2.6.0 -Phive -DskipTests -Dscala-2.11 2. Tested pyspark, mlib - running as well as compare results with 1.3.0 pyspark works well with the new iPython 3.0.0 release 2.1. statistics (min,max,mean,Pearson,Spearman) OK 2.2. Linear/Ridge/Laso Regression OK 2.3. Decision Tree, Naive Bayes OK 2.4. KMeans OK Center And Scale OK 2.5. RDD operations OK State of the Union Texts - MapReduce, Filter,sortByKey (word count) 2.6. Recommendation (Movielens medium dataset ~1 M ratings) OK Model evaluation/optimization (rank, numIter, lambda) with itertools OK On Sat, Apr 4, 2015 at 5:13 PM, Reynold Xin r...@databricks.com wrote: +1 Tested some DataFrame functions locally on Mac OS X. On Sat, Apr 4, 2015 at 5:09 PM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.3.1! The tag to be voted on is v1.3.1-rc1 (commit 0dcb5d9f): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=0dcb5d9f31b713ed90bcec63ebc4e530cbb69851 The list of fixes present in this release can be found at: http://bit.ly/1C2nVPY The release files, including signatures, digests, etc. can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc1/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-1080 The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-1.3.1-rc1-docs/ Please vote on releasing this package as Apache Spark 1.3.1! The vote is open until Wednesday, April 08, at 01:10 UTC and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 1.3.1 [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.apache.org/ - Patrick - To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
