spark git commit: Revert SPARK-3039: Allow spark to be built using avro-mapred for hadoop2

2014-11-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.1 5c0032a47 - 45a01b6cd Revert SPARK-3039: Allow spark to be built using avro-mapred for hadoop2 This reverts commit 78887f94a0ae9cdcfb851910ab9c7d51a1ef2acb. Conflicts: pom.xml Project:

[2/2] spark git commit: Revert [maven-release-plugin] prepare release v1.1.1-rc1

2014-11-12 Thread andrewor14
Revert [maven-release-plugin] prepare release v1.1.1-rc1 This reverts commit f3e62ffa4ccea62911207b918ef1c23c1f50467f. Conflicts: pom.xml Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/4ac56796 Tree:

[1/2] spark git commit: Revert [maven-release-plugin] prepare for next development iteration

2014-11-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.1 45a01b6cd - 4ac56796b Revert [maven-release-plugin] prepare for next development iteration This reverts commit 5c0032a471d858fb010b1737ea14375f1af3ed88. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [maven-release-plugin] prepare release v1.1.1-rc1

2014-11-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.1 4ac56796b - 837deabeb [maven-release-plugin] prepare release v1.1.1-rc1 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/837deabe Tree:

spark git commit: [SPARK-4355][MLLIB] fix OnlineSummarizer.merge when other.mean is zero

2014-11-12 Thread meng
Repository: spark Updated Branches: refs/heads/master faeb41de2 - 84324fbcb [SPARK-4355][MLLIB] fix OnlineSummarizer.merge when other.mean is zero See inline comment about the bug. I also did some code clean-up. dbtsai I moved `update` to a private method of `MultivariateOnlineSummarizer`. I

spark git commit: [SPARK-4355][MLLIB] fix OnlineSummarizer.merge when other.mean is zero

2014-11-12 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.2 4a4cc7e91 - 14b933fb6 [SPARK-4355][MLLIB] fix OnlineSummarizer.merge when other.mean is zero See inline comment about the bug. I also did some code clean-up. dbtsai I moved `update` to a private method of

spark git commit: Revert [maven-release-plugin] prepare release v1.1.1-rc1

2014-11-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.1 837deabeb - 86c285c0c Revert [maven-release-plugin] prepare release v1.1.1-rc1 This reverts commit 837deabebf0714e3f3aca135d77169cc825824f3. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

Git Push Summary

2014-11-12 Thread andrewor14
Repository: spark Updated Tags: refs/tags/v1.1.1-rc1 [deleted] 490ffd35a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [Release] Log build output for each distribution

2014-11-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.1 86c285c0c - e3a5ee9a7 [Release] Log build output for each distribution Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/e3a5ee9a Tree:

[1/2] spark git commit: [SPARK-3530][MLLIB] pipeline and parameters with examples

2014-11-12 Thread meng
Repository: spark Updated Branches: refs/heads/master 84324fbcb - 4b736dbab http://git-wip-us.apache.org/repos/asf/spark/blob/4b736dba/mllib/src/test/java/org/apache/spark/ml/JavaPipelineSuite.java -- diff --git

[1/2] spark git commit: [SPARK-3530][MLLIB] pipeline and parameters with examples

2014-11-12 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.2 14b933fb6 - 38f9f2e1c http://git-wip-us.apache.org/repos/asf/spark/blob/38f9f2e1/mllib/src/main/scala/org/apache/spark/mllib/regression/LabeledPoint.scala -- diff --git

[2/2] spark git commit: [SPARK-3530][MLLIB] pipeline and parameters with examples

2014-11-12 Thread meng
[SPARK-3530][MLLIB] pipeline and parameters with examples This PR adds package org.apache.spark.ml with pipeline and parameters, as discussed on the JIRA. This is a joint work of jkbradley etrain shivaram and many others who helped on the design, also with help from marmbrus and liancheng on

[2/2] spark git commit: [SPARK-3530][MLLIB] pipeline and parameters with examples

2014-11-12 Thread meng
[SPARK-3530][MLLIB] pipeline and parameters with examples This PR adds package org.apache.spark.ml with pipeline and parameters, as discussed on the JIRA. This is a joint work of jkbradley etrain shivaram and many others who helped on the design, also with help from marmbrus and liancheng on

spark git commit: [maven-release-plugin] prepare release v1.1.1-rc1

2014-11-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.1 e3a5ee9a7 - 702930177 [maven-release-plugin] prepare release v1.1.1-rc1 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/70293017 Tree:

Git Push Summary

2014-11-12 Thread andrewor14
Repository: spark Updated Tags: refs/tags/v1.1.1-rc1 [created] 9cffba01e - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [maven-release-plugin] prepare for next development iteration

2014-11-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.1 702930177 - db22a9e2c [maven-release-plugin] prepare for next development iteration Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/db22a9e2 Tree:

Git Push Summary

2014-11-12 Thread andrewor14
Repository: spark Updated Tags: refs/tags/v1.1.1-rc1 [deleted] 9cffba01e - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[2/2] spark git commit: Revert [maven-release-plugin] prepare release v1.1.1-rc1

2014-11-12 Thread andrewor14
Revert [maven-release-plugin] prepare release v1.1.1-rc1 This reverts commit 7029301778895427216f2e0710c6e72a523c0897. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/8fe1c8cb Tree:

[1/2] spark git commit: Revert [maven-release-plugin] prepare for next development iteration

2014-11-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.1 db22a9e2c - 8fe1c8cb7 Revert [maven-release-plugin] prepare for next development iteration This reverts commit db22a9e2cb51eae2f8a79648ce3c6bf4fecdd641. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [maven-release-plugin] prepare release v1.1.1-rc1

2014-11-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.1 8fe1c8cb7 - 3f9e073ff [maven-release-plugin] prepare release v1.1.1-rc1 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/3f9e073f Tree:

spark git commit: [maven-release-plugin] prepare for next development iteration

2014-11-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.1 3f9e073ff - 6de888129 [maven-release-plugin] prepare for next development iteration Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/6de88812 Tree:

Git Push Summary

2014-11-12 Thread andrewor14
Repository: spark Updated Tags: refs/tags/v1.1.1-rc1 [created] d18bc77d3 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [SPARK-3660][STREAMING] Initial RDD for updateStateByKey transformation

2014-11-12 Thread tdas
Repository: spark Updated Branches: refs/heads/master 4b736dbab - 36ddeb7bf [SPARK-3660][STREAMING] Initial RDD for updateStateByKey transformation SPARK-3660 : Initial RDD for updateStateByKey transformation I have added a sample StatefulNetworkWordCountWithInitial inspired by

spark git commit: [Test] Better exception message from SparkSubmitSuite

2014-11-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 36ddeb7bf - 6e3c5a296 [Test] Better exception message from SparkSubmitSuite Before: ``` Exception in thread main java.lang.Exception: Could not load user defined classes inside of executors at

spark git commit: [Test] Better exception message from SparkSubmitSuite

2014-11-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.2 38f9f2e1c - 233f0377a [Test] Better exception message from SparkSubmitSuite Before: ``` Exception in thread main java.lang.Exception: Could not load user defined classes inside of executors at

spark git commit: [SPARK-4281][Build] Package Yarn shuffle service into its own jar

2014-11-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.2 233f0377a - f50c0881b [SPARK-4281][Build] Package Yarn shuffle service into its own jar This is another addendum to #3082, which added the Yarn shuffle service to run inside the NM. This PR makes the feature much more usable by

spark git commit: [SPARK-4281][Build] Package Yarn shuffle service into its own jar

2014-11-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 6e3c5a296 - aa43a8da0 [SPARK-4281][Build] Package Yarn shuffle service into its own jar This is another addendum to #3082, which added the Yarn shuffle service to run inside the NM. This PR makes the feature much more usable by packaging

spark git commit: Internal cleanup for aggregateMessages

2014-11-12 Thread rxin
Repository: spark Updated Branches: refs/heads/master aa43a8da0 - 0402be90f Internal cleanup for aggregateMessages 1. Add EdgeActiveness enum to represent activeness criteria more cleanly than using booleans. 2. Comments and whitespace. Author: Ankur Dave ankurd...@gmail.com Closes #3231

spark git commit: Internal cleanup for aggregateMessages

2014-11-12 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.2 f50c0881b - 5d5c8fd53 Internal cleanup for aggregateMessages 1. Add EdgeActiveness enum to represent activeness criteria more cleanly than using booleans. 2. Comments and whitespace. Author: Ankur Dave ankurd...@gmail.com Closes

spark git commit: [Release] Correct make-distribution.sh log path

2014-11-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 0402be90f - c3afd3266 [Release] Correct make-distribution.sh log path Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c3afd326 Tree:

spark git commit: [SPARK-3666] Extract interfaces for EdgeRDD and VertexRDD

2014-11-12 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.2 dbac77ebd - 127c19b44 [SPARK-3666] Extract interfaces for EdgeRDD and VertexRDD This discourages users from calling the VertexRDD and EdgeRDD constructor and makes it easier for future changes to ensure backward compatibility.

spark git commit: [SPARK-4369] [MLLib] fix TreeModel.predict() with RDD

2014-11-12 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.2 127c19b44 - 16da988c5 [SPARK-4369] [MLLib] fix TreeModel.predict() with RDD Fix TreeModel.predict() with RDD, added tests for it. (Also checked that other models don't have this issue) Author: Davies Liu dav...@databricks.com

spark git commit: [SPARK-4369] [MLLib] fix TreeModel.predict() with RDD

2014-11-12 Thread meng
Repository: spark Updated Branches: refs/heads/master a5ef58113 - bd86118c4 [SPARK-4369] [MLLib] fix TreeModel.predict() with RDD Fix TreeModel.predict() with RDD, added tests for it. (Also checked that other models don't have this issue) Author: Davies Liu dav...@databricks.com Closes

spark git commit: [SPARK-2672] support compressed file in wholeTextFile

2014-11-12 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.2 16da988c5 - 44f67ac7d [SPARK-2672] support compressed file in wholeTextFile The wholeFile() can not read compressed files, it should be, just like textFile(). Author: Davies Liu dav...@databricks.com Closes #3005 from davies/whole

spark git commit: [SPARK-2672] support compressed file in wholeTextFile

2014-11-12 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master bd86118c4 - d7d54a44e [SPARK-2672] support compressed file in wholeTextFile The wholeFile() can not read compressed files, it should be, just like textFile(). Author: Davies Liu dav...@databricks.com Closes #3005 from davies/whole and

spark git commit: [Release] Bring audit scripts up-to-date

2014-11-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.1 6de888129 - 88bc482d4 [Release] Bring audit scripts up-to-date This involves a few main changes: - Log all output message to the log file. Previously the log file was not useful because it did not indicate progress. - Remove

spark git commit: [Release] Bring audit scripts up-to-date

2014-11-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.2 44f67ac7d - 675df2afd [Release] Bring audit scripts up-to-date This involves a few main changes: - Log all output message to the log file. Previously the log file was not useful because it did not indicate progress. - Remove

spark git commit: [Release] Bring audit scripts up-to-date

2014-11-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master d7d54a44e - 723a86b04 [Release] Bring audit scripts up-to-date This involves a few main changes: - Log all output message to the log file. Previously the log file was not useful because it did not indicate progress. - Remove

spark git commit: [Release] Correct make-distribution.sh log path

2014-11-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.1 88bc482d4 - ba6d81d29 [Release] Correct make-distribution.sh log path Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/ba6d81d2 Tree:

spark git commit: [SPARK-4373][MLLIB] fix MLlib maven tests

2014-11-12 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.2 675df2afd - 5f14cdeaa [SPARK-4373][MLLIB] fix MLlib maven tests We want to make sure there is at most one spark context inside the same jvm. JoshRosen Author: Xiangrui Meng m...@databricks.com Closes #3235 from mengxr/SPARK-4373 and

Git Push Summary

2014-11-12 Thread andrewor14
Repository: spark Updated Tags: refs/tags/v1.1.1-rc1 [created] defa2f1a5 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[1/2] spark git commit: Revert [maven-release-plugin] prepare for next development iteration

2014-11-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.1 ba6d81d29 - 6f7b1bc4b Revert [maven-release-plugin] prepare for next development iteration This reverts commit 6de888129fcfe6e592458a4217fc66140747b54f. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

[2/2] spark git commit: Revert [maven-release-plugin] prepare release v1.1.1-rc1

2014-11-12 Thread andrewor14
Revert [maven-release-plugin] prepare release v1.1.1-rc1 This reverts commit 3f9e073ff0bb18b6079fda419d4e9dbf594545b0. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/6f7b1bc4 Tree:

Git Push Summary

2014-11-12 Thread andrewor14
Repository: spark Updated Tags: refs/tags/v1.1.1-rc1 [deleted] d18bc77d3 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [SPARK-4373][MLLIB] fix MLlib maven tests

2014-11-12 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 723a86b04 - 23f5bdf06 [SPARK-4373][MLLIB] fix MLlib maven tests We want to make sure there is at most one spark context inside the same jvm. JoshRosen Author: Xiangrui Meng m...@databricks.com Closes #3235 from mengxr/SPARK-4373 and

spark git commit: [SPARK-4370] [Core] Limit number of Netty cores based on executor size

2014-11-12 Thread rxin
Repository: spark Updated Branches: refs/heads/master 23f5bdf06 - b9e1c2eb9 [SPARK-4370] [Core] Limit number of Netty cores based on executor size Author: Aaron Davidson aa...@databricks.com Closes #3155 from aarondav/conf and squashes the following commits: 7045e77 [Aaron Davidson] Add

spark git commit: [SPARK-4370] [Core] Limit number of Netty cores based on executor size

2014-11-12 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.2 5f14cdeaa - 70efcd886 [SPARK-4370] [Core] Limit number of Netty cores based on executor size Author: Aaron Davidson aa...@databricks.com Closes #3155 from aarondav/conf and squashes the following commits: 7045e77 [Aaron Davidson] Add

spark git commit: [maven-release-plugin] prepare for next development iteration

2014-11-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.1 72a4fdbe8 - 685bdd2b7 [maven-release-plugin] prepare for next development iteration Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/685bdd2b Tree:

spark git commit: [SPARK-4256] Make Binary Evaluation Metrics functions defined in cases where there ar...

2014-11-12 Thread meng
Repository: spark Updated Branches: refs/heads/master b9e1c2eb9 - 484fecbf1 [SPARK-4256] Make Binary Evaluation Metrics functions defined in cases where there ar... ...e 0 positive or 0 negative examples. Author: Andrew Bullen andrew.bul...@workday.com Closes #3118 from abull/master and

spark git commit: [SPARK-4256] Make Binary Evaluation Metrics functions defined in cases where there ar...

2014-11-12 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.2 70efcd886 - ad872a5ba [SPARK-4256] Make Binary Evaluation Metrics functions defined in cases where there ar... ...e 0 positive or 0 negative examples. Author: Andrew Bullen andrew.bul...@workday.com Closes #3118 from abull/master