spark git commit: [SPARK-7883] [DOCS] [MLLIB] Fixing broken trainImplicit Scala example in MLlib Collaborative Filtering documentation.

2015-05-26 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.0 0afb04250 - 86ad12d44 [SPARK-7883] [DOCS] [MLLIB] Fixing broken trainImplicit Scala example in MLlib Collaborative Filtering documentation. Fixing broken trainImplicit Scala example in MLlib Collaborative Filtering documentation to

spark git commit: [SPARK-7637] [SQL] O(N) merge implementation for StructType merge

2015-05-26 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 0463428b6 - 03668348e [SPARK-7637] [SQL] O(N) merge implementation for StructType merge Contribution is my original work and I license the work to the project under the projects open source license. Author: rowan

spark git commit: [SPARK-7883] [DOCS] [MLLIB] Fixing broken trainImplicit Scala example in MLlib Collaborative Filtering documentation.

2015-05-26 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.4 f9dfa4d0f - 311fcf67e [SPARK-7883] [DOCS] [MLLIB] Fixing broken trainImplicit Scala example in MLlib Collaborative Filtering documentation. Fixing broken trainImplicit Scala example in MLlib Collaborative Filtering documentation to

spark git commit: [SPARK-7883] [DOCS] [MLLIB] Fixing broken trainImplicit Scala example in MLlib Collaborative Filtering documentation.

2015-05-26 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.2 6c41e1cb9 - d5763c3b9 [SPARK-7883] [DOCS] [MLLIB] Fixing broken trainImplicit Scala example in MLlib Collaborative Filtering documentation. Fixing broken trainImplicit Scala example in MLlib Collaborative Filtering documentation to

spark git commit: [SPARK-7883] [DOCS] [MLLIB] Fixing broken trainImplicit Scala example in MLlib Collaborative Filtering documentation.

2015-05-26 Thread meng
Repository: spark Updated Branches: refs/heads/master 8f2082426 - 0463428b6 [SPARK-7883] [DOCS] [MLLIB] Fixing broken trainImplicit Scala example in MLlib Collaborative Filtering documentation. Fixing broken trainImplicit Scala example in MLlib Collaborative Filtering documentation to match

spark git commit: [SPARK-7868] [SQL] Ignores _temporary directories in HadoopFsRelation

2015-05-26 Thread yhuai
Repository: spark Updated Branches: refs/heads/branch-1.4 faadbd4d9 - d0bd68ff8 [SPARK-7868] [SQL] Ignores _temporary directories in HadoopFsRelation So that potential partial/corrupted data files left by failed tasks/jobs won't affect normal data scan. Author: Cheng Lian

spark git commit: [SPARK-7042] [BUILD] use the standard akka artifacts with hadoop-2.x

2015-05-26 Thread srowen
Repository: spark Updated Branches: refs/heads/master c9adcad81 - 43aa819c0 [SPARK-7042] [BUILD] use the standard akka artifacts with hadoop-2.x Both akka 2.3.x and hadoop-2.x use protobuf 2.5 so only hadoop-1 build needs custom 2.3.4-spark akka version that shades protobuf-2.5 This

spark git commit: [SQL][minor] Removed unused Catalyst logical plan DSL.

2015-05-26 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.4 44b22a3f1 - 4b31a07b6 [SQL][minor] Removed unused Catalyst logical plan DSL. The Catalyst DSL is no longer used as a public facing API. This pull request removes the UDF and writeToFile feature from it since they are not used in unit

spark git commit: [SPARK-7854] [TEST] refine Kryo test suite

2015-05-26 Thread srowen
Repository: spark Updated Branches: refs/heads/master e5a63a0e3 - 63099122d [SPARK-7854] [TEST] refine Kryo test suite this modification is according to JoshRosen 's comments, for details, please refer to [#5934](https://github.com/apache/spark/pull/5934/files#r30949751). Author: Zhang,

spark git commit: [SPARK-7806][EC2] Fixes that allow the spark_ec2.py tool to run with Python3

2015-05-26 Thread davies
Repository: spark Updated Branches: refs/heads/master 8948ad3fb - 8dbe0 [SPARK-7806][EC2] Fixes that allow the spark_ec2.py tool to run with Python3 I have used this script to launch, destroy, start, and stop clusters successfully. Author: meawoppl meawo...@gmail.com Closes #6336 from

spark git commit: [SPARK-7806][EC2] Fixes that allow the spark_ec2.py tool to run with Python3

2015-05-26 Thread davies
Repository: spark Updated Branches: refs/heads/branch-1.4 25b2f95fe - 42070f096 [SPARK-7806][EC2] Fixes that allow the spark_ec2.py tool to run with Python3 I have used this script to launch, destroy, start, and stop clusters successfully. Author: meawoppl meawo...@gmail.com Closes #6336

spark git commit: [DOCS] [MLLIB] Fixing misformatted links in v1.4 MLlib Naive Bayes documentation by removing space and newline characters.

2015-05-26 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.4 42070f096 - dfd905df5 [DOCS] [MLLIB] Fixing misformatted links in v1.4 MLlib Naive Bayes documentation by removing space and newline characters. A couple of links in the MLlib Naive Bayes documentation for v1.4 were broken due to the

spark git commit: [DOCS] [MLLIB] Fixing misformatted links in v1.4 MLlib Naive Bayes documentation by removing space and newline characters.

2015-05-26 Thread srowen
Repository: spark Updated Branches: refs/heads/master 8dbe0 - e5a63a0e3 [DOCS] [MLLIB] Fixing misformatted links in v1.4 MLlib Naive Bayes documentation by removing space and newline characters. A couple of links in the MLlib Naive Bayes documentation for v1.4 were broken due to the

spark git commit: [SPARK-7844] [MLLIB] Fix broken tests in KernelDensity

2015-05-26 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.4 dfd905df5 - 51d98b0e9 [SPARK-7844] [MLLIB] Fix broken tests in KernelDensity The densities in KernelDensity are scaled down by (number of parallel processes X number of points). It should be just no.of samples. This results in broken

spark git commit: [SPARK-7844] [MLLIB] Fix broken tests in KernelDensity

2015-05-26 Thread meng
Repository: spark Updated Branches: refs/heads/master b7d808594 - 61664732b [SPARK-7844] [MLLIB] Fix broken tests in KernelDensity The densities in KernelDensity are scaled down by (number of parallel processes X number of points). It should be just no.of samples. This results in broken

spark git commit: [CORE] [TEST] Fix SimpleDateParamTest

2015-05-26 Thread irashid
Repository: spark Updated Branches: refs/heads/master 43aa819c0 - bf49c2213 [CORE] [TEST] Fix SimpleDateParamTest ``` sbt.ForkMain$ForkError: 1424424077190 was not equal to 1424474477190 at org.scalatest.MatchersHelper$.newTestFailedException(MatchersHelper.scala:160) at

spark git commit: [CORE] [TEST] Fix SimpleDateParamTest

2015-05-26 Thread irashid
Repository: spark Updated Branches: refs/heads/branch-1.4 4b31a07b6 - 79bb7dcec [CORE] [TEST] Fix SimpleDateParamTest ``` sbt.ForkMain$ForkError: 1424424077190 was not equal to 1424474477190 at org.scalatest.MatchersHelper$.newTestFailedException(MatchersHelper.scala:160) at

spark git commit: [SPARK-6602] [CORE] Remove some places in core that calling SparkEnv.actorSystem

2015-05-26 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 2e9a5f229 - 9f742241c [SPARK-6602] [CORE] Remove some places in core that calling SparkEnv.actorSystem Author: zsxwing zsxw...@gmail.com Closes #6333 from zsxwing/remove-actor-system-usage and squashes the following commits: f125aa6

spark git commit: [SPARK-7864] [UI] Do not kill innocent stages from visualization

2015-05-26 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.4 b5ee7eefd - f9dfa4d0f [SPARK-7864] [UI] Do not kill innocent stages from visualization **Reproduction.** Run a long-running job, go to the job page, expand the DAG visualization, and click into a stage. Your stage is now killed. Why?

spark git commit: [SPARK-7864] [UI] Do not kill innocent stages from visualization

2015-05-26 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 836a75898 - 8f2082426 [SPARK-7864] [UI] Do not kill innocent stages from visualization **Reproduction.** Run a long-running job, go to the job page, expand the DAG visualization, and click into a stage. Your stage is now killed. Why? This

spark git commit: [SPARK-3674] YARN support in Spark EC2

2015-05-26 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.4 51d98b0e9 - d014a447a [SPARK-3674] YARN support in Spark EC2 This corresponds to https://github.com/mesos/spark-ec2/pull/116 in the spark-ec2 repo. The only changes required on the spark_ec2.py script is to open the RM port. cc

spark git commit: [SPARK-7748] [MLLIB] Graduate spark.ml from alpha

2015-05-26 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.4 d014a447a - b5ee7eefd [SPARK-7748] [MLLIB] Graduate spark.ml from alpha With descent coverage of feature transformers, algorithms, and model tuning support, it is time to graduate `spark.ml` from alpha. This PR changes all

spark git commit: [SPARK-7339] [PYSPARK] PySpark shuffle spill memory sometimes are not correct

2015-05-26 Thread davies
Repository: spark Updated Branches: refs/heads/branch-1.4 79bb7dcec - 25b2f95fe [SPARK-7339] [PYSPARK] PySpark shuffle spill memory sometimes are not correct In PySpark we get memory used before and after spill, then use the difference of these two value as memorySpilled, but if the before

spark git commit: Revert [SPARK-7042] [BUILD] use the standard akka artifacts with hadoop-2.x

2015-05-26 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 63099122d - b7d808594 Revert [SPARK-7042] [BUILD] use the standard akka artifacts with hadoop-2.x This reverts commit 43aa819c041f6e8301ad1b8f82eb68e14254f636. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: