spark git commit: [SPARK-14019][SQL] Remove noop SortOrder in Sort

2016-03-20 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 454a00df2 -> f58319a24 [SPARK-14019][SQL] Remove noop SortOrder in Sort What changes were proposed in this pull request? This PR is to add a new Optimizer rule for pruning Sort if its SortOrder is no-op. In the phase of **Optimizer**

spark git commit: [SPARK-13993][PYSPARK] Add pyspark Rformula/RforumlaModel save/load

2016-03-20 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 811a52472 -> 454a00df2 [SPARK-13993][PYSPARK] Add pyspark Rformula/RforumlaModel save/load ## What changes were proposed in this pull request? https://issues.apache.org/jira/browse/SPARK-13993 ## How was this patch tested? doctest Autho

spark git commit: [SPARK-12182][ML] Distributed binning for trees in spark.ml

2016-03-20 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master d630a203d -> 811a52472 [SPARK-12182][ML] Distributed binning for trees in spark.ml This PR changes the `findSplits` method in spark.ml to perform split calculations on the workers. This PR is meant to copy [PR-8246](https://github.com/apa

spark git commit: [SPARK-13901][CORE] correct the logDebug information when jump to the next locality level

2016-03-20 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.6 765307f41 -> 1fcd17f84 [SPARK-13901][CORE] correct the logDebug information when jump to the next locality level JIRA Issue:https://issues.apache.org/jira/browse/SPARK-13901 In getAllowedLocalityLevel method of TaskSetManager,we get wr

spark git commit: [SPARK-13761][ML] Deprecate validateParams

2016-03-20 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master d4d84936f -> 92b70576e [SPARK-13761][ML] Deprecate validateParams ## What changes were proposed in this pull request? Deprecate validateParams() method here: https://github.com/apache/spark/blob/035d3acdf3c1be5b309a861d5c5beb803b946b5e/ml

spark git commit: [MINOR][DOC] Add JavaStreamingTestExample

2016-03-20 Thread mlnick
Repository: spark Updated Branches: refs/heads/master 30c18841e -> 204c9dec2 [MINOR][DOC] Add JavaStreamingTestExample ## What changes were proposed in this pull request? Add the java example of StreamingTest ## How was this patch tested? manual tests in CLI: bin/run-example mllib.JavaStrea

spark git commit: [SPARK-13942][CORE][DOCS] Remove Shark-related docs for 2.x

2016-03-20 Thread rxin
Repository: spark Updated Branches: refs/heads/master 27e1f3885 -> 4ce2d24e2 [SPARK-13942][CORE][DOCS] Remove Shark-related docs for 2.x ## What changes were proposed in this pull request? `Shark` was merged into `Spark SQL` since [July 2014](https://databricks.com/blog/2014/07/01/shark-spar

spark git commit: [SPARK-13885][YARN] Fix attempt id regression for Spark running on Yarn

2016-03-20 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 9c23c818c -> 353778216 [SPARK-13885][YARN] Fix attempt id regression for Spark running on Yarn ## What changes were proposed in this pull request? This regression is introduced in #9182, previously attempt id is simply as counter "1" or "

spark git commit: [SPARK-13927][MLLIB] add row/column iterator to local matrices

2016-03-20 Thread dbtsai
Repository: spark Updated Branches: refs/heads/master 6fc2b6541 -> 85c42fda9 [SPARK-13927][MLLIB] add row/column iterator to local matrices ## What changes were proposed in this pull request? Add row/column iterator to local matrices to simplify tasks like BlockMatrix => RowMatrix conversion