spark git commit: [SPARK-12321][SQL] JSON format for TreeNode (use reflection)

2015-12-21 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 474eb21a3 -> 7634fe951 [SPARK-12321][SQL] JSON format for TreeNode (use reflection) An alternative solution for https://github.com/apache/spark/pull/10295 , instead of implementing json format for all logical/physical plans and

spark git commit: Doc typo: ltrim = trim from left end, not right

2015-12-21 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 1eb90bc9c -> fc6dbcc70 Doc typo: ltrim = trim from left end, not right Author: pshearer Closes #10414 from pshearer/patch-1. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: Doc typo: ltrim = trim from left end, not right

2015-12-21 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.6 d6a519ff2 -> c754a0879 Doc typo: ltrim = trim from left end, not right Author: pshearer Closes #10414 from pshearer/patch-1. (cherry picked from commit fc6dbcc7038c2b030ef6a2dc8be5848499ccee1c)

spark git commit: Doc typo: ltrim = trim from left end, not right

2015-12-21 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.5 eb54c914a -> 4d54ba896 Doc typo: ltrim = trim from left end, not right Author: pshearer Closes #10414 from pshearer/patch-1. (cherry picked from commit fc6dbcc7038c2b030ef6a2dc8be5848499ccee1c)

spark git commit: [SPARK-12374][SPARK-12150][SQL] Adding logical/physical operators for Range

2015-12-21 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 7634fe951 -> 4883a5087 [SPARK-12374][SPARK-12150][SQL] Adding logical/physical operators for Range Based on the suggestions from marmbrus , added logical/physical operators for Range for improving the performance. Also added another API

spark git commit: [SPARK-12392][CORE] Optimize a location order of broadcast blocks by considering preferred local hosts

2015-12-21 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 4883a5087 -> 935f46630 [SPARK-12392][CORE] Optimize a location order of broadcast blocks by considering preferred local hosts When multiple workers exist in a host, we can bypass unnecessary remote access for broadcasts; block managers

spark git commit: [SPARK-12339][SPARK-11206][WEBUI] Added a null check that was removed in

2015-12-21 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master fc6dbcc70 -> b0849b8ae [SPARK-12339][SPARK-11206][WEBUI] Added a null check that was removed in Updates made in SPARK-11206 missed an edge case which cause's a NullPointerException when a task is killed. In some cases when a task ends in

spark git commit: [SPARK-12388] change default compression to lz4

2015-12-21 Thread davies
Repository: spark Updated Branches: refs/heads/master d655d37dd -> 29cecd4a4 [SPARK-12388] change default compression to lz4 According the benchmark [1], LZ4-java could be 80% (or 30%) faster than Snappy. After changing the compressor to LZ4, I saw 20% improvement on end-to-end time for a

spark git commit: [SPARK-2331] SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T]

2015-12-21 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master b0849b8ae -> a820ca19d [SPARK-2331] SparkContext.emptyRDD should return RDD[T] not EmptyRDD[T] Author: Reynold Xin Closes #10394 from rxin/SPARK-2331. Project: http://git-wip-us.apache.org/repos/asf/spark/repo

spark git commit: [SPARK-12466] Fix harmless NPE in tests

2015-12-21 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.6 c754a0879 -> ca3998512 [SPARK-12466] Fix harmless NPE in tests ``` [info] ReplayListenerSuite: [info] - Simple replay (58 milliseconds) java.lang.NullPointerException at

spark git commit: [SPARK-12466] Fix harmless NPE in tests

2015-12-21 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master a820ca19d -> d655d37dd [SPARK-12466] Fix harmless NPE in tests ``` [info] ReplayListenerSuite: [info] - Simple replay (58 milliseconds) java.lang.NullPointerException at

spark git commit: [SPARK-11807] Remove support for Hadoop < 2.2

2015-12-21 Thread rxin
Repository: spark Updated Branches: refs/heads/master 29cecd4a4 -> 0a38637d0 [SPARK-11807] Remove support for Hadoop < 2.2 i.e. Hadoop 1 and Hadoop 2.0 Author: Reynold Xin Closes #10404 from rxin/SPARK-11807. Project: http://git-wip-us.apache.org/repos/asf/spark/repo

spark git commit: [MINOR] Fix typos in JavaStreamingContext

2015-12-21 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.6 5b19e7cfd -> 309ef355f [MINOR] Fix typos in JavaStreamingContext Author: Shixiong Zhu Closes #10424 from zsxwing/typo. (cherry picked from commit 93da8565fea42d8ac978df411daced4a9ea3a9c8) Signed-off-by:

spark git commit: [MINOR] Fix typos in JavaStreamingContext

2015-12-21 Thread rxin
Repository: spark Updated Branches: refs/heads/master 0a38637d0 -> 93da8565f [MINOR] Fix typos in JavaStreamingContext Author: Shixiong Zhu Closes #10424 from zsxwing/typo. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

[2/2] spark git commit: Preparing development version 1.6.0-SNAPSHOT

2015-12-21 Thread pwendell
Preparing development version 1.6.0-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/5b19e7cf Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/5b19e7cf Diff:

[spark] Git Push Summary

2015-12-21 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.6.0-rc4 [created] 4062cda30 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[1/2] spark git commit: Preparing Spark release v1.6.0-rc4

2015-12-21 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.6 ca3998512 -> 5b19e7cfd Preparing Spark release v1.6.0-rc4 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/4062cda3 Tree:

spark git commit: [SPARK-12296][PYSPARK][MLLIB] Feature parity for pyspark mllib standard scaler model

2015-12-21 Thread mlnick
Repository: spark Updated Branches: refs/heads/master 2235cd444 -> 969d5665b [SPARK-12296][PYSPARK][MLLIB] Feature parity for pyspark mllib standard scaler model Some methods are missing, such as ways to access the std, mean, etc. This PR is for feature parity for

spark git commit: [SPARK-12349][ML] Make spark.ml PCAModel load backwards compatible

2015-12-21 Thread srowen
Repository: spark Updated Branches: refs/heads/master ce1798b3a -> d0f695089 [SPARK-12349][ML] Make spark.ml PCAModel load backwards compatible Only load explainedVariance in PCAModel if it was written with Spark > 1.6.x jkbradley is this kind of what you had in mind? Author: Sean Owen

spark git commit: [SPARK-11823][SQL] Fix flaky JDBC cancellation test in HiveThriftBinaryServerSuite

2015-12-21 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 93da8565f -> 2235cd444 [SPARK-11823][SQL] Fix flaky JDBC cancellation test in HiveThriftBinaryServerSuite This patch fixes a flaky "test jdbc cancel" test in HiveThriftBinaryServerSuite. This test is prone to a race-condition which

spark git commit: [SPARK-11823][SQL] Fix flaky JDBC cancellation test in HiveThriftBinaryServerSuite

2015-12-21 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.6 309ef355f -> 0f905d7df [SPARK-11823][SQL] Fix flaky JDBC cancellation test in HiveThriftBinaryServerSuite This patch fixes a flaky "test jdbc cancel" test in HiveThriftBinaryServerSuite. This test is prone to a race-condition which

spark git commit: [PYSPARK] Pyspark typo & Add missing abstractmethod annotation

2015-12-21 Thread davies
Repository: spark Updated Branches: refs/heads/master d0f695089 -> 1920d72a1 [PYSPARK] Pyspark typo & Add missing abstractmethod annotation No jira is created since this is a trivial change. davies Please help review it Author: Jeff Zhang Closes #10143 from