spark git commit: [SPARK-17213][SQL] Disable Parquet filter push-down for string and binary columns due to PARQUET-686

2016-12-01 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.1 0f0903d17 -> a7f8ebb86 [SPARK-17213][SQL] Disable Parquet filter push-down for string and binary columns due to PARQUET-686 This PR targets to both master and branch-2.1. ## What changes were proposed in this pull request? Due to

spark git commit: [SPARK-17213][SQL] Disable Parquet filter push-down for string and binary columns due to PARQUET-686

2016-12-01 Thread rxin
Repository: spark Updated Branches: refs/heads/master c82f16c15 -> ca6391637 [SPARK-17213][SQL] Disable Parquet filter push-down for string and binary columns due to PARQUET-686 This PR targets to both master and branch-2.1. ## What changes were proposed in this pull request? Due to

spark git commit: [SPARK-18284][SQL] Make ExpressionEncoder.serializer.nullable precise

2016-12-01 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.1 b9eb10043 -> fce1be6cc [SPARK-18284][SQL] Make ExpressionEncoder.serializer.nullable precise ## What changes were proposed in this pull request? This PR makes `ExpressionEncoder.serializer.nullable` for flat encoder for a primitive

spark git commit: [SPARK-18284][SQL] Make ExpressionEncoder.serializer.nullable precise

2016-12-01 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 70c5549ee -> 38b9e6962 [SPARK-18284][SQL] Make ExpressionEncoder.serializer.nullable precise ## What changes were proposed in this pull request? This PR makes `ExpressionEncoder.serializer.nullable` for flat encoder for a primitive type

spark git commit: [SPARK-18538][SQL][BACKPORT-2.1] Fix Concurrent Table Fetching Using DataFrameReader JDBC APIs

2016-12-01 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.1 2f91b0154 -> b9eb10043 [SPARK-18538][SQL][BACKPORT-2.1] Fix Concurrent Table Fetching Using DataFrameReader JDBC APIs ### What changes were proposed in this pull request? This PR is to backport

spark git commit: [SPARK-18141][SQL] Fix to quote column names in the predicate clause of the JDBC RDD generated sql statement

2016-12-01 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.1 2d2e80180 -> 2f91b0154 [SPARK-18141][SQL] Fix to quote column names in the predicate clause of the JDBC RDD generated sql statement ## What changes were proposed in this pull request? SQL query generated for the JDBC data source is

spark git commit: [SPARK-18141][SQL] Fix to quote column names in the predicate clause of the JDBC RDD generated sql statement

2016-12-01 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 37e52f879 -> 70c5549ee [SPARK-18141][SQL] Fix to quote column names in the predicate clause of the JDBC RDD generated sql statement ## What changes were proposed in this pull request? SQL query generated for the JDBC data source is not

spark git commit: [SPARK-18639] Build only a single pip package

2016-12-01 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.1 4746674ad -> 2d2e80180 [SPARK-18639] Build only a single pip package ## What changes were proposed in this pull request? We current build 5 separate pip binary tar balls, doubling the release script runtime. It'd be better to build

spark git commit: [SPARK-18639] Build only a single pip package

2016-12-01 Thread rxin
Repository: spark Updated Branches: refs/heads/master 086b0c8f6 -> 37e52f879 [SPARK-18639] Build only a single pip package ## What changes were proposed in this pull request? We current build 5 separate pip binary tar balls, doubling the release script runtime. It'd be better to build one,

spark git commit: [SPARK-18617][SPARK-18560][TESTS] Fix flaky test: StreamingContextSuite. Receiver data should be deserialized properly

2016-12-01 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.1 4c673c656 -> 4746674ad [SPARK-18617][SPARK-18560][TESTS] Fix flaky test: StreamingContextSuite. Receiver data should be deserialized properly ## What changes were proposed in this pull request? Avoid to create multiple threads to

spark git commit: [SPARK-18274][ML][PYSPARK] Memory leak in PySpark JavaWrapper

2016-12-01 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master e65348471 -> 78bb7f807 [SPARK-18274][ML][PYSPARK] Memory leak in PySpark JavaWrapper ## What changes were proposed in this pull request? In`JavaWrapper `'s destructor make Java Gateway dereference object in destructor, using

spark git commit: [SPARK-18274][ML][PYSPARK] Memory leak in PySpark JavaWrapper

2016-12-01 Thread jkbradley
Repository: spark Updated Branches: refs/heads/branch-2.1 6916ddc38 -> 4c673c656 [SPARK-18274][ML][PYSPARK] Memory leak in PySpark JavaWrapper ## What changes were proposed in this pull request? In`JavaWrapper `'s destructor make Java Gateway dereference object in destructor, using

spark git commit: [SPARK-18674][SQL] improve the error message of using join

2016-12-01 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/branch-2.0 6e3fd2b98 -> 729cadb6b [SPARK-18674][SQL] improve the error message of using join ## What changes were proposed in this pull request? The current error message of USING join is quite confusing, for example: ``` scala> val df1 =

spark git commit: [SPARK-18674][SQL] improve the error message of using join

2016-12-01 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/master 2ab8551e7 -> e65348471 [SPARK-18674][SQL] improve the error message of using join ## What changes were proposed in this pull request? The current error message of USING join is quite confusing, for example: ``` scala> val df1 =

spark git commit: [SPARK-18553][CORE][BRANCH-1.6] Fix leak of TaskSetManager following executor loss

2016-12-01 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.6 9136e2693 -> 8f25cb26f [SPARK-18553][CORE][BRANCH-1.6] Fix leak of TaskSetManager following executor loss ## What changes were proposed in this pull request? _This is the master branch-1.6 version of #15986; the original description

spark git commit: [SPARK-18617][BACKPORT] Follow up PR to Close "kryo auto pick" feature for Spark Streaming

2016-12-01 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 5ecd3c23a -> 6e3fd2b98 [SPARK-18617][BACKPORT] Follow up PR to Close "kryo auto pick" feature for Spark Streaming ## What changes were proposed in this pull request? This is a follow-up PR to backport #16052 to branch-2.0 with

spark git commit: [SPARK-18645][DEPLOY] Fix spark-daemon.sh arguments error lead to throws Unrecognized option

2016-12-01 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.1 8579ab5d7 -> cbbe21777 [SPARK-18645][DEPLOY] Fix spark-daemon.sh arguments error lead to throws Unrecognized option ## What changes were proposed in this pull request? spark-daemon.sh will lost single quotes around after #15338. as

spark git commit: [SPARK-18645][DEPLOY] Fix spark-daemon.sh arguments error lead to throws Unrecognized option

2016-12-01 Thread srowen
Repository: spark Updated Branches: refs/heads/master dbf842b7a -> 2ab8551e7 [SPARK-18645][DEPLOY] Fix spark-daemon.sh arguments error lead to throws Unrecognized option ## What changes were proposed in this pull request? spark-daemon.sh will lost single quotes around after #15338. as

spark git commit: [SPARK-18666][WEB UI] Remove the codes checking deprecated config spark.sql.unsafe.enabled

2016-12-01 Thread rxin
Repository: spark Updated Branches: refs/heads/master 88f559f20 -> dbf842b7a [SPARK-18666][WEB UI] Remove the codes checking deprecated config spark.sql.unsafe.enabled ## What changes were proposed in this pull request? `spark.sql.unsafe.enabled` is deprecated since 1.6. There still are

spark git commit: [SPARK-18666][WEB UI] Remove the codes checking deprecated config spark.sql.unsafe.enabled

2016-12-01 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.1 9dc3ef6e1 -> 8579ab5d7 [SPARK-18666][WEB UI] Remove the codes checking deprecated config spark.sql.unsafe.enabled ## What changes were proposed in this pull request? `spark.sql.unsafe.enabled` is deprecated since 1.6. There still are

spark git commit: [SPARK-18635][SQL] Partition name/values not escaped correctly in some cases

2016-12-01 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.1 e8d8e3509 -> 9dc3ef6e1 [SPARK-18635][SQL] Partition name/values not escaped correctly in some cases ## What changes were proposed in this pull request? Due to confusion between URI vs paths, in certain cases we escape partition

spark git commit: [SPARK-18635][SQL] Partition name/values not escaped correctly in some cases

2016-12-01 Thread wenchen
Repository: spark Updated Branches: refs/heads/master b28fe4a4a -> 88f559f20 [SPARK-18635][SQL] Partition name/values not escaped correctly in some cases ## What changes were proposed in this pull request? Due to confusion between URI vs paths, in certain cases we escape partition values