spark git commit: [SPARK-15698][SQL][STREAMING][FOLLW-UP] Fix FileStream source and sink log get configuration issue

2016-09-20 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 1ea49916a -> e48ebc4e4 [SPARK-15698][SQL][STREAMING][FOLLW-UP] Fix FileStream source and sink log get configuration issue ## What changes were proposed in this pull request? This issue was introduced in the previous commit of

spark git commit: [MINOR][BUILD] Fix CheckStyle Error

2016-09-20 Thread rxin
Repository: spark Updated Branches: refs/heads/master 976f3b122 -> 1ea49916a [MINOR][BUILD] Fix CheckStyle Error ## What changes were proposed in this pull request? This PR is to fix the code style errors before 2.0.1 release. ## How was this patch tested? Manual. Before: ```

spark git commit: [SPARK-17513][SQL] Make StreamExecution garbage-collect its metadata

2016-09-20 Thread rxin
Repository: spark Updated Branches: refs/heads/master 7e418e99c -> 976f3b122 [SPARK-17513][SQL] Make StreamExecution garbage-collect its metadata ## What changes were proposed in this pull request? This PR modifies StreamExecution such that it discards metadata for batches that have already

spark git commit: [SPARK-17513][SQL] Make StreamExecution garbage-collect its metadata

2016-09-20 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 8d8e2332c -> 726f05716 [SPARK-17513][SQL] Make StreamExecution garbage-collect its metadata ## What changes were proposed in this pull request? This PR modifies StreamExecution such that it discards metadata for batches that have

spark git commit: [SPARK-17611][YARN][TEST] Make shuffle service test really test auth.

2016-09-20 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 9ac68dbc5 -> 7e418e99c [SPARK-17611][YARN][TEST] Make shuffle service test really test auth. Currently, the code is just swallowing exceptions, and not really checking whether the auth information was being recorded properly. Fix both

spark git commit: [SPARK-15698][SQL][STREAMING] Add the ability to remove the old MetadataLog in FileStreamSource (branch-2.0)

2016-09-20 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.0 2bd37ce96 -> 8d8e2332c [SPARK-15698][SQL][STREAMING] Add the ability to remove the old MetadataLog in FileStreamSource (branch-2.0) ## What changes were proposed in this pull request? Backport #13513 to branch 2.0. ## How was this

spark git commit: [SPARK-17549][SQL] Revert "[] Only collect table size stat in driver for cached relation."

2016-09-20 Thread yhuai
Repository: spark Updated Branches: refs/heads/master a6aade004 -> 9ac68dbc5 [SPARK-17549][SQL] Revert "[] Only collect table size stat in driver for cached relation." This reverts commit 39e2bad6a866d27c3ca594d15e574a1da3ee84cc because of the problem mentioned at

spark git commit: [SPARK-17549][SQL] Revert "[] Only collect table size stat in driver for cached relation."

2016-09-20 Thread yhuai
Repository: spark Updated Branches: refs/heads/branch-2.0 e76f4f47f -> 2bd37ce96 [SPARK-17549][SQL] Revert "[] Only collect table size stat in driver for cached relation." This reverts commit 39e2bad6a866d27c3ca594d15e574a1da3ee84cc because of the problem mentioned at

spark git commit: [SPARK-15698][SQL][STREAMING] Add the ability to remove the old MetadataLog in FileStreamSource

2016-09-20 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master eb004c662 -> a6aade004 [SPARK-15698][SQL][STREAMING] Add the ability to remove the old MetadataLog in FileStreamSource ## What changes were proposed in this pull request? Current `metadataLog` in `FileStreamSource` will add a checkpoint

spark git commit: [SPARK-17051][SQL] we should use hadoopConf in InsertIntoHiveTable

2016-09-20 Thread yhuai
Repository: spark Updated Branches: refs/heads/branch-2.0 643f161d5 -> e76f4f47f [SPARK-17051][SQL] we should use hadoopConf in InsertIntoHiveTable ## What changes were proposed in this pull request? Hive confs in hive-site.xml will be loaded in `hadoopConf`, so we should use `hadoopConf`

spark git commit: [SPARK-17051][SQL] we should use hadoopConf in InsertIntoHiveTable

2016-09-20 Thread yhuai
Repository: spark Updated Branches: refs/heads/master d5ec5dbb0 -> eb004c662 [SPARK-17051][SQL] we should use hadoopConf in InsertIntoHiveTable ## What changes were proposed in this pull request? Hive confs in hive-site.xml will be loaded in `hadoopConf`, so we should use `hadoopConf` in

spark git commit: Revert "[SPARK-17513][SQL] Make StreamExecution garbage-collect its metadata"

2016-09-20 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.0 5456a1b4f -> 643f161d5 Revert "[SPARK-17513][SQL] Make StreamExecution garbage-collect its metadata" This reverts commit 5456a1b4fcd85d0d7f2f1cc64e44967def0950bf. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [SPARK-17502][SQL] Fix Multiple Bugs in DDL Statements on Temporary Views

2016-09-20 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 4a426ff8a -> d5ec5dbb0 [SPARK-17502][SQL] Fix Multiple Bugs in DDL Statements on Temporary Views ### What changes were proposed in this pull request? - When the permanent tables/views do not exist but the temporary view exists, the

spark git commit: [SPARK-17437] Add uiWebUrl to JavaSparkContext and pyspark.SparkContext

2016-09-20 Thread srowen
Repository: spark Updated Branches: refs/heads/master f039d964d -> 4a426ff8a [SPARK-17437] Add uiWebUrl to JavaSparkContext and pyspark.SparkContext ## What changes were proposed in this pull request? The Scala version of `SparkContext` has a handy field called `uiWebUrl` that tells you

spark git commit: Revert "[SPARK-17513][SQL] Make StreamExecution garbage-collect its metadata"

2016-09-20 Thread wenchen
Repository: spark Updated Branches: refs/heads/master be9d57fc9 -> f039d964d Revert "[SPARK-17513][SQL] Make StreamExecution garbage-collect its metadata" This reverts commit be9d57fc9d8b10e4234c01c06ed43fd7dd12c07b. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: