spark git commit: [SPARK-19082][SQL] Make ignoreCorruptFiles work for Parquet

2017-01-15 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.1 bf2f233e4 -> 4f3ce062c [SPARK-19082][SQL] Make ignoreCorruptFiles work for Parquet ## What changes were proposed in this pull request? We have a config `spark.sql.files.ignoreCorruptFiles` which can be used to ignore corrupt files

spark git commit: [SPARK-19082][SQL] Make ignoreCorruptFiles work for Parquet

2017-01-15 Thread wenchen
Repository: spark Updated Branches: refs/heads/master de62ddf7f -> 61e48f52d [SPARK-19082][SQL] Make ignoreCorruptFiles work for Parquet ## What changes were proposed in this pull request? We have a config `spark.sql.files.ignoreCorruptFiles` which can be used to ignore corrupt files when

spark git commit: [SPARK-19092][SQL][BACKPORT-2.1] Save() API of DataFrameWriter should not scan all the saved files #16481

2017-01-15 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.1 db37049da -> bf2f233e4 [SPARK-19092][SQL][BACKPORT-2.1] Save() API of DataFrameWriter should not scan all the saved files #16481 ### What changes were proposed in this pull request? This PR is to backport

spark git commit: [SPARK-19120] Refresh Metadata Cache After Loading Hive Tables

2017-01-15 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.1 5e9be1e1f -> db37049da [SPARK-19120] Refresh Metadata Cache After Loading Hive Tables ```Scala sql("CREATE TABLE tab (a STRING) STORED AS PARQUET") // This table fetch is to fill the cache with zero leaf files

spark git commit: [SPARK-19120] Refresh Metadata Cache After Loading Hive Tables

2017-01-15 Thread wenchen
Repository: spark Updated Branches: refs/heads/master a5e651f4c -> de62ddf7f [SPARK-19120] Refresh Metadata Cache After Loading Hive Tables ### What changes were proposed in this pull request? ```Scala sql("CREATE TABLE tab (a STRING) STORED AS PARQUET") // This table fetch

spark git commit: [SPARK-19206][DOC][DSTREAM] Fix outdated parameter descriptions in kafka010

2017-01-15 Thread srowen
Repository: spark Updated Branches: refs/heads/master a8567e34d -> a5e651f4c [SPARK-19206][DOC][DSTREAM] Fix outdated parameter descriptions in kafka010 ## What changes were proposed in this pull request? Fix outdated parameter descriptions in kafka010 ## How was this patch tested? cc

spark git commit: [SPARK-18971][CORE] Upgrade Netty to 4.0.43.Final

2017-01-15 Thread srowen
Repository: spark Updated Branches: refs/heads/master 3df2d9314 -> a8567e34d [SPARK-18971][CORE] Upgrade Netty to 4.0.43.Final ## What changes were proposed in this pull request? Upgrade Netty to `4.0.43.Final` to add the fix for https://github.com/netty/netty/issues/6153 ## How was this

spark git commit: [MINOR][DOC] Document local[*,F] master modes

2017-01-15 Thread srowen
Repository: spark Updated Branches: refs/heads/master c9d612f82 -> 3df2d9314 [MINOR][DOC] Document local[*,F] master modes ## What changes were proposed in this pull request? core/src/main/scala/org/apache/spark/SparkContext.scala contains LOCAL_N_FAILURES_REGEX master mode, but this was

spark git commit: [SPARK-19042] spark executor can't download the jars when uber jar's http url contains any query strings

2017-01-15 Thread srowen
Repository: spark Updated Branches: refs/heads/master 9112f31bb -> c9d612f82 [SPARK-19042] spark executor can't download the jars when uber jar's http url contains any query strings If the uber jars' https contains any query strings, the Executor.updateDependencies method can't can't

spark git commit: [SPARK-19207][SQL] LocalSparkSession should use Slf4JLoggerFactory.INSTANCE

2017-01-15 Thread srowen
Repository: spark Updated Branches: refs/heads/master 894235390 -> 9112f31bb [SPARK-19207][SQL] LocalSparkSession should use Slf4JLoggerFactory.INSTANCE ## What changes were proposed in this pull request? Using Slf4JLoggerFactory.INSTANCE instead of creating Slf4JLoggerFactory's object with