spark git commit: [SPARK-24244][SPARK-24368][SQL] Passing only required columns to the CSV parser

2018-05-24 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 3b20b34ab -> 64fad0b51 [SPARK-24244][SPARK-24368][SQL] Passing only required columns to the CSV parser ## What changes were proposed in this pull request? uniVocity parser allows to specify only required column names or indexes for [parsi

svn commit: r27097 - in /dev/spark/2.4.0-SNAPSHOT-2018_05_24_20_01-0fd68cb-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-05-24 Thread pwendell
Author: pwendell Date: Fri May 25 03:15:45 2018 New Revision: 27097 Log: Apache Spark 2.4.0-SNAPSHOT-2018_05_24_20_01-0fd68cb docs [This commit notification would consist of 1464 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-24367][SQL] Parquet: use JOB_SUMMARY_LEVEL instead of deprecated flag ENABLE_JOB_SUMMARY

2018-05-24 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 0fd68cb72 -> 3b20b34ab [SPARK-24367][SQL] Parquet: use JOB_SUMMARY_LEVEL instead of deprecated flag ENABLE_JOB_SUMMARY ## What changes were proposed in this pull request? In current parquet version,the conf ENABLE_JOB_SUMMARY is deprecate

spark git commit: [SPARK-24234][SS] Support multiple row writers in continuous processing shuffle reader.

2018-05-24 Thread tdas
Repository: spark Updated Branches: refs/heads/master 53c06ddab -> 0fd68cb72 [SPARK-24234][SS] Support multiple row writers in continuous processing shuffle reader. ## What changes were proposed in this pull request? https://docs.google.com/document/d/1IL4kJoKrZWeyIhklKUJqsW-yEN7V7aL05MmM65A

svn commit: r27096 - in /dev/spark/2.4.0-SNAPSHOT-2018_05_24_16_01-53c06dd-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-05-24 Thread pwendell
Author: pwendell Date: Thu May 24 23:15:15 2018 New Revision: 27096 Log: Apache Spark 2.4.0-SNAPSHOT-2018_05_24_16_01-53c06dd docs [This commit notification would consist of 1464 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-24332][SS][MESOS] Fix places reading 'spark.network.timeout' as milliseconds

2018-05-24 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 0d8994344 -> 53c06ddab [SPARK-24332][SS][MESOS] Fix places reading 'spark.network.timeout' as milliseconds ## What changes were proposed in this pull request? This PR replaces `getTimeAsMs` with `getTimeAsSeconds` to fix the issue that r

svn commit: r27092 - in /dev/spark/2.4.0-SNAPSHOT-2018_05_24_12_01-0d89943-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-05-24 Thread pwendell
Author: pwendell Date: Thu May 24 19:16:12 2018 New Revision: 27092 Log: Apache Spark 2.4.0-SNAPSHOT-2018_05_24_12_01-0d89943 docs [This commit notification would consist of 1464 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

svn commit: r27089 - in /dev/spark/2.3.2-SNAPSHOT-2018_05_24_10_01-d0f30e3-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-05-24 Thread pwendell
Author: pwendell Date: Thu May 24 17:15:52 2018 New Revision: 27089 Log: Apache Spark 2.3.2-SNAPSHOT-2018_05_24_10_01-d0f30e3 docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-24378][SQL] Fix date_trunc function incorrect examples

2018-05-24 Thread gurwls223
Repository: spark Updated Branches: refs/heads/branch-2.3 f48d62400 -> d0f30e3f3 [SPARK-24378][SQL] Fix date_trunc function incorrect examples ## What changes were proposed in this pull request? Fix `date_trunc` function incorrect examples. ## How was this patch tested? N/A Author: Yuming

spark git commit: [SPARK-24378][SQL] Fix date_trunc function incorrect examples

2018-05-24 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 13bedc05c -> 0d8994344 [SPARK-24378][SQL] Fix date_trunc function incorrect examples ## What changes were proposed in this pull request? Fix `date_trunc` function incorrect examples. ## How was this patch tested? N/A Author: Yuming Wang

svn commit: r27087 - in /dev/spark/2.4.0-SNAPSHOT-2018_05_24_08_02-13bedc0-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-05-24 Thread pwendell
Author: pwendell Date: Thu May 24 15:21:11 2018 New Revision: 27087 Log: Apache Spark 2.4.0-SNAPSHOT-2018_05_24_08_02-13bedc0 docs [This commit notification would consist of 1464 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

spark git commit: [SPARK-24329][SQL] Test for skipping multi-space lines

2018-05-24 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 3469f5c98 -> 13bedc05c [SPARK-24329][SQL] Test for skipping multi-space lines ## What changes were proposed in this pull request? The PR is a continue of https://github.com/apache/spark/pull/21380 . It checks cases that are handled by the

spark git commit: [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase with dictionary filters.

2018-05-24 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.3 068c4ae34 -> f48d62400 [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase with dictionary filters. ## What changes were proposed in this pull request? I missed this commit when preparing #21070. When Parquet is able to filter bloc

spark git commit: [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase with dictionary filters.

2018-05-24 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 4a14dc0af -> 3469f5c98 [SPARK-24230][SQL] Fix SpecificParquetRecordReaderBase with dictionary filters. ## What changes were proposed in this pull request? I missed this commit when preparing #21070. When Parquet is able to filter blocks w

svn commit: r27082 - in /dev/spark/2.3.2-SNAPSHOT-2018_05_24_02_01-068c4ae-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-05-24 Thread pwendell
Author: pwendell Date: Thu May 24 09:17:13 2018 New Revision: 27082 Log: Apache Spark 2.3.2-SNAPSHOT-2018_05_24_02_01-068c4ae docs [This commit notification would consist of 1443 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---

svn commit: r27076 - in /dev/spark/2.4.0-SNAPSHOT-2018_05_24_00_01-4a14dc0-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-05-24 Thread pwendell
Author: pwendell Date: Thu May 24 07:18:05 2018 New Revision: 27076 Log: Apache Spark 2.4.0-SNAPSHOT-2018_05_24_00_01-4a14dc0 docs [This commit notification would consist of 1464 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.] ---