svn commit: r29904 - in /dev/spark/2.4.1-SNAPSHOT-2018_10_05_22_02-a2991d2-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-10-05 Thread pwendell
Author: pwendell Date: Sat Oct 6 05:16:47 2018 New Revision: 29904 Log: Apache Spark 2.4.1-SNAPSHOT-2018_10_05_22_02-a2991d2 docs [This commit notification would consist of 1472 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25646][K8S] Fix docker-image-tool.sh on dev build.

2018-10-05 Thread dongjoon
Repository: spark Updated Branches: refs/heads/branch-2.4 0a70afdc0 -> a2991d233 [SPARK-25646][K8S] Fix docker-image-tool.sh on dev build. The docker file was referencing a path that only existed in the distribution tarball; it needs to be parameterized so that the right path can be used in a

spark git commit: [SPARK-25646][K8S] Fix docker-image-tool.sh on dev build.

2018-10-05 Thread dongjoon
Repository: spark Updated Branches: refs/heads/master 2c6f4d61b -> 58287a398 [SPARK-25646][K8S] Fix docker-image-tool.sh on dev build. The docker file was referencing a path that only existed in the distribution tarball; it needs to be parameterized so that the right path can be used in a dev

svn commit: r29903 - in /dev/spark/3.0.0-SNAPSHOT-2018_10_05_20_02-2c6f4d6-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-10-05 Thread pwendell
Author: pwendell Date: Sat Oct 6 03:17:28 2018 New Revision: 29903 Log: Apache Spark 3.0.0-SNAPSHOT-2018_10_05_20_02-2c6f4d6 docs [This commit notification would consist of 1485 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25610][SQL][TEST] Improve execution time of DatasetCacheSuite: cache UDF result correctly

2018-10-05 Thread lixiao
Repository: spark Updated Branches: refs/heads/master bbd038d24 -> 2c6f4d61b [SPARK-25610][SQL][TEST] Improve execution time of DatasetCacheSuite: cache UDF result correctly ## What changes were proposed in this pull request? In this test case, we are verifying that the result of an UDF is

spark git commit: [SPARK-25653][TEST] Add tag ExtendedHiveTest for HiveSparkSubmitSuite

2018-10-05 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 1c9486c1a -> bbd038d24 [SPARK-25653][TEST] Add tag ExtendedHiveTest for HiveSparkSubmitSuite ## What changes were proposed in this pull request? The total run time of `HiveSparkSubmitSuite` is about 10 minutes. While the related code is

spark git commit: [SPARK-25635][SQL][BUILD] Support selective direct encoding in native ORC write

2018-10-05 Thread lixiao
Repository: spark Updated Branches: refs/heads/master a433fbcee -> 1c9486c1a [SPARK-25635][SQL][BUILD] Support selective direct encoding in native ORC write ## What changes were proposed in this pull request? Before ORC 1.5.3, `orc.dictionary.key.threshold` and

svn commit: r29902 - in /dev/spark/3.0.0-SNAPSHOT-2018_10_05_16_03-a433fbc-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-10-05 Thread pwendell
Author: pwendell Date: Fri Oct 5 23:17:20 2018 New Revision: 29902 Log: Apache Spark 3.0.0-SNAPSHOT-2018_10_05_16_03-a433fbc docs [This commit notification would consist of 1485 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25626][SQL][TEST] Improve the test execution time of HiveClientSuites

2018-10-05 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 7dcc90fbb -> a433fbcee [SPARK-25626][SQL][TEST] Improve the test execution time of HiveClientSuites ## What changes were proposed in this pull request? Improve the runtime by reducing the number of partitions created in the test. The

svn commit: r29901 - in /dev/spark/2.4.1-SNAPSHOT-2018_10_05_14_03-0a70afd-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-10-05 Thread pwendell
Author: pwendell Date: Fri Oct 5 21:17:19 2018 New Revision: 29901 Log: Apache Spark 2.4.1-SNAPSHOT-2018_10_05_14_03-0a70afd docs [This commit notification would consist of 1472 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r29900 - in /dev/spark/3.0.0-SNAPSHOT-2018_10_05_12_03-7dcc90f-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-10-05 Thread pwendell
Author: pwendell Date: Fri Oct 5 19:17:42 2018 New Revision: 29900 Log: Apache Spark 3.0.0-SNAPSHOT-2018_10_05_12_03-7dcc90f docs [This commit notification would consist of 1485 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25644][SS] Fix java foreachBatch in DataStreamWriter

2018-10-05 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.4 2c700ee30 -> 0a70afdc0 [SPARK-25644][SS] Fix java foreachBatch in DataStreamWriter ## What changes were proposed in this pull request? The java `foreachBatch` API in `DataStreamWriter` should accept `java.lang.Long` rather

spark git commit: [SPARK-25644][SS] Fix java foreachBatch in DataStreamWriter

2018-10-05 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 434ada12a -> 7dcc90fbb [SPARK-25644][SS] Fix java foreachBatch in DataStreamWriter ## What changes were proposed in this pull request? The java `foreachBatch` API in `DataStreamWriter` should accept `java.lang.Long` rather `scala.Long`.

svn commit: r29898 - in /dev/spark/2.4.1-SNAPSHOT-2018_10_05_10_02-2c700ee-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-10-05 Thread pwendell
Author: pwendell Date: Fri Oct 5 17:17:28 2018 New Revision: 29898 Log: Apache Spark 2.4.1-SNAPSHOT-2018_10_05_10_02-2c700ee docs [This commit notification would consist of 1472 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

svn commit: r29892 - in /dev/spark/3.0.0-SNAPSHOT-2018_10_05_04_02-434ada1-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-10-05 Thread pwendell
Author: pwendell Date: Fri Oct 5 11:17:07 2018 New Revision: 29892 Log: Apache Spark 3.0.0-SNAPSHOT-2018_10_05_04_02-434ada1 docs [This commit notification would consist of 1485 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]

spark git commit: [SPARK-25521][SQL] Job id showing null in the logs when insert into command Job is finished.

2018-10-05 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.4 c9bb83a7d -> 2c700ee30 [SPARK-25521][SQL] Job id showing null in the logs when insert into command Job is finished. ## What changes were proposed in this pull request? ``As part of insert command in FileFormatWriter, a job context

spark git commit: [SPARK-17952][SQL] Nested Java beans support in createDataFrame

2018-10-05 Thread ueshin
Repository: spark Updated Branches: refs/heads/master ab1650d29 -> 434ada12a [SPARK-17952][SQL] Nested Java beans support in createDataFrame ## What changes were proposed in this pull request? When constructing a DataFrame from a Java bean, using nested beans throws an error despite

spark git commit: [SPARK-24601] Update Jackson to 2.9.6

2018-10-05 Thread gurwls223
Repository: spark Updated Branches: refs/heads/master 459700727 -> ab1650d29 [SPARK-24601] Update Jackson to 2.9.6 Hi all, Jackson is incompatible with upstream versions, therefore bump the Jackson version to a more recent one. I bumped into some issues with Azure CosmosDB that is using a

svn commit: r29888 - in /dev/spark/3.0.0-SNAPSHOT-2018_10_05_00_02-4597007-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _s

2018-10-05 Thread pwendell
Author: pwendell Date: Fri Oct 5 07:17:16 2018 New Revision: 29888 Log: Apache Spark 3.0.0-SNAPSHOT-2018_10_05_00_02-4597007 docs [This commit notification would consist of 1485 parts, which exceeds the limit of 50 ones, so it was shortened to the summary.]