spark git commit: [SPARK-22587] Spark job fails if fs.defaultFS and application jar are different url

2018-01-10 Thread jshao
Repository: spark Updated Branches: refs/heads/master 9b33dfc40 -> a6647ffbf [SPARK-22587] Spark job fails if fs.defaultFS and application jar are different url ## What changes were proposed in this pull request? Two filesystems comparing does not consider the authority of URI. This is spec

spark git commit: [SPARK-22587] Spark job fails if fs.defaultFS and application jar are different url

2018-01-10 Thread jshao
Repository: spark Updated Branches: refs/heads/branch-2.3 551ccfba5 -> 317b0aaed [SPARK-22587] Spark job fails if fs.defaultFS and application jar are different url ## What changes were proposed in this pull request? Two filesystems comparing does not consider the authority of URI. This is

spark git commit: [SPARK-22976][CORE] Cluster mode driver dir removed while running

2018-01-21 Thread jshao
Repository: spark Updated Branches: refs/heads/master 602c6d82d -> 11daeb833 [SPARK-22976][CORE] Cluster mode driver dir removed while running ## What changes were proposed in this pull request? The clean up logic on the worker perviously determined the liveness of a particular applicaiton ba

spark git commit: [SPARK-22976][CORE] Cluster mode driver dir removed while running

2018-01-21 Thread jshao
Repository: spark Updated Branches: refs/heads/branch-2.3 7520491bf -> 5781fa79e [SPARK-22976][CORE] Cluster mode driver dir removed while running ## What changes were proposed in this pull request? The clean up logic on the worker perviously determined the liveness of a particular applicaito

spark git commit: [MINOR][DOC] Fix the path to the examples jar

2018-01-22 Thread jshao
Repository: spark Updated Branches: refs/heads/master ec2289761 -> 60175e959 [MINOR][DOC] Fix the path to the examples jar ## What changes were proposed in this pull request? The example jar file is now in ./examples/jars directory of Spark distribution. Author: Arseniy Tashoyan Closes #20

spark git commit: [MINOR][DOC] Fix the path to the examples jar

2018-01-22 Thread jshao
Repository: spark Updated Branches: refs/heads/branch-2.3 57c320a0d -> cf078a205 [MINOR][DOC] Fix the path to the examples jar ## What changes were proposed in this pull request? The example jar file is now in ./examples/jars directory of Spark distribution. Author: Arseniy Tashoyan Closes

spark git commit: [SPARK-23200] Reset Kubernetes-specific config on Checkpoint restore

2018-01-25 Thread jshao
Repository: spark Updated Branches: refs/heads/master 70a68b328 -> d1721816d [SPARK-23200] Reset Kubernetes-specific config on Checkpoint restore ## What changes were proposed in this pull request? When using the Kubernetes cluster-manager and spawning a Streaming workload, it is important t

spark git commit: [SPARK-23088][CORE] History server not showing incomplete/running applications

2018-01-29 Thread jshao
Repository: spark Updated Branches: refs/heads/master f235df66a -> 31bd1dab1 [SPARK-23088][CORE] History server not showing incomplete/running applications ## What changes were proposed in this pull request? History server not showing incomplete/running applications when spark.history.ui.max

spark git commit: [SPARK-23279][SS] Avoid triggering distributed job for Console sink

2018-01-30 Thread jshao
Repository: spark Updated Branches: refs/heads/master ca04c3ff2 -> 8c6a9c90a [SPARK-23279][SS] Avoid triggering distributed job for Console sink ## What changes were proposed in this pull request? Console sink will redistribute collected local data and trigger a distributed job in each batch

spark git commit: [SPARK-23279][SS] Avoid triggering distributed job for Console sink

2018-01-30 Thread jshao
Repository: spark Updated Branches: refs/heads/branch-2.3 b8778321b -> ab5a51055 [SPARK-23279][SS] Avoid triggering distributed job for Console sink ## What changes were proposed in this pull request? Console sink will redistribute collected local data and trigger a distributed job in each b

spark git commit: Revert "[SPARK-23200] Reset Kubernetes-specific config on Checkpoint restore"

2018-01-31 Thread jshao
Repository: spark Updated Branches: refs/heads/master b6b50efc8 -> 4b7cd479a Revert "[SPARK-23200] Reset Kubernetes-specific config on Checkpoint restore" This reverts commit d1721816d26bedee3c72eeb75db49da500568376. The patch is not fully tested and out-of-date. So revert it. Project: http

spark git commit: [MINOR][YARN] Add disable yarn.nodemanager.vmem-check-enabled option to memLimitExceededLogMessage

2018-03-07 Thread jshao
Repository: spark Updated Branches: refs/heads/master 4c587eb48 -> 04e71c316 [MINOR][YARN] Add disable yarn.nodemanager.vmem-check-enabled option to memLimitExceededLogMessage My spark application sometimes will throw `Container killed by YARN for exceeding memory limits`. Even I increased `

spark git commit: [SPARK-23644][CORE][UI] Use absolute path for REST call in SHS

2018-03-16 Thread jshao
Repository: spark Updated Branches: refs/heads/master c2632edeb -> ca83526de [SPARK-23644][CORE][UI] Use absolute path for REST call in SHS ## What changes were proposed in this pull request? SHS is using a relative path for the REST API call to get the list of the application is a relative

spark git commit: [SPARK-23635][YARN] AM env variable should not overwrite same name env variable set through spark.executorEnv.

2018-03-16 Thread jshao
Repository: spark Updated Branches: refs/heads/master ca83526de -> c95200048 [SPARK-23635][YARN] AM env variable should not overwrite same name env variable set through spark.executorEnv. ## What changes were proposed in this pull request? In the current Spark on YARN code, AM always will co

spark git commit: [SPARK-23708][CORE] Correct comment for function addShutDownHook in ShutdownHookManager

2018-03-18 Thread jshao
Repository: spark Updated Branches: refs/heads/master 61487b308 -> 745c8c090 [SPARK-23708][CORE] Correct comment for function addShutDownHook in ShutdownHookManager ## What changes were proposed in this pull request? Minor modification.Comment below is not right. ``` /** * Adds a shutdown

spark git commit: [SPARK-23644][CORE][UI][BACKPORT-2.3] Use absolute path for REST call in SHS

2018-03-19 Thread jshao
Repository: spark Updated Branches: refs/heads/branch-2.3 5c1c03d08 -> 2f82c037d [SPARK-23644][CORE][UI][BACKPORT-2.3] Use absolute path for REST call in SHS ## What changes were proposed in this pull request? SHS is using a relative path for the REST API call to get the list of the applicat

spark git commit: [SPARK-23361][YARN] Allow AM to restart after initial tokens expire.

2018-03-22 Thread jshao
Repository: spark Updated Branches: refs/heads/master b2edc30db -> 5fa438471 [SPARK-23361][YARN] Allow AM to restart after initial tokens expire. Currently, the Spark AM relies on the initial set of tokens created by the submission client to be able to talk to HDFS and other services that requ

spark git commit: [SPARK-23787][TESTS] Fix file download test in SparkSubmitSuite for Hadoop 2.9.

2018-03-25 Thread jshao
Repository: spark Updated Branches: refs/heads/master 087fb3142 -> eb48edf9c [SPARK-23787][TESTS] Fix file download test in SparkSubmitSuite for Hadoop 2.9. This particular test assumed that Hadoop libraries did not support http as a file system. Hadoop 2.9 does, so the test failed. The test n

spark git commit: [SPARK-23743][SQL] Changed a comparison logic from containing 'slf4j' to starting with 'org.slf4j'

2018-03-29 Thread jshao
Repository: spark Updated Branches: refs/heads/master b34890119 -> df05fb63a [SPARK-23743][SQL] Changed a comparison logic from containing 'slf4j' to starting with 'org.slf4j' ## What changes were proposed in this pull request? isSharedClass returns if some classes can/should be shared or not

spark git commit: [SPARK-22290][CORE] Avoid creating Hive delegation tokens when not necessary.

2017-10-18 Thread jshao
Repository: spark Updated Branches: refs/heads/master 6f1d0dea1 -> dc2714da5 [SPARK-22290][CORE] Avoid creating Hive delegation tokens when not necessary. Hive delegation tokens are only needed when the Spark driver has no access to the kerberos TGT. That happens only in two situations: - whe

spark git commit: [SPARK-22319][CORE] call loginUserFromKeytab before accessing hdfs

2017-10-22 Thread jshao
Repository: spark Updated Branches: refs/heads/master ca2a780e7 -> 57accf6e3 [SPARK-22319][CORE] call loginUserFromKeytab before accessing hdfs In `SparkSubmit`, call `loginUserFromKeytab` before attempting to make RPC calls to the NameNode. I manually tested this patch by: 1. Confirming th

spark git commit: [SPARK-22319][CORE][BACKPORT-2.2] call loginUserFromKeytab before accessing hdfs

2017-10-22 Thread jshao
Repository: spark Updated Branches: refs/heads/branch-2.2 f8c83fdc5 -> bf8163f5b [SPARK-22319][CORE][BACKPORT-2.2] call loginUserFromKeytab before accessing hdfs In SparkSubmit, call loginUserFromKeytab before attempting to make RPC calls to the NameNode. Same as #https://github.com/apache/s

spark git commit: [SPARK-21840][CORE] Add trait that allows conf to be directly set in application.

2017-10-26 Thread jshao
Repository: spark Updated Branches: refs/heads/master 592cfeab9 -> 3073344a2 [SPARK-21840][CORE] Add trait that allows conf to be directly set in application. Currently SparkSubmit uses system properties to propagate configuration to applications. This makes it hard to implement features such

spark git commit: [SPARK-22172][CORE] Worker hangs when the external shuffle service port is already in use

2017-11-01 Thread jshao
Repository: spark Updated Branches: refs/heads/master 556b5d215 -> 96798d14f [SPARK-22172][CORE] Worker hangs when the external shuffle service port is already in use ## What changes were proposed in this pull request? Handling the NonFatal exceptions while starting the external shuffle serv

[spark] Git Push Summary

2018-09-15 Thread jshao
Repository: spark Updated Tags: refs/tags/v2.3.2-rc6 [created] 02b510728 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[2/2] spark git commit: Preparing development version 2.3.3-SNAPSHOT

2018-09-15 Thread jshao
Preparing development version 2.3.3-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/7b5da37c Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/7b5da37c Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/7b5

[1/2] spark git commit: Preparing Spark release v2.3.2-rc6

2018-09-15 Thread jshao
Repository: spark Updated Branches: refs/heads/branch-2.3 0c1e3d109 -> 7b5da37c0 Preparing Spark release v2.3.2-rc6 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/02b51072 Tree: http://git-wip-us.apache.org/repos/asf/spar

svn commit: r29421 - /dev/spark/v2.3.2-rc6-bin/

2018-09-16 Thread jshao
Author: jshao Date: Sun Sep 16 13:30:43 2018 New Revision: 29421 Log: Apache Spark v2.3.2-rc6 Added: dev/spark/v2.3.2-rc6-bin/ dev/spark/v2.3.2-rc6-bin/SparkR_2.3.2.tar.gz (with props) dev/spark/v2.3.2-rc6-bin/SparkR_2.3.2.tar.gz.asc dev/spark/v2.3.2-rc6-bin/SparkR_2.3.2

svn commit: r29438 - in /dev/spark/v2.3.2-rc6-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _site/api/java/org/apache/spark

2018-09-17 Thread jshao
Author: jshao Date: Mon Sep 17 12:13:30 2018 New Revision: 29438 Log: Apache Spark v2.3.2-rc6 docs [This commit notification would consist of 1447 parts, which exceeds the limit of 50 ones, so it was shortened to the summary

[spark] Git Push Summary

2018-09-24 Thread jshao
Repository: spark Updated Tags: refs/tags/v2.3.2 [created] 02b510728 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[10/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/api/java/JavaPairRDD.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/api/java/JavaPairRDD.html b/site/doc

[44/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/R/dropTempView.html -- diff --git a/site/docs/2.3.2/api/R/dropTempView.html b/site/docs/2.3.2/api/R/dropTempView.html new file mode 100644 inde

[41/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/R/rbind.html -- diff --git a/site/docs/2.3.2/api/R/rbind.html b/site/docs/2.3.2/api/R/rbind.html new file mode 100644 index 000..890ab98 ---

[09/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/api/java/JavaRDD.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/api/java/JavaRDD.html b/site/docs/2.3.2/a

[47/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/R/attach.html -- diff --git a/site/docs/2.3.2/api/R/attach.html b/site/docs/2.3.2/api/R/attach.html new file mode 100644 index 000..3d0058b

[42/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/R/match.html -- diff --git a/site/docs/2.3.2/api/R/match.html b/site/docs/2.3.2/api/R/match.html new file mode 100644 index 000..d405b90 ---

[20/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/SimpleFutureAction.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/SimpleFutureAction.html b/site/docs/2.3

[01/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
Repository: spark-website Updated Branches: refs/heads/asf-site 806a1bd52 -> 04a27dbf1 http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/api/r/PairwiseRRDD.html -- diff

[04/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/api/java/function/ForeachFunction.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/api/java/function/Foreac

[06/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/api/java/JavaSparkStatusTracker.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/api/java/JavaSparkStatusTr

[36/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/R/write.jdbc.html -- diff --git a/site/docs/2.3.2/api/R/write.jdbc.html b/site/docs/2.3.2/api/R/write.jdbc.html new file mode 100644 index

[33/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/constant-values.html -- diff --git a/site/docs/2.3.2/api/java/constant-values.html b/site/docs/2.3.2/api/java/constant-values.html new fil

[08/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/api/java/JavaRDDLike.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/api/java/JavaRDDLike.html b/site/doc

[15/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/TaskCommitDenied.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/TaskCommitDenied.html b/site/docs/2.3.2/a

[37/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/R/summary.html -- diff --git a/site/docs/2.3.2/api/R/summary.html b/site/docs/2.3.2/api/R/summary.html new file mode 100644 index 000..0cd3

[12/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/api/java/JavaFutureAction.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/api/java/JavaFutureAction.html

[48/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/R/00frame_toc.html -- diff --git a/site/docs/2.3.2/api/R/00frame_toc.html b/site/docs/2.3.2/api/R/00frame_toc.html new file mode 100644 index 0

[28/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/AccumulatorParam.IntAccumulatorParam$.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/AccumulatorParam.Int

[50/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/_layouts/global.html -- diff --git a/_layouts/global.html b/_layouts/global.html index 874138f..03a34db 100644 --- a/_layouts/global.html +++ b/_layouts/global.html

[13/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/api/java/JavaDoubleRDD.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/api/java/JavaDoubleRDD.html b/site

[14/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/TaskKilledException.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/TaskKilledException.html b/site/docs/2

[27/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/Aggregator.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/Aggregator.html b/site/docs/2.3.2/api/java/org/

[30/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/lib/jquery.js -- diff --git a/site/docs/2.3.2/api/java/lib/jquery.js b/site/docs/2.3.2/api/java/lib/jquery.js new file mode 100644 index 0

[26/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/CleanShuffle.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/CleanShuffle.html b/site/docs/2.3.2/api/java/

[43/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/R/head.html -- diff --git a/site/docs/2.3.2/api/R/head.html b/site/docs/2.3.2/api/R/head.html new file mode 100644 index 000..30bee19 --- /d

[02/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/api/java/function/package-summary.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/api/java/function/packag

[40/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/R/schema.html -- diff --git a/site/docs/2.3.2/api/R/schema.html b/site/docs/2.3.2/api/R/schema.html new file mode 100644 index 000..9cbcd1a

[31/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/index.html -- diff --git a/site/docs/2.3.2/api/java/index.html b/site/docs/2.3.2/api/java/index.html new file mode 100644 index 000..b

[03/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/api/java/function/MapGroupsWithStateFunction.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/api/java/func

[16/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/SparkStageInfoImpl.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/SparkStageInfoImpl.html b/site/docs/2.3

[11/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/api/java/JavaNewHadoopRDD.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/api/java/JavaNewHadoopRDD.html

[07/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/api/java/JavaSparkContext.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/api/java/JavaSparkContext.html

[05/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/api/java/function/CoGroupFunction.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/api/java/function/CoGrou

[17/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/SparkFirehoseListener.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/SparkFirehoseListener.html b/site/d

[18/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/SparkEnv.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/SparkEnv.html b/site/docs/2.3.2/api/java/org/apac

[38/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/R/spark.survreg.html -- diff --git a/site/docs/2.3.2/api/R/spark.survreg.html b/site/docs/2.3.2/api/R/spark.survreg.html new file mode 100644 i

[23/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/InternalAccumulator.output$.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/InternalAccumulator.output$.ht

[46/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/R/column_datetime_diff_functions.html -- diff --git a/site/docs/2.3.2/api/R/column_datetime_diff_functions.html b/site/docs/2.3.2/api/R/column_

[21/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/Resubmitted.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/Resubmitted.html b/site/docs/2.3.2/api/java/or

[29/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/Accumulable.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/Accumulable.html b/site/docs/2.3.2/api/java/or

[39/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/R/spark.gbt.html -- diff --git a/site/docs/2.3.2/api/R/spark.gbt.html b/site/docs/2.3.2/api/R/spark.gbt.html new file mode 100644 index 000

[24/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/FetchFailed.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/FetchFailed.html b/site/docs/2.3.2/api/java/or

[22/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/NarrowDependency.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/NarrowDependency.html b/site/docs/2.3.2/a

[32/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/index-all.html -- diff --git a/site/docs/2.3.2/api/java/index-all.html b/site/docs/2.3.2/api/java/index-all.html new file mode 100644 inde

[34/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/allclasses-noframe.html -- diff --git a/site/docs/2.3.2/api/java/allclasses-noframe.html b/site/docs/2.3.2/api/java/allclasses-noframe.htm

[25/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/ExceptionFailure.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/ExceptionFailure.html b/site/docs/2.3.2/a

[19/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/org/apache/spark/SparkContext.html -- diff --git a/site/docs/2.3.2/api/java/org/apache/spark/SparkContext.html b/site/docs/2.3.2/api/java/

[35/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/java/allclasses-frame.html -- diff --git a/site/docs/2.3.2/api/java/allclasses-frame.html b/site/docs/2.3.2/api/java/allclasses-frame.html new

[49/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/R/00Index.html -- diff --git a/site/docs/2.3.2/api/R/00Index.html b/site/docs/2.3.2/api/R/00Index.html new file mode 100644 index 000..ec58

[45/51] [partial] spark-website git commit: Add docs for Spark 2.3.2

2018-09-25 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/04a27dbf/site/docs/2.3.2/api/R/cov.html -- diff --git a/site/docs/2.3.2/api/R/cov.html b/site/docs/2.3.2/api/R/cov.html new file mode 100644 index 000..ec96abb --- /dev/

spark-website git commit: Empty commit to trigger asf to github sync

2018-09-26 Thread jshao
Repository: spark-website Updated Branches: refs/heads/asf-site 04a27dbf1 -> 546f35143 Empty commit to trigger asf to github sync Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.apache.org/repos/asf/spark-website/commit/546f3514 Tree: http://git-wi

[1/3] spark-website git commit: Update some missing changes for 2.3.2 release

2018-09-26 Thread jshao
Repository: spark-website Updated Branches: refs/heads/asf-site 546f35143 -> 74d902cdc http://git-wip-us.apache.org/repos/asf/spark-website/blob/74d902cd/site/releases/spark-release-1-3-0.html -- diff --git a/site/releases/spar

[2/3] spark-website git commit: Update some missing changes for 2.3.2 release

2018-09-26 Thread jshao
http://git-wip-us.apache.org/repos/asf/spark-website/blob/74d902cd/site/news/spark-2-3-0-released.html -- diff --git a/site/news/spark-2-3-0-released.html b/site/news/spark-2-3-0-released.html index 022adba..e36b8b3 100644 --- a/s

[3/3] spark-website git commit: Update some missing changes for 2.3.2 release

2018-09-26 Thread jshao
Update some missing changes for 2.3.2 release `downloads.md` and `_post` should be updated to use new 2.3.2 release. Sorry about missing it. Author: jerryshao Closes #150 from jerryshao/update-2.3.2. Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us

spark-website git commit: Update my affiliation

2018-09-27 Thread jshao
Repository: spark-website Updated Branches: refs/heads/asf-site 74d902cdc -> 8b7444182 Update my affiliation Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.apache.org/repos/asf/spark-website/commit/8b744418 Tree: http://git-wip-us.apache.org/repos

svn commit: r27854 - /dev/spark/KEYS

2018-07-02 Thread jshao
Author: jshao Date: Mon Jul 2 12:18:41 2018 New Revision: 27854 Log: Update KEYS Modified: dev/spark/KEYS Modified: dev/spark/KEYS == --- dev/spark/KEYS (original) +++ dev/spark/KEYS Mon Jul 2 12:18:41 2018

[spark] Git Push Summary

2018-07-07 Thread jshao
Repository: spark Updated Tags: refs/tags/v2.3.2-rc1 [created] 4df06b451 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[2/2] spark git commit: Preparing development version 2.3.3-SNAPSHOT

2018-07-07 Thread jshao
Preparing development version 2.3.3-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/72eb97ce Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/72eb97ce Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/72e

[1/2] spark git commit: Preparing Spark release v2.3.2-rc1

2018-07-07 Thread jshao
Repository: spark Updated Branches: refs/heads/branch-2.3 64c72b4de -> 72eb97ce9 Preparing Spark release v2.3.2-rc1 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/4df06b45 Tree: http://git-wip-us.apache.org/repos/asf/spar

svn commit: r27981 - /dev/spark/v2.3.2-rc1-bin/

2018-07-07 Thread jshao
Author: jshao Date: Sun Jul 8 04:41:12 2018 New Revision: 27981 Log: Apache Spark 2.3.2 Added: dev/spark/v2.3.2-rc1-bin/ dev/spark/v2.3.2-rc1-bin/SparkR_2.3.2.tar.gz (with props) dev/spark/v2.3.2-rc1-bin/SparkR_2.3.2.tar.gz.asc dev/spark/v2.3.2-rc1-bin/SparkR_2.3.2

svn commit: r27983 - in /dev/spark/v2.3.2-rc1-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _site/api/java/org/apache/spark

2018-07-08 Thread jshao
Author: jshao Date: Sun Jul 8 07:45:04 2018 New Revision: 27983 Log: Apache Spark v2.3.2-rc1 docs [This commit notification would consist of 1446 parts, which exceeds the limit of 50 ones, so it was shortened to the summary

spark git commit: [SPARK-24646][CORE] Minor change to spark.yarn.dist.forceDownloadSchemes to support wildcard '*'

2018-07-08 Thread jshao
Repository: spark Updated Branches: refs/heads/master 79c668942 -> e2c7e09f7 [SPARK-24646][CORE] Minor change to spark.yarn.dist.forceDownloadSchemes to support wildcard '*' ## What changes were proposed in this pull request? In the case of getting tokens via customized `ServiceCredentialPro

spark git commit: [SPARK-24678][SPARK-STREAMING] Give priority in use of 'PROCESS_LOCAL' for spark-streaming

2018-07-10 Thread jshao
Repository: spark Updated Branches: refs/heads/master a28900956 -> 6fe32869c [SPARK-24678][SPARK-STREAMING] Give priority in use of 'PROCESS_LOCAL' for spark-streaming ## What changes were proposed in this pull request? Currently, `BlockRDD.getPreferredLocations` only get hosts info of bloc

[1/2] spark git commit: Preparing Spark release v2.3.2-rc2

2018-07-10 Thread jshao
Repository: spark Updated Branches: refs/heads/branch-2.3 19542f5de -> 86457a16d Preparing Spark release v2.3.2-rc2 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/307499e1 Tree: http://git-wip-us.apache.org/repos/asf/spar

[spark] Git Push Summary

2018-07-10 Thread jshao
Repository: spark Updated Tags: refs/tags/v2.3.2-rc2 [created] 307499e1a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[2/2] spark git commit: Preparing development version 2.3.3-SNAPSHOT

2018-07-10 Thread jshao
Preparing development version 2.3.3-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/86457a16 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/86457a16 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/864

[2/2] spark git commit: Preparing development version 2.3.3-SNAPSHOT

2018-07-14 Thread jshao
Preparing development version 2.3.3-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/f9a2b0a8 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/f9a2b0a8 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/f9a

[spark] Git Push Summary

2018-07-14 Thread jshao
Repository: spark Updated Tags: refs/tags/v2.3.2-rc3 [created] b3726dadc - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[1/2] spark git commit: Preparing Spark release v2.3.2-rc3

2018-07-14 Thread jshao
Repository: spark Updated Branches: refs/heads/branch-2.3 9cf375f5b -> f9a2b0a87 Preparing Spark release v2.3.2-rc3 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/b3726dad Tree: http://git-wip-us.apache.org/repos/asf/spar

svn commit: r28118 - /dev/spark/v2.3.2-rc3-bin/

2018-07-14 Thread jshao
Author: jshao Date: Sun Jul 15 03:04:30 2018 New Revision: 28118 Log: Apache Spark v2.3.2-rc3 Added: dev/spark/v2.3.2-rc3-bin/ dev/spark/v2.3.2-rc3-bin/SparkR_2.3.2.tar.gz (with props) dev/spark/v2.3.2-rc3-bin/SparkR_2.3.2.tar.gz.asc dev/spark/v2.3.2-rc3-bin/SparkR_2.3.2

  1   2   >