[spark] 01/01: change SparkR maintainer

2021-05-23 Thread felixcheung
This is an automated email from the ASF dual-hosted git repository. felixcheung pushed a commit to branch r-maintainer in repository https://gitbox.apache.org/repos/asf/spark.git commit 5dbe3bbc5d29e2ba6caca022c37eb0b5a2f13202 Author: Felix Cheung AuthorDate: Sun May 23 16:49:27 2021 -0700

[spark] branch r-maintainer created (now 5dbe3bb)

2021-05-23 Thread felixcheung
This is an automated email from the ASF dual-hosted git repository. felixcheung pushed a change to branch r-maintainer in repository https://gitbox.apache.org/repos/asf/spark.git. at 5dbe3bb change SparkR maintainer This branch includes the following new commits: new 5dbe3bb

[spark-website] branch asf-site updated: Add test-hadoop3.2 phrase (#203)

2019-05-12 Thread felixcheung
This is an automated email from the ASF dual-hosted git repository. felixcheung pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/spark-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 16052cb Add test-hadoop3.2

[spark] branch branch-2.3 updated: [R][BACKPORT-2.4] update package description

2019-02-21 Thread felixcheung
This is an automated email from the ASF dual-hosted git repository. felixcheung pushed a commit to branch branch-2.3 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-2.3 by this push: new 6691c04 [R][BACKPORT-2.4] update

[spark] branch branch-2.4 updated: [R][BACKPORT-2.4] update package description

2019-02-21 Thread felixcheung
This is an automated email from the ASF dual-hosted git repository. felixcheung pushed a commit to branch branch-2.4 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-2.4 by this push: new d8576301 [R][BACKPORT-2.4] update

[spark-website] branch asf-site updated: Update Thomas Graves Information (#179)

2019-02-11 Thread felixcheung
This is an automated email from the ASF dual-hosted git repository. felixcheung pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/spark-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 6205847 Update Thomas Graves

[spark] branch master updated: [SPARK-26603][K8S] Update minikube backend

2019-02-03 Thread felixcheung
This is an automated email from the ASF dual-hosted git repository. felixcheung pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 196ca0c [SPARK-26603][K8S] Update minikube

[spark] branch master updated: [SPARK-26642][K8S] Add --num-executors option to spark-submit for Spark on K8S.

2019-01-20 Thread felixcheung
This is an automated email from the ASF dual-hosted git repository. felixcheung pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/master by this push: new 6c18d8d [SPARK-26642][K8S] Add --num

[spark] branch branch-2.3 updated: [SPARK-26120][TESTS][SS][SPARKR] Fix a streaming query leak in Structured Streaming R tests

2019-01-12 Thread felixcheung
This is an automated email from the ASF dual-hosted git repository. felixcheung pushed a commit to branch branch-2.3 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-2.3 by this push: new 1979712 [SPARK-26120][TESTS][SS

[spark-website] branch asf-site updated: add checker note (#168)

2019-01-12 Thread felixcheung
This is an automated email from the ASF dual-hosted git repository. felixcheung pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/spark-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 70a6071 add checker note (#168

[spark] branch branch-2.3 updated: [SPARK-26010][R] fix vignette eval with Java 11

2019-01-12 Thread felixcheung
This is an automated email from the ASF dual-hosted git repository. felixcheung pushed a commit to branch branch-2.3 in repository https://gitbox.apache.org/repos/asf/spark.git The following commit(s) were added to refs/heads/branch-2.3 by this push: new a9a1bc7 [SPARK-26010][R] fix

spark git commit: [SPARK-26189][R] Fix unionAll doc in SparkR

2018-11-30 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 28d337440 -> 2f6e88fec [SPARK-26189][R] Fix unionAll doc in SparkR ## What changes were proposed in this pull request? Fix unionAll doc in SparkR ## How was this patch tested? Manually ran test Author: Huaxin Gao Closes #23161 from hu

spark git commit: [SPARK-26010][R] fix vignette eval with Java 11

2018-11-12 Thread felixcheung
ses #23007 from felixcheung/rjavavervig. (cherry picked from commit 88c82627267a9731b2438f0cc28dd656eb3dc834) Signed-off-by: Felix Cheung Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/65e5b265 Tree: http://git-

spark git commit: [SPARK-26010][R] fix vignette eval with Java 11

2018-11-12 Thread felixcheung
ses #23007 from felixcheung/rjavavervig. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/88c82627 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/88c82627 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/88c82

spark git commit: [SPARK-25859][ML] add scala/java/python example and doc for PrefixSpan

2018-10-27 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.4 313a1f0a7 -> f575616db [SPARK-25859][ML] add scala/java/python example and doc for PrefixSpan ## What changes were proposed in this pull request? add scala/java/python example and doc for PrefixSpan in branch 2.4 ## How was this patch

spark git commit: [SPARK-16693][SPARKR] Remove methods deprecated

2018-10-27 Thread felixcheung
ile registerTempTable saveAsParquetFile unionAll createExternalTable dropTempTable ## How was this patch tested? jenkins Author: Felix Cheung Closes #22843 from felixcheung/rrddapi. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/com

spark git commit: [SPARK-24572][SPARKR] "eager execution" for R shell, IDE

2018-10-24 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 19ada15d1 -> ddd1b1e8a [SPARK-24572][SPARKR] "eager execution" for R shell, IDE ## What changes were proposed in this pull request? Check the `spark.sql.repl.eagerEval.enabled` configuration property in SparkDataFrame `show()` method. If

spark git commit: [SPARK-24516][K8S] Change Python default to Python3

2018-10-24 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master b2e325625 -> 19ada15d1 [SPARK-24516][K8S] Change Python default to Python3 ## What changes were proposed in this pull request? As this is targeted for 3.0.0 and Python2 will be deprecated by Jan 1st, 2020, I feel it is appropriate to chan

spark git commit: [SPARK-24207][R] add R API for PrefixSpan

2018-10-21 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 4c6c6711d -> fc64e83f9 [SPARK-24207][R] add R API for PrefixSpan ## What changes were proposed in this pull request? add R API for PrefixSpan ## How was this patch tested? add test in test_mllib_fpm.R Author: Huaxin Gao Closes #21710 f

spark git commit: [SPARK-25675][SPARK JOB HISTORY] Job UI page does not show pagination with one page

2018-10-21 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master ffe256ce1 -> 4c6c6711d [SPARK-25675][SPARK JOB HISTORY] Job UI page does not show pagination with one page ## What changes were proposed in this pull request? Currently in PagedTable.scala pageNavigation() method, if it is having only one

spark git commit: [SPARK-25730][K8S] Delete executor pods from kubernetes after figuring out why they died

2018-10-21 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master c77aa42f5 -> ffe256ce1 [SPARK-25730][K8S] Delete executor pods from kubernetes after figuring out why they died ## What changes were proposed in this pull request? `removeExecutorFromSpark` tries to fetch the reason the executor exited fr

spark git commit: [SPARK-25572][SPARKR] test only if not cran

2018-09-29 Thread felixcheung
on Java 10, which unfortunately Spark does not start on. For 2.4, lets attempt skipping all tests ## How was this patch tested? manual, jenkins, appveyor Author: Felix Cheung Closes #22589 from felixcheung/ralltests. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http:/

spark git commit: [SPARK-25572][SPARKR] test only if not cran

2018-09-29 Thread felixcheung
s run on Java 10, which unfortunately Spark does not start on. For 2.4, lets attempt skipping all tests ## How was this patch tested? manual, jenkins, appveyor Author: Felix Cheung Closes #22589 from felixcheung/ralltests. (cherry picked from commit f4b138082ff91be74b0f5bbe19cdb90dd9e5f131) Si

spark git commit: [SPARK-23648][R][SQL] Adds more types for hint in SparkR

2018-09-19 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 76399d75e -> 95b177c8f [SPARK-23648][R][SQL] Adds more types for hint in SparkR ## What changes were proposed in this pull request? Addition of numeric and list hints for SparkR. ## How was this patch tested? Add test in test_sparkSQL.R

spark git commit: [MINOR][DOCS] Axe deprecated doc refs

2018-09-16 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.4 60af706b4 -> 1cb1e4301 [MINOR][DOCS] Axe deprecated doc refs Continuation of #22370. Summary of discussion there: There is some inconsistency in the R manual w.r.t. supercedent functions linking back to deprecated functions. - `crea

spark git commit: [MINOR][DOCS] Axe deprecated doc refs

2018-09-16 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master bfcf74260 -> a1dd78255 [MINOR][DOCS] Axe deprecated doc refs Continuation of #22370. Summary of discussion there: There is some inconsistency in the R manual w.r.t. supercedent functions linking back to deprecated functions. - `createOr

spark git commit: [SPARK-25167][SPARKR][TEST][MINOR] Minor fixes for R sql tests (timestamp comparison)

2018-09-03 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 64bbd134e -> 39d3d6cc9 [SPARK-25167][SPARKR][TEST][MINOR] Minor fixes for R sql tests (timestamp comparison) ## What changes were proposed in this pull request? The "date function on DataFrame" test fails consistently on my laptop. In this

spark git commit: [SPARK-25007][R] Add array_intersect/array_except/array_union/shuffle to SparkR

2018-09-02 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master a3dccd24c -> a481794ca [SPARK-25007][R] Add array_intersect/array_except/array_union/shuffle to SparkR ## What changes were proposed in this pull request? Add the R version of array_intersect/array_except/array_union/shuffle ## How was th

spark git commit: [SPARK-25117][R] Add EXEPT ALL and INTERSECT ALL support in R

2018-08-17 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master c1ffb3c10 -> 162326c0e [SPARK-25117][R] Add EXEPT ALL and INTERSECT ALL support in R ## What changes were proposed in this pull request? [SPARK-21274](https://issues.apache.org/jira/browse/SPARK-21274) added support for EXCEPT ALL and INTE

spark git commit: [TRIVIAL][ML] GMM unpersist RDD after training

2018-07-15 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master bbc2ffc8a -> bcf7121ed [TRIVIAL][ML] GMM unpersist RDD after training ## What changes were proposed in this pull request? unpersist `instances` after training ## How was this patch tested? existing tests Author: 郑瑞峰 Closes #21562

spark git commit: [SPARK-23461][R] vignettes should include model predictions for some ML models

2018-07-10 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 5ff1b9ba1 -> 006e798e4 [SPARK-23461][R] vignettes should include model predictions for some ML models ## What changes were proposed in this pull request? Add model predictions for Linear Support Vector Machine (SVM) Classifier, Logistic R

spark git commit: [SPARK-23529][K8S] Support mounting volumes

2018-07-10 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 74a8d6308 -> 5ff1b9ba1 [SPARK-23529][K8S] Support mounting volumes This PR continues #21095 and intersects with #21238. I've added volume mounts as a separate step and added PersistantVolumeClaim support. There is a fundamental problem wi

spark git commit: [SPARK-24535][SPARKR] fix tests on java check error

2018-07-06 Thread felixcheung
h tested? unit test, manual, win-builder Author: Felix Cheung Closes #21666 from felixcheung/rjavaskip. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/141953f4 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/14195

spark git commit: [SPARK-24535][SPARKR] fix tests on java check error

2018-07-06 Thread felixcheung
is patch tested? unit test, manual, win-builder Author: Felix Cheung Closes #21666 from felixcheung/rjavaskip. (cherry picked from commit 141953f4c44dbad1c2a7059e92bec5fe770af932) Signed-off-by: Felix Cheung Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.ap

spark git commit: [SPARK-23668][K8S] Added missing config property in running-on-kubernetes.md

2018-06-01 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 8ef167a5f -> a36c1a6bb [SPARK-23668][K8S] Added missing config property in running-on-kubernetes.md ## What changes were proposed in this pull request? PR https://github.com/apache/spark/pull/20811 introduced a new Spark configuration prop

spark git commit: [SPARK-24326][MESOS] add support for local:// scheme for the app jar

2018-05-31 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master b2d022656 -> 22df953f6 [SPARK-24326][MESOS] add support for local:// scheme for the app jar ## What changes were proposed in this pull request? * Adds support for local:// scheme like in k8s case for image based deployments where the jar

spark git commit: [SPARK-24331][SPARKR][SQL] Adding arrays_overlap, array_repeat, map_entries to SparkR

2018-05-29 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master f48938800 -> a4be981c0 [SPARK-24331][SPARKR][SQL] Adding arrays_overlap, array_repeat, map_entries to SparkR ## What changes were proposed in this pull request? The PR adds functions `arrays_overlap`, `array_repeat`, `map_entries` to Spar

spark git commit: [R][BACKPORT-2.2] backport lint fix

2018-05-15 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.1 1d569f684 -> b51ce9377 [R][BACKPORT-2.2] backport lint fix ## What changes were proposed in this pull request? backport part of the commit that addresses lintr issue Author: Felix Cheung Closes #21325 from felixcheung/rlintfi

spark git commit: [R][BACKPORT-2.2] backport lint fix

2018-05-15 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.2 f96d13dd6 -> 8c223b651 [R][BACKPORT-2.2] backport lint fix ## What changes were proposed in this pull request? backport part of the commit that addresses lintr issue Author: Felix Cheung Closes #21325 from felixcheung/rlintfi

spark git commit: [SPARK-23780][R] Failed to use googleVis library with new SparkR

2018-05-14 Thread felixcheung
ted? manual test, unit tests Author: Felix Cheung Closes #21315 from felixcheung/googvis. (cherry picked from commit 9059f1ee6ae13c8636c9b7fdbb708a349256fb8e) Signed-off-by: Felix Cheung Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/

spark git commit: [SPARK-23780][R] Failed to use googleVis library with new SparkR

2018-05-14 Thread felixcheung
ted? manual test, unit tests Author: Felix Cheung Closes #21315 from felixcheung/googvis. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/9059f1ee Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/9059f1ee Diff: http://git-

spark-website git commit: add merge script

2018-04-10 Thread felixcheung
Repository: spark-website Updated Branches: refs/heads/asf-site b75134190 -> 91b561749 add merge script Ported the merge script from spark main repo Author: Felix Cheung Closes #105 from felixcheung/merge. Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: h

spark git commit: [SPARK-23291][SQL][R] R's substr should not reduce starting position by 1 when calling Scala API

2018-03-07 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master aff7d81cb -> 53561d27c [SPARK-23291][SQL][R] R's substr should not reduce starting position by 1 when calling Scala API ## What changes were proposed in this pull request? Seems R's substr API treats Scala substr API as zero based and so

[2/3] spark git commit: [SPARK-22430][R][DOCS] Unknown tag warnings when building R docs with Roxygen 6.0.1

2018-03-05 Thread felixcheung
http://git-wip-us.apache.org/repos/asf/spark/blob/4586eada/R/pkg/R/generics.R -- diff --git a/R/pkg/R/generics.R b/R/pkg/R/generics.R index e0dde33..6fba4b6 100644 --- a/R/pkg/R/generics.R +++ b/R/pkg/R/generics.R @@ -19,7 +19,6 @@

[1/3] spark git commit: [SPARK-22430][R][DOCS] Unknown tag warnings when building R docs with Roxygen 6.0.1

2018-03-05 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 947b4e6f0 -> 4586eada4 http://git-wip-us.apache.org/repos/asf/spark/blob/4586eada/R/pkg/R/mllib_tree.R -- diff --git a/R/pkg/R/mllib_tree.R b/R/pkg/R/mllib_tree.R index 4e5d

[3/3] spark git commit: [SPARK-22430][R][DOCS] Unknown tag warnings when building R docs with Roxygen 6.0.1

2018-03-05 Thread felixcheung
[SPARK-22430][R][DOCS] Unknown tag warnings when building R docs with Roxygen 6.0.1 ## What changes were proposed in this pull request? Removed export tag to get rid of unknown tag warnings ## How was this patch tested? Existing tests Author: Rekha Joshi Author: rjoshi2 Closes #20501 from re

spark-website git commit: update committer

2018-03-05 Thread felixcheung
Repository: spark-website Updated Branches: refs/heads/asf-site 32ff6fa97 -> 8bd24fb6d update committer Author: Felix Cheung Closes #103 from felixcheung/fc. Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-wip-us.apache.org/repos/asf/spark-webs

spark git commit: [SPARKR][DOC] fix link in vignettes

2018-03-02 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.3 56cfbd932 -> 8fe20e151 [SPARKR][DOC] fix link in vignettes ## What changes were proposed in this pull request? Fix doc link that was changed in 2.3 shivaram Author: Felix Cheung Closes #20711 from felixcheung/rvigmean. (che

spark git commit: [SPARKR][DOC] fix link in vignettes

2018-03-02 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 119f6a0e4 -> 0b6ceadeb [SPARKR][DOC] fix link in vignettes ## What changes were proposed in this pull request? Fix doc link that was changed in 2.3 shivaram Author: Felix Cheung Closes #20711 from felixcheung/rvigmean. Project: h

spark git commit: [SPARK-23163][DOC][PYTHON] Sync ML Python API with Scala

2018-01-25 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master e29b08add -> 39ee2acf9 [SPARK-23163][DOC][PYTHON] Sync ML Python API with Scala ## What changes were proposed in this pull request? This syncs the ML Python API with Scala for differences found after the 2.3 QA audit. ## How was this pat

spark git commit: [SPARK-23163][DOC][PYTHON] Sync ML Python API with Scala

2018-01-25 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.3 abd3e1bb5 -> e66c66cd2 [SPARK-23163][DOC][PYTHON] Sync ML Python API with Scala ## What changes were proposed in this pull request? This syncs the ML Python API with Scala for differences found after the 2.3 QA audit. ## How was this

spark git commit: [SPARK-20906][SPARKR] Add API doc example for Constrained Logistic Regression

2018-01-24 Thread felixcheung
ung Closes #20380 from felixcheung/rclrdoc. (cherry picked from commit e18d6f5326e0d9ea03d31de5ce04cb84d3b8ab37) Signed-off-by: Felix Cheung Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/4336e67f Tree: http://git-

spark git commit: [SPARK-20906][SPARKR] Add API doc example for Constrained Logistic Regression

2018-01-24 Thread felixcheung
ung Closes #20380 from felixcheung/rclrdoc. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/e18d6f53 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/e18d6f53 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/e18d6

spark git commit: [SPARK-21727][R] Allow multi-element atomic vector as column type in SparkR DataFrame

2018-01-23 Thread felixcheung
Df$data <- list(rep(0, 20)) mySparkDf <- as.DataFrame(myDf) collect(mySparkDf) ``` [2018-01-22 SPARK-21727 Test Results.txt](https://github.com/apache/spark/files/1653535/2018-01-22.SPARK-21727.Test.Results.txt) felixcheung yanboliang sun-rui shivaram _The contribution is my original work

spark git commit: [SPARK-21727][R] Allow multi-element atomic vector as column type in SparkR DataFrame

2018-01-23 Thread felixcheung
$data <- list(rep(0, 20)) mySparkDf <- as.DataFrame(myDf) collect(mySparkDf) ``` [2018-01-22 SPARK-21727 Test Results.txt](https://github.com/apache/spark/files/1653535/2018-01-22.SPARK-21727.Test.Results.txt) felixcheung yanboliang sun-rui shivaram _The contribution is my original work and

spark git commit: [SPARK-21293][SS][SPARKR] Add doc example for streaming join, dedup

2018-01-21 Thread felixcheung
lix Cheung Closes #20340 from felixcheung/rstreamdoc. (cherry picked from commit 2239d7a410e906ccd40aa8e84d637e9d06cd7b8a) Signed-off-by: Felix Cheung Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/7520491b Tree: http://

spark git commit: [SPARK-21293][SS][SPARKR] Add doc example for streaming join, dedup

2018-01-21 Thread felixcheung
lix Cheung Closes #20340 from felixcheung/rstreamdoc. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/2239d7a4 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/2239d7a4 Diff: http://git-wip-us.apache.org/repos/asf/spark/d

spark git commit: [SPARK-23038][TEST] Update docker/spark-test (JDK/OS)

2018-01-13 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.2 105ae8680 -> 7022ef800 [SPARK-23038][TEST] Update docker/spark-test (JDK/OS) ## What changes were proposed in this pull request? This PR aims to update the followings in `docker/spark-test`. - JDK7 -> JDK8 Spark 2.2+ supports JDK8 onl

spark git commit: [SPARK-23038][TEST] Update docker/spark-test (JDK/OS)

2018-01-13 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.3 1f4a08b15 -> a335a49ce [SPARK-23038][TEST] Update docker/spark-test (JDK/OS) ## What changes were proposed in this pull request? This PR aims to update the followings in `docker/spark-test`. - JDK7 -> JDK8 Spark 2.2+ supports JDK8 onl

spark git commit: [SPARK-23038][TEST] Update docker/spark-test (JDK/OS)

2018-01-13 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master c3548d11c -> 7a3d0aad2 [SPARK-23038][TEST] Update docker/spark-test (JDK/OS) ## What changes were proposed in this pull request? This PR aims to update the followings in `docker/spark-test`. - JDK7 -> JDK8 Spark 2.2+ supports JDK8 only.

spark git commit: [SPARK-23063][K8S] K8s changes for publishing scripts (and a couple of other misses)

2018-01-13 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master afae8f2bc -> c3548d11c [SPARK-23063][K8S] K8s changes for publishing scripts (and a couple of other misses) ## What changes were proposed in this pull request? Including the `-Pkubernetes` flag in a few places it was missed. ## How was t

spark git commit: [SPARK-23063][K8S] K8s changes for publishing scripts (and a couple of other misses)

2018-01-13 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.3 bcd87ae07 -> 1f4a08b15 [SPARK-23063][K8S] K8s changes for publishing scripts (and a couple of other misses) ## What changes were proposed in this pull request? Including the `-Pkubernetes` flag in a few places it was missed. ## How w

spark git commit: [SPARK-22993][ML] Clarify HasCheckpointInterval param doc

2018-01-09 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.3 ecc24ec7f -> 2db523959 [SPARK-22993][ML] Clarify HasCheckpointInterval param doc ## What changes were proposed in this pull request? Add a note to the `HasCheckpointInterval` parameter doc that clarifies that this setting is ignored w

spark git commit: [SPARK-22993][ML] Clarify HasCheckpointInterval param doc

2018-01-09 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master eaac60a1e -> 70bcc9d5a [SPARK-22993][ML] Clarify HasCheckpointInterval param doc ## What changes were proposed in this pull request? Add a note to the `HasCheckpointInterval` parameter doc that clarifies that this setting is ignored when

spark git commit: [SPARK-22998][K8S] Set missing value for SPARK_MOUNTED_CLASSPATH in the executors

2018-01-09 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 0959aa581 -> 6a4206ff0 [SPARK-22998][K8S] Set missing value for SPARK_MOUNTED_CLASSPATH in the executors ## What changes were proposed in this pull request? The environment variable `SPARK_MOUNTED_CLASSPATH` is referenced in the executor

spark git commit: [SPARK-22998][K8S] Set missing value for SPARK_MOUNTED_CLASSPATH in the executors

2018-01-09 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.3 e79480e5d -> 47f975b42 [SPARK-22998][K8S] Set missing value for SPARK_MOUNTED_CLASSPATH in the executors ## What changes were proposed in this pull request? The environment variable `SPARK_MOUNTED_CLASSPATH` is referenced in the exec

spark git commit: [SPARK-21293][SPARKR][DOCS] structured streaming doc update

2018-01-08 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.3 911a4dbe7 -> a23c07ecb [SPARK-21293][SPARKR][DOCS] structured streaming doc update ## What changes were proposed in this pull request? doc update Author: Felix Cheung Closes #20197 from felixcheung/rwadoc. (cherry picked f

spark git commit: [SPARK-21293][SPARKR][DOCS] structured streaming doc update

2018-01-08 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 8486ad419 -> 02214b094 [SPARK-21293][SPARKR][DOCS] structured streaming doc update ## What changes were proposed in this pull request? doc update Author: Felix Cheung Closes #20197 from felixcheung/rwadoc. Project: http://git-

spark git commit: [SPARK-21292][DOCS] refreshtable example

2018-01-08 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.3 fd46a276c -> 911a4dbe7 [SPARK-21292][DOCS] refreshtable example ## What changes were proposed in this pull request? doc update Author: Felix Cheung Closes #20198 from felixcheung/rrefreshdoc. (cherry picked from com

spark git commit: [SPARK-21292][DOCS] refreshtable example

2018-01-08 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master f20131dd3 -> 8486ad419 [SPARK-21292][DOCS] refreshtable example ## What changes were proposed in this pull request? doc update Author: Felix Cheung Closes #20198 from felixcheung/rrefreshdoc. Project: http://git-wip-us.apache.

spark git commit: [SPARK-22757][K8S] Enable spark.jars and spark.files in KUBERNETES mode

2018-01-04 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.3 5b524cc0c -> f9dcdbcef [SPARK-22757][K8S] Enable spark.jars and spark.files in KUBERNETES mode ## What changes were proposed in this pull request? We missed enabling `spark.files` and `spark.jars` in https://github.com/apache/spark/pu

spark git commit: [SPARK-22757][K8S] Enable spark.jars and spark.files in KUBERNETES mode

2018-01-04 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master cf0aa6557 -> 6cff7d19f [SPARK-22757][K8S] Enable spark.jars and spark.files in KUBERNETES mode ## What changes were proposed in this pull request? We missed enabling `spark.files` and `spark.jars` in https://github.com/apache/spark/pull/1

spark git commit: [SPARK-22933][SPARKR] R Structured Streaming API for withWatermark, trigger, partitionBy

2018-01-03 Thread felixcheung
nBy ## How was this patch tested? manual, unit tests Author: Felix Cheung Closes #20129 from felixcheung/rwater. (cherry picked from commit df95a908baf78800556636a76d58bba9b3dd943f) Signed-off-by: Felix Cheung Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-

spark git commit: [SPARK-22933][SPARKR] R Structured Streaming API for withWatermark, trigger, partitionBy

2018-01-03 Thread felixcheung
nBy ## How was this patch tested? manual, unit tests Author: Felix Cheung Closes #20129 from felixcheung/rwater. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/df95a908 Tree: http://git-wip-us.apache.org/repos/asf/spark/t

spark git commit: [SPARK-22920][SPARKR] sql functions for current_date, current_timestamp, rtrim/ltrim/trim with trimString

2017-12-29 Thread felixcheung
ual, unit tests Author: Felix Cheung Closes #20105 from felixcheung/rsqlfuncs. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/66a7d6b3 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/66a7d6b3 Diff: http://git-

spark git commit: [SPARK-22889][SPARKR] Set overwrite=T when install SparkR in tests

2017-12-23 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.2 7a97943f2 -> 41f705a57 [SPARK-22889][SPARKR] Set overwrite=T when install SparkR in tests ## What changes were proposed in this pull request? Since all CRAN checks go through the same machine, if there is an older partial download or

spark git commit: [SPARK-22889][SPARKR] Set overwrite=T when install SparkR in tests

2017-12-23 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master aeb45df66 -> 1219d7a43 [SPARK-22889][SPARKR] Set overwrite=T when install SparkR in tests ## What changes were proposed in this pull request? Since all CRAN checks go through the same machine, if there is an older partial download or part

[spark] Git Push Summary

2017-12-06 Thread felixcheung
Repository: spark Updated Tags: refs/tags/v2.2.1 [created] e30e2698a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

svn commit: r23279 - in /dev/spark/spark-2.2.1-rc2-docs: ./ _site/ _site/api/ _site/api/R/ _site/api/java/ _site/api/java/lib/ _site/api/java/org/ _site/api/java/org/apache/ _site/api/java/org/apache/

2017-11-24 Thread felixcheung
Author: felixcheung Date: Sat Nov 25 03:01:04 2017 New Revision: 23279 Log: Apache Spark spark-2.2.1-rc2 docs [This commit notification would consist of 1388 parts, which exceeds the limit of 50 ones, so it was shortened to the summary

svn commit: r23278 - /dev/spark/spark-2.2.1-rc2-bin/

2017-11-24 Thread felixcheung
Author: felixcheung Date: Sat Nov 25 02:44:26 2017 New Revision: 23278 Log: Apache Spark spark-2.2.1-rc2 Added: dev/spark/spark-2.2.1-rc2-bin/ dev/spark/spark-2.2.1-rc2-bin/SparkR_2.2.1.tar.gz (with props) dev/spark/spark-2.2.1-rc2-bin/SparkR_2.2.1.tar.gz.asc dev/spark/spark

[1/2] spark git commit: Preparing Spark release v2.2.1-rc2

2017-11-24 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.2 c3b5df22a -> 455cea622 Preparing Spark release v2.2.1-rc2 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/e30e2698 Tree: http://git-wip-us.apache.org/repos/asf/spar

[2/2] spark git commit: Preparing development version 2.2.2-SNAPSHOT

2017-11-24 Thread felixcheung
Preparing development version 2.2.2-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/455cea62 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/455cea62 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/455

[spark] Git Push Summary

2017-11-24 Thread felixcheung
Repository: spark Updated Tags: refs/tags/v2.2.1-rc2 [created] e30e2698a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[2/2] spark git commit: fix typo

2017-11-24 Thread felixcheung
fix typo Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c3b5df22 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/c3b5df22 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/c3b5df22 Branch: refs/heads/branch-2.2

[1/2] spark git commit: [SPARK-22495] Fix setup of SPARK_HOME variable on Windows

2017-11-24 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.2 ad57141f9 -> c3b5df22a [SPARK-22495] Fix setup of SPARK_HOME variable on Windows ## What changes were proposed in this pull request? This is a cherry pick of the original PR 19370 onto branch-2.2 as suggested in https://github.com/apa

svn commit: r23090 - in /dev/spark: ./ spark-2.2.1-rc1-bin/ spark-2.2.1-rc1-docs/ spark-2.2.1-rc1-docs/_site/ spark-2.2.1-rc1-docs/_site/api/ spark-2.2.1-rc1-docs/_site/api/R/ spark-2.2.1-rc1-docs/_si

2017-11-13 Thread felixcheung
Author: felixcheung Date: Tue Nov 14 04:30:04 2017 New Revision: 23090 Log: Apache Spark spark-2.2.1-rc1 [This commit notification would consist of 1388 parts, which exceeds the limit of 50 ones, so it was shortened to the summary

[2/2] spark git commit: Preparing development version 2.2.2-SNAPSHOT

2017-11-13 Thread felixcheung
Preparing development version 2.2.2-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/af0b1855 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/af0b1855 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/af0

[1/2] spark git commit: Preparing Spark release v2.2.1-rc1

2017-11-13 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.2 c68b4c54f -> af0b1855f Preparing Spark release v2.2.1-rc1 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/41116ab7 Tree: http://git-wip-us.apache.org/repos/asf/spar

[spark] Git Push Summary

2017-11-13 Thread felixcheung
Repository: spark Updated Tags: refs/tags/v2.2.1-rc1 [created] 41116ab7f - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[spark] Git Push Summary

2017-11-13 Thread felixcheung
Repository: spark Updated Tags: refs/tags/v2.2.1-rc1 [deleted] 124b9a106 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[spark] Git Push Summary [forced push!] [Forced Update!]

2017-11-13 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.2 b1f8c84ef -> c68b4c54f (forced update) - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[spark] Git Push Summary

2017-11-13 Thread felixcheung
Repository: spark Updated Tags: refs/tags/v2.2.1-rc1 [created] 124b9a106 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[2/2] spark git commit: Preparing development version 2.2.2-SNAPSHOT

2017-11-13 Thread felixcheung
Preparing development version 2.2.2-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/b1f8c84e Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/b1f8c84e Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/b1f

[1/2] spark git commit: Preparing Spark release v2.2.1-rc1

2017-11-13 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.2 c68b4c54f -> b1f8c84ef Preparing Spark release v2.2.1-rc1 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/124b9a10 Tree: http://git-wip-us.apache.org/repos/asf/spar

spark git commit: [SPARK-22442][SQL][BRANCH-2.2] ScalaReflection should produce correct field names for special characters

2017-11-12 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.2 8acd02f42 -> f73637798 [SPARK-22442][SQL][BRANCH-2.2] ScalaReflection should produce correct field names for special characters ## What changes were proposed in this pull request? For a class with field name of special characters, e.g

spark git commit: [SPARK-21694][R][ML] Reduce max iterations in Linear SVM test in R to speed up AppVeyor build

2017-11-12 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.2 2a04cfaad -> 8acd02f42 [SPARK-21694][R][ML] Reduce max iterations in Linear SVM test in R to speed up AppVeyor build This PR proposes to reduce max iteration in Linear SVM test in SparkR. This particular test elapses roughly 5 mins on

spark git commit: [SPARK-21693][R][ML] Reduce max iterations in Linear SVM test in R to speed up AppVeyor build

2017-11-12 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 9bf696dbe -> 3d90b2cb3 [SPARK-21693][R][ML] Reduce max iterations in Linear SVM test in R to speed up AppVeyor build ## What changes were proposed in this pull request? This PR proposes to reduce max iteration in Linear SVM test in SparkR

spark git commit: [SPARK-19606][BUILD][BACKPORT-2.2][MESOS] fix mesos break

2017-11-12 Thread felixcheung
ses #19732 from felixcheung/fixmesosdriverconstraint. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/2a04cfaa Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/2a04cfaa Diff: http://git-wip-us.apache.org/repos/asf/spark/d

spark git commit: [SPARK-19606][MESOS] Support constraints in spark-dispatcher

2017-11-12 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.2 4ef0bef9e -> f6ee3d90d [SPARK-19606][MESOS] Support constraints in spark-dispatcher A discussed in SPARK-19606, the addition of a new config property named "spark.mesos.constraints.driver" for constraining drivers running on a Mesos c

spark git commit: [SPARK-19606][MESOS] Support constraints in spark-dispatcher

2017-11-12 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master 21a7bfd5c -> b3f9dbf48 [SPARK-19606][MESOS] Support constraints in spark-dispatcher ## What changes were proposed in this pull request? A discussed in SPARK-19606, the addition of a new config property named "spark.mesos.constraints.drive

  1   2   3   4   >