[GitHub] spark issue #21199: [SPARK-24127][SS] Continuous text socket source

2018-05-13 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21199 I was thinking if it is too overkill to receive data in the driver side and publish them to the executors via RPC? This might give user a wrong impression that data should be received in the drive

[GitHub] spark issue #21222: [SPARK-24161][SS] Enable debug package feature on struct...

2018-05-13 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/21222 Kindly ping. I guess debugging last batch might not be attractive that much, but printing codegen would be helpful to someone who want to investigate or debug in detail. --- -

[GitHub] spark issue #21290: [SPARK-24241][Submit]Do not fail fast when dynamic resou...

2018-05-13 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21290 LGTM, just some minor comments. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-m

[GitHub] spark pull request #21290: [SPARK-24241][Submit]Do not fail fast when dynami...

2018-05-13 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21290#discussion_r187847736 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala --- @@ -76,6 +75,7 @@ private[deploy] class SparkSubmitArguments(args: S

[GitHub] spark pull request #21290: [SPARK-24241][Submit]Do not fail fast when dynami...

2018-05-13 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21290#discussion_r187847656 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala --- @@ -180,6 +180,25 @@ class SparkSubmitSuite appArgs.toString sho

[GitHub] spark issue #21312: [SPARK-24259][SQL] ArrayWriter for Arrow produces wrong ...

2018-05-13 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21312 Thanks for catching this @viirya! Looks good from a first glance, but my only concern is that `clear()` will release the vector buffers, where `reset()` just zeros them out. Let me look into t

[GitHub] spark issue #21318: [minor] Update docs for functions.scala to make it clear...

2018-05-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21318 @rxin, should we maybe we mention that SQL functions are usually added to match other DBMSs (unlike functions.scala)? --- -

[GitHub] spark pull request #21318: [minor] Update docs for functions.scala to make i...

2018-05-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21318#discussion_r187843889 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -39,7 +39,21 @@ import org.apache.spark.util.Utils /**

[GitHub] spark pull request #21318: [minor] Update docs for functions.scala to make i...

2018-05-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21318#discussion_r187843283 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -39,7 +39,21 @@ import org.apache.spark.util.Utils /**

[GitHub] spark pull request #21318: [minor] Update docs for functions.scala to make i...

2018-05-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21318#discussion_r187843125 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -39,7 +39,21 @@ import org.apache.spark.util.Utils /**

[GitHub] spark issue #21299: [SPARK-24250][SQL] support accessing SQLConf inside task...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21299 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3188/ Tes

[GitHub] spark issue #21299: [SPARK-24250][SQL] support accessing SQLConf inside task...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21299 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21299: [SPARK-24250][SQL] support accessing SQLConf inside task...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21299 **[Test build #90568 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90568/testReport)** for PR 21299 at commit [`a100dea`](https://github.com/apache/spark/commit/a1

[GitHub] spark pull request #21318: [minor] Update docs for functions.scala to make i...

2018-05-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21318#discussion_r187842050 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -39,7 +39,21 @@ import org.apache.spark.util.Utils /**

[GitHub] spark pull request #21299: [SPARK-24250][SQL] support accessing SQLConf insi...

2018-05-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21299#discussion_r187841313 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SQLExecution.scala --- @@ -90,13 +92,33 @@ object SQLExecution { * thread from th

[GitHub] spark issue #21318: [minor] Update docs for functions.scala to make it clear...

2018-05-13 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/21318 Hm the failure doesn't look like it's caused by this PR. Do you guys know what's going on? --- - To unsubscribe, e-mail: reviews-u

[GitHub] spark issue #21318: [minor] Update docs for functions.scala to make it clear...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21318 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21246: [SPARK-23901][SQL] Add masking functions

2018-05-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21246#discussion_r187837123 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/maskExpressions.scala --- @@ -0,0 +1,569 @@ +/* + * Licensed to the

[GitHub] spark pull request #21246: [SPARK-23901][SQL] Add masking functions

2018-05-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21246#discussion_r187835032 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/maskExpressions.scala --- @@ -0,0 +1,569 @@ +/* + * Licensed to the

[GitHub] spark pull request #21246: [SPARK-23901][SQL] Add masking functions

2018-05-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21246#discussion_r187839186 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/MaskExpressionsSuite.scala --- @@ -0,0 +1,236 @@ +/* + * Licensed to

[GitHub] spark pull request #21246: [SPARK-23901][SQL] Add masking functions

2018-05-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21246#discussion_r187835331 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/MaskExpressionsUtils.java --- @@ -0,0 +1,80 @@ +/* + * Licensed to th

[GitHub] spark pull request #21246: [SPARK-23901][SQL] Add masking functions

2018-05-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21246#discussion_r187835834 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/maskExpressions.scala --- @@ -0,0 +1,569 @@ +/* + * Licensed to the

[GitHub] spark issue #21318: [minor] Update docs for functions.scala to make it clear...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21318 **[Test build #90567 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90567/testReport)** for PR 21318 at commit [`83c191f`](https://github.com/apache/spark/commit/8

[GitHub] spark pull request #21246: [SPARK-23901][SQL] Add masking functions

2018-05-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21246#discussion_r187839605 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/MaskExpressionsSuite.scala --- @@ -0,0 +1,236 @@ +/* + * Licensed to

[GitHub] spark pull request #21246: [SPARK-23901][SQL] Add masking functions

2018-05-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21246#discussion_r187836985 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/maskExpressions.scala --- @@ -0,0 +1,569 @@ +/* + * Licensed to the

[GitHub] spark issue #21318: [minor] Update docs for functions.scala to make it clear...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21318 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90567/ Test FAILed. ---

[GitHub] spark pull request #21246: [SPARK-23901][SQL] Add masking functions

2018-05-13 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21246#discussion_r187835126 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/MaskExpressionsUtils.java --- @@ -0,0 +1,80 @@ +/* + * Licensed to th

[GitHub] spark issue #21318: [minor] Update docs for functions.scala to make it clear...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21318 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3187/ Tes

[GitHub] spark issue #21318: [minor] Update docs for functions.scala to make it clear...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21318 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21318: [minor] Update docs for functions.scala to make it clear...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21318 **[Test build #90567 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90567/testReport)** for PR 21318 at commit [`83c191f`](https://github.com/apache/spark/commit/83

[GitHub] spark issue #21318: [minor] Update docs for functions.scala to make it clear...

2018-05-13 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/21318 cc @gatorsmile @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark pull request #21318: [minor] Update docs for functions.scala to make i...

2018-05-13 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/21318 [minor] Update docs for functions.scala to make it clear not all the built-in functions are defined there The title summarizes the change. You can merge this pull request into a Git repository by ru

[GitHub] spark pull request #21316: [SPARK-20538][SQL] Wrap Dataset.reduce with withN...

2018-05-13 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/21316#discussion_r187838099 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1607,7 +1607,9 @@ class Dataset[T] private[sql]( */ @Experimental

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21092 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3091/ ---

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21092 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/3091/ --- --

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21092 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21092 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3186/ Tes

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-13 Thread ifilonenko
Github user ifilonenko commented on the issue: https://github.com/apache/spark/pull/21092 jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21299: [SPARK-24250][SQL] support accessing SQLConf insi...

2018-05-13 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/21299#discussion_r187833760 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SQLExecution.scala --- @@ -90,13 +92,33 @@ object SQLExecution { * thread from

[GitHub] spark pull request #21316: [SPARK-20538][SQL] Wrap Dataset.reduce with withN...

2018-05-13 Thread sohama4
Github user sohama4 commented on a diff in the pull request: https://github.com/apache/spark/pull/21316#discussion_r187832727 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1607,7 +1607,9 @@ class Dataset[T] private[sql]( */ @Experimental

[GitHub] spark pull request #21296: [SPARK-24244][SQL] Passing only required columns ...

2018-05-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21296#discussion_r187832513 --- Diff: docs/sql-programming-guide.md --- @@ -1814,6 +1814,7 @@ working with timestamps in `pandas_udf`s to get the best performance, see - In ve

[GitHub] spark pull request #20800: [SPARK-23627][SQL] Provide isEmpty in Dataset

2018-05-13 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20800#discussion_r187831971 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -511,6 +511,14 @@ class Dataset[T] private[sql]( */ def isLocal:

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21317 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90566/ Test FAILed. ---

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21317 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21317 **[Test build #90566 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90566/testReport)** for PR 21317 at commit [`24832c5`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21317 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3185/ Tes

[GitHub] spark issue #21208: [SPARK-23925][SQL] Add array_repeat collection function

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21208 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90563/ Test PASSed. ---

[GitHub] spark pull request #21301: [SPARK-24228][SQL] Fix Java lint errors

2018-05-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21301 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21317 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21208: [SPARK-23925][SQL] Add array_repeat collection function

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21208 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21208: [SPARK-23925][SQL] Add array_repeat collection function

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21208 **[Test build #90563 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90563/testReport)** for PR 21208 at commit [`471597a`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21317 **[Test build #90566 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90566/testReport)** for PR 21317 at commit [`24832c5`](https://github.com/apache/spark/commit/24

[GitHub] spark issue #21317: [SPARK-24232][k8s] Add support for secret env vars

2018-05-13 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21317 @liyinan926 @dharmeshkakadia pls review. Hope its close to what we discussed. --- - To unsubscribe, e-mail: reviews-unsubscr...@s

[GitHub] spark issue #21301: [SPARK-24228][SQL] Fix Java lint errors

2018-05-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21301 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #21317: [SPARK-24232][Kubernetes] Add support for secret ...

2018-05-13 Thread skonto
GitHub user skonto opened a pull request: https://github.com/apache/spark/pull/21317 [SPARK-24232][Kubernetes] Add support for secret env vars ## What changes were proposed in this pull request? * Allows to refer a secret as an env var. * Introduces new properties in the

[GitHub] spark issue #21301: [SPARK-24228][SQL] Fix Java lint errors

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21301 **[Test build #4177 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4177/testReport)** for PR 21301 at commit [`5403bae`](https://github.com/apache/spark/commit/

[GitHub] spark issue #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work with Py...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21267 **[Test build #90565 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90565/testReport)** for PR 21267 at commit [`68be3ba`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work with Py...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21267 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work with Py...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21267 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90565/ Test PASSed. ---

[GitHub] spark pull request #21221: [SPARK-23429][CORE] Add executor memory metrics t...

2018-05-13 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/21221#discussion_r187825561 --- Diff: core/src/main/scala/org/apache/spark/Heartbeater.scala --- @@ -0,0 +1,52 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] spark issue #21208: [SPARK-23925][SQL] Add array_repeat collection function

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21208 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21221: [SPARK-23429][CORE] Add executor memory metrics t...

2018-05-13 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/21221#discussion_r187824094 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1753,9 +1766,21 @@ class DAGScheduler( messageScheduler.shutdow

[GitHub] spark issue #21208: [SPARK-23925][SQL] Add array_repeat collection function

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21208 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90562/ Test PASSed. ---

[GitHub] spark pull request #21221: [SPARK-23429][CORE] Add executor memory metrics t...

2018-05-13 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/21221#discussion_r187823298 --- Diff: core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala --- @@ -169,6 +179,27 @@ private[spark] class EventLoggingListener(

[GitHub] spark issue #21208: [SPARK-23925][SQL] Add array_repeat collection function

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21208 **[Test build #90562 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90562/testReport)** for PR 21208 at commit [`5381ac0`](https://github.com/apache/spark/commit/5

[GitHub] spark pull request #21296: [SPARK-24244][SQL] Passing only required columns ...

2018-05-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21296#discussion_r187823861 --- Diff: docs/sql-programming-guide.md --- @@ -1814,6 +1814,7 @@ working with timestamps in `pandas_udf`s to get the best performance, see - In

[GitHub] spark pull request #21296: [SPARK-24244][SQL] Passing only required columns ...

2018-05-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21296#discussion_r187823801 --- Diff: docs/sql-programming-guide.md --- @@ -1814,6 +1814,7 @@ working with timestamps in `pandas_udf`s to get the best performance, see - In

[GitHub] spark issue #21311: [SPARK-24257][SQL]LongToUnsafeRowMap calculate the new s...

2018-05-13 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21311 @gatorsmile @hvanhovell Could you trigger tests? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...

2018-05-13 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/21114 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark pull request #21273: [SPARK-17916][SQL] Fix empty string being parsed ...

2018-05-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21273 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20068: [SPARK-17916][SQL] Fix empty string being parsed ...

2018-05-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20068 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21310: [SPARK-24256][SQL] SPARK-24256: ExpressionEncoder should...

2018-05-13 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21310 You need to add tests first. Could you? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark pull request #21114: [SPARK-22371][CORE] Return None instead of throwi...

2018-05-13 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/21114#discussion_r187823469 --- Diff: core/src/test/scala/org/apache/spark/AccumulatorSuite.scala --- @@ -237,6 +236,65 @@ class AccumulatorSuite extends SparkFunSuite with Matche

[GitHub] spark issue #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work with Py...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21267 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21092 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21092 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90561/ Test FAILed. ---

[GitHub] spark issue #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work with Py...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21267 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3184/ Tes

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21092 **[Test build #90561 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90561/testReport)** for PR 21092 at commit [`72953a3`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #21273: [SPARK-17916][SQL] Fix empty string being parsed as null...

2018-05-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21273 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work with Py...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21267 **[Test build #90565 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90565/testReport)** for PR 21267 at commit [`68be3ba`](https://github.com/apache/spark/commit/68

[GitHub] spark issue #21273: [SPARK-17916][SQL] Fix empty string being parsed as null...

2018-05-13 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21273 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache

[GitHub] spark pull request #21307: [SPARK-24186][R][SQL]change reverse and concat to...

2018-05-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21307 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21288: [SPARK-24206][SQL] Improve FilterPushdownBenchmar...

2018-05-13 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21288#discussion_r187822729 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/FilterPushdownBenchmark.scala --- @@ -32,14 +32,14 @@ import org.apache.spark.util.{Benchmark, Utils}

[GitHub] spark issue #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work with Py...

2018-05-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21267 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #21307: [SPARK-24186][R][SQL]change reverse and concat to collec...

2018-05-13 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21307 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #21307: [SPARK-24186][R][SQL]change reverse and concat to...

2018-05-13 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21307#discussion_r187822362 --- Diff: R/pkg/R/functions.R --- @@ -2055,20 +2058,10 @@ setMethod("countDistinct", #' @details #' \code{concat}: Concatenates multiple

[GitHub] spark issue #21114: [SPARK-22371][CORE] Return None instead of throwing an e...

2018-05-13 Thread artemrd
Github user artemrd commented on the issue: https://github.com/apache/spark/pull/21114 Yes, this is correct. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #21114: [SPARK-22371][CORE] Return None instead of throwi...

2018-05-13 Thread artemrd
Github user artemrd commented on a diff in the pull request: https://github.com/apache/spark/pull/21114#discussion_r187821160 --- Diff: core/src/test/scala/org/apache/spark/AccumulatorSuite.scala --- @@ -237,6 +236,65 @@ class AccumulatorSuite extends SparkFunSuite with Matchers wi

[GitHub] spark pull request #20800: [SPARK-23627][SQL] Provide isEmpty in Dataset

2018-05-13 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20800#discussion_r187820731 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -511,6 +511,14 @@ class Dataset[T] private[sql]( */ def isLocal: Bo

[GitHub] spark issue #18903: [SPARK-21590][SS]Window start time should support negati...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18903 **[Test build #4178 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4178/testReport)** for PR 18903 at commit [`07e98e7`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #21316: [SPARK-20538][SQL] Wrap Dataset.reduce with withN...

2018-05-13 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21316#discussion_r187820636 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -1607,7 +1607,9 @@ class Dataset[T] private[sql]( */ @Experimental

[GitHub] spark issue #21236: [SPARK-23935][SQL] Adding map_entries function

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21236 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90557/ Test PASSed. ---

[GitHub] spark issue #21236: [SPARK-23935][SQL] Adding map_entries function

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21236 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21236: [SPARK-23935][SQL] Adding map_entries function

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21236 **[Test build #90557 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90557/testReport)** for PR 21236 at commit [`56ff20a`](https://github.com/apache/spark/commit/5

[GitHub] spark pull request #21162: shaded guava is not used anywhere, seems guava is...

2018-05-13 Thread yileic
Github user yileic closed the pull request at: https://github.com/apache/spark/pull/21162 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21314: [SPARK-24263][R] SparkR java check breaks with op...

2018-05-13 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21314#discussion_r187817762 --- Diff: R/pkg/R/client.R --- @@ -82,7 +82,7 @@ checkJavaVersion <- function() { }) javaVersionFilter <- Filter(

[GitHub] spark issue #20274: [SPARK-20120][SQL][FOLLOW-UP] Better way to support spar...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20274 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20274: [SPARK-20120][SQL][FOLLOW-UP] Better way to support spar...

2018-05-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20274 **[Test build #90564 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90564/testReport)** for PR 20274 at commit [`0501653`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #20274: [SPARK-20120][SQL][FOLLOW-UP] Better way to support spar...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20274 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90564/ Test PASSed. ---

[GitHub] spark issue #20274: [SPARK-20120][SQL][FOLLOW-UP] Better way to support spar...

2018-05-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20274 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3183/ Tes

  1   2   3   >