[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-05-02 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r185434976 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/FailureTracker.scala --- @@ -0,0 +1,85 @@ +/* + * Licensed to the

[GitHub] spark issue #21206: [SPARK-24133][SQL] Check for integer overflows when resi...

2018-05-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21206 **[Test build #90037 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90037/testReport)** for PR 21206 at commit [`ddfd147`](https://github.com/apache/spark/commit/dd

[GitHub] spark issue #21109: [SPARK-24020][SQL] Sort-merge join inner range optimizat...

2018-05-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21109 @viirya I think predicate reordering based on cost estimation and others is an interesting topic in optimizer. But, IMO the topic is not directly related to this pr (I'm not 100% sure though), it'd

[GitHub] spark issue #21206: [SPARK-24133][SQL] Check for integer overflows when resi...

2018-05-02 Thread ala
Github user ala commented on the issue: https://github.com/apache/spark/pull/21206 Seems like something's broken with the R test, since it fails in the same way for other PRs too, e.g., https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90020/ --- --

[GitHub] spark issue #21206: [SPARK-24133][SQL] Check for integer overflows when resi...

2018-05-02 Thread ala
Github user ala commented on the issue: https://github.com/apache/spark/pull/21206 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@s

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-05-02 Thread attilapiros
Github user attilapiros commented on a diff in the pull request: https://github.com/apache/spark/pull/21068#discussion_r185432837 --- Diff: core/src/test/scala/org/apache/spark/HeartbeatReceiverSuite.scala --- @@ -73,6 +73,7 @@ class HeartbeatReceiverSuite sc = spy(new Spa

[GitHub] spark pull request #21178: [SPARK-24110][Thrift-Server] Avoid UGI.loginUserF...

2018-05-02 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21178#discussion_r185432085 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIService.scala --- @@ -52,8 +52,22 @@ private[hive] class

[GitHub] spark issue #21109: [SPARK-24020][SQL] Sort-merge join inner range optimizat...

2018-05-02 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21109 Since I feel this is a limited case, I'm not certainly sure this optimization needs to be handled in smj. For spatial or temporal use cases, is it not enough to add dummy join keys to split tasks int

[GitHub] spark issue #21121: [SPARK-24042][SQL] Collection function: zip_with_index

2018-05-02 Thread mn-mikke
Github user mn-mikke commented on the issue: https://github.com/apache/spark/pull/21121 @rxin Oh, I see. In that case, I'm happy to close the PR. @hvanhovell Can you confirm that the `transform` function will pass the index into lambda functions? --- ---

[GitHub] spark issue #21213: [SPARK-24120] Show `Jobs` page when `jobId` is missing

2018-05-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21213 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2799/ Tes

[GitHub] spark issue #21213: [SPARK-24120] Show `Jobs` page when `jobId` is missing

2018-05-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21213 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20940: [SPARK-23429][CORE] Add executor memory metrics to heart...

2018-05-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20940 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21213: [SPARK-24120] Show `Jobs` page when `jobId` is missing

2018-05-02 Thread jongyoul
Github user jongyoul commented on the issue: https://github.com/apache/spark/pull/21213 As I described in issue description, yarn filter couldn't pass query strings when redirecting to yarn application. It's already fixed with the latest version of yarn, but some old versions still ha

[GitHub] spark pull request #21064: [SPARK-23976][Core] Detect length overflow in UTF...

2018-05-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21064 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21213: [SPARK-24120] Show `Jobs` page when `jobId` is missing

2018-05-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21213 **[Test build #90036 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90036/testReport)** for PR 21213 at commit [`5e7d4a3`](https://github.com/apache/spark/commit/5e

[GitHub] spark pull request #21213: [SPARK-24120]

2018-05-02 Thread jongyoul
GitHub user jongyoul opened a pull request: https://github.com/apache/spark/pull/21213 [SPARK-24120] ## What changes were proposed in this pull request? Change the case which doesn't have jobId as a parameter so that it will redirect to `Jobs` page. ## How was this pat

[GitHub] spark issue #21064: [SPARK-23976][Core] Detect length overflow in UTF8String...

2018-05-02 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/21064 @kiszk sorry about the delay. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-ma

[GitHub] spark pull request #21178: [SPARK-24110][Thrift-Server] Avoid UGI.loginUserF...

2018-05-02 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21178#discussion_r185427751 --- Diff: sql/hive-thriftserver/src/main/java/org/apache/hive/service/auth/HiveAuthFactory.java --- @@ -92,7 +95,26 @@ public String getAuthName() {

[GitHub] spark pull request #21178: [SPARK-24110][Thrift-Server] Avoid UGI.loginUserF...

2018-05-02 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/21178#discussion_r185423983 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIService.scala --- @@ -52,8 +52,22 @@ private[hive] class S

[GitHub] spark pull request #21082: [SPARK-22239][SQL][Python] Enable grouped aggrega...

2018-05-02 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21082#discussion_r185423199 --- Diff: python/pyspark/sql/functions.py --- @@ -2301,10 +2301,12 @@ def pandas_udf(f=None, returnType=None, functionType=None): The return

[GitHub] spark pull request #21178: [SPARK-24110][Thrift-Server] Avoid UGI.loginUserF...

2018-05-02 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/21178#discussion_r185423072 --- Diff: sql/hive-thriftserver/src/main/java/org/apache/hive/service/auth/HiveAuthFactory.java --- @@ -92,7 +95,26 @@ public String getAuthName() {

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21092 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/2729/ ---

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21092 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/2729/ --- --

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21092 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21092: [SPARK-23984][K8S] Initial Python Bindings for PySpark o...

2018-05-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21092 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2798/ Tes

[GitHub] spark issue #21175: [SPARK-24107][CORE] ChunkedByteBuffer.writeFully method ...

2018-05-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21175 **[Test build #90035 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90035/testReport)** for PR 21175 at commit [`e78ef39`](https://github.com/apache/spark/commit/e7

[GitHub] spark pull request #21175: [SPARK-24107][CORE] ChunkedByteBuffer.writeFully ...

2018-05-02 Thread manbuyun
Github user manbuyun commented on a diff in the pull request: https://github.com/apache/spark/pull/21175#discussion_r185415529 --- Diff: core/src/test/scala/org/apache/spark/io/ChunkedByteBufferSuite.scala --- @@ -20,12 +20,12 @@ package org.apache.spark.io import java.nio.Byt

[GitHub] spark pull request #21178: [SPARK-24110][Thrift-Server] Avoid UGI.loginUserF...

2018-05-02 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21178#discussion_r185413718 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIService.scala --- @@ -52,8 +52,22 @@ private[hive] class

[GitHub] spark issue #21201: [SPARK-24128][SQL] Mention configuration option in impli...

2018-05-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21201 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21201: [SPARK-24128][SQL] Mention configuration option in impli...

2018-05-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21201 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2797/ Tes

[GitHub] spark issue #21201: [SPARK-24128][SQL] Mention configuration option in impli...

2018-05-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21201 **[Test build #90034 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90034/testReport)** for PR 21201 at commit [`42d0241`](https://github.com/apache/spark/commit/42

[GitHub] spark pull request #21054: [SPARK-23907][SQL] Add regr_* functions

2018-05-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21054#discussion_r185411834 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/regression.scala --- @@ -0,0 +1,193 @@ +/* + * Licensed to

[GitHub] spark issue #21185: [SPARK-23894][CORE][SQL] Defensively clear ActiveSession...

2018-05-02 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21185 > that's the whole problem. Its only meant to be available on the driver, but it ends up getting set on the executor when running in local mode. Yea I think this is the root cause. I'm mak

[GitHub] spark pull request #21054: [SPARK-23907][SQL] Add regr_* functions

2018-05-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21054#discussion_r185409726 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Corr.scala --- @@ -51,7 +47,27 @@ case class Corr(x: Expression, y

[GitHub] spark pull request #21054: [SPARK-23907][SQL] Add regr_* functions

2018-05-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21054#discussion_r185409402 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/CentralMomentAgg.scala --- @@ -128,6 +100,36 @@ abstract class Cen

[GitHub] spark issue #21050: [SPARK-23912][SQL]add array_distinct

2018-05-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21050 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90028/ Test FAILed. ---

[GitHub] spark issue #21210: [SPARK-23489][SQL][TEST] HiveExternalCatalogVersionsSuit...

2018-05-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21210 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90029/ Test FAILed. ---

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-05-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21021 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90031/ Test FAILed. ---

[GitHub] spark issue #21050: [SPARK-23912][SQL]add array_distinct

2018-05-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21050 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21210: [SPARK-23489][SQL][TEST] HiveExternalCatalogVersionsSuit...

2018-05-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21210 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21212: [SPARK-24143] filter empty blocks when convert mapstatus...

2018-05-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21212 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90032/ Test FAILed. ---

[GitHub] spark issue #21202: [SPARK-24129] [K8S] Add option to pass --build-arg's to ...

2018-05-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21202 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90033/ Test FAILed. ---

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-05-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21021 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21212: [SPARK-24143] filter empty blocks when convert mapstatus...

2018-05-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21212 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21021: [SPARK-23921][SQL] Add array_sort function

2018-05-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21021 **[Test build #90031 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90031/testReport)** for PR 21021 at commit [`9f63a76`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #21212: [SPARK-24143] filter empty blocks when convert mapstatus...

2018-05-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21212 **[Test build #90032 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90032/testReport)** for PR 21212 at commit [`5211ebd`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #21202: [SPARK-24129] [K8S] Add option to pass --build-arg's to ...

2018-05-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21202 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21050: [SPARK-23912][SQL]add array_distinct

2018-05-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21050 **[Test build #90028 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90028/testReport)** for PR 21050 at commit [`e57056b`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #21202: [SPARK-24129] [K8S] Add option to pass --build-arg's to ...

2018-05-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21202 **[Test build #90033 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90033/testReport)** for PR 21202 at commit [`3e410cd`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #21210: [SPARK-23489][SQL][TEST] HiveExternalCatalogVersionsSuit...

2018-05-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21210 **[Test build #90029 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90029/testReport)** for PR 21210 at commit [`51d4c0e`](https://github.com/apache/spark/commit/5

[GitHub] spark pull request #21054: [SPARK-23907][SQL] Add regr_* functions

2018-05-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21054#discussion_r185407773 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Average.scala --- @@ -85,4 +67,28 @@ case class Average(child: Exp

[GitHub] spark pull request #21054: [SPARK-23907][SQL] Add regr_* functions

2018-05-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21054#discussion_r185407606 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Average.scala --- @@ -85,4 +67,28 @@ case class Average(child: Exp

[GitHub] spark pull request #21178: [SPARK-24110][Thrift-Server] Avoid UGI.loginUserF...

2018-05-02 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21178#discussion_r185407534 --- Diff: sql/hive-thriftserver/src/main/java/org/apache/hive/service/auth/HiveAuthFactory.java --- @@ -92,7 +95,26 @@ public String getAuthName() {

<    2   3   4   5   6   7