[GitHub] spark issue #21850: [SPARK-24892] [SQL] Simplify `CaseWhen` to `If` when the...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21850 **[Test build #93484 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93484/testReport)** for PR 21850 at commit [`a9c97ce`](https://github.com/apache/spark/commit/a9

[GitHub] spark issue #21850: [SPARK-24892] [SQL] Simplify `CaseWhen` to `If` when the...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21850 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #21845: [SPARK-24886][INFRA] Fix the testing script to increase ...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21845 Hm, yea then. I actually opened this PR to make the tests passed in your PR. Let me leave this closed then and reopen when we hit the issue next time. --- -

[GitHub] spark pull request #21845: [SPARK-24886][INFRA] Fix the testing script to in...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon closed the pull request at: https://github.com/apache/spark/pull/21845 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.or

[GitHub] spark issue #21845: [SPARK-24886][INFRA] Fix the testing script to increase ...

2018-07-23 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/21845 If that's the only one I think that PR itself needs to be fixed (significantly increases test runtime), and I wouldn't increase the time here. On Mon, Jul 23, 2018 at 11:44 PM Hyukj

[GitHub] spark issue #21822: [SPARK-24865] Remove AnalysisBarrier - WIP

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21822 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21822: [SPARK-24865] Remove AnalysisBarrier - WIP

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21822 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93470/ Test FAILed. ---

[GitHub] spark issue #21845: [SPARK-24886][INFRA] Fix the testing script to increase ...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21845 Yup, looks so in your PR https://github.com/apache/spark/pull/21822#issuecomment-407298841 --- - To unsubscribe, e-mail: rev

[GitHub] spark issue #21822: [SPARK-24865] Remove AnalysisBarrier - WIP

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21822 **[Test build #93470 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93470/testReport)** for PR 21822 at commit [`38980ad`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #21855: [SPARK-22499][FOLLOWUP][SQL] Reduce input string express...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21855 **[Test build #93483 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93483/testReport)** for PR 21855 at commit [`0efe041`](https://github.com/apache/spark/commit/0e

[GitHub] spark issue #21855: [SPARK-22499][FOLLOWUP][SQL] Reduce input string express...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21855 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21855: [SPARK-22499][FOLLOWUP][SQL] Reduce input string express...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21855 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1260/

[GitHub] spark pull request #21855: [SPARK-22499][FOLLOWUP][SQL] Reduce input string ...

2018-07-23 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/21855 [SPARK-22499][FOLLOWUP][SQL] Reduce input string expressions for Least and Greatest to reduce time ## What changes were proposed in this pull request? It's minor and trivial but looks

[GitHub] spark issue #21789: [SPARK-24829][STS]In Spark Thrift Server, CAST AS FLOAT ...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21789 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21789: [SPARK-24829][STS]In Spark Thrift Server, CAST AS FLOAT ...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21789 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93482/ Test PASSed. ---

[GitHub] spark issue #21789: [SPARK-24829][STS]In Spark Thrift Server, CAST AS FLOAT ...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21789 **[Test build #93482 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93482/testReport)** for PR 21789 at commit [`88dff75`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #21809: [SPARK-24851] : Map a Stage ID to it's Associated Job ID...

2018-07-23 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/21809 @pgandhi999 could you update the title to ` [SPARK-24851][UI] Map a Stage ID to it's Associated Job ID` --- - To unsubs

[GitHub] spark issue #21822: [SPARK-24865] Remove AnalysisBarrier - WIP

2018-07-23 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/21822 Yea the extra check in test cases might've contributed to the longer test time. Let me think about how to reduce it. On Mon, Jul 23, 2018 at 11:28 PM Hyukjin Kwon wrote: >

[GitHub] spark issue #21822: [SPARK-24865] Remove AnalysisBarrier - WIP

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21822 @rxin, I think this PR could possibly cause some performance effect given the latest test ran above https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93470/ and from a rough sc

[GitHub] spark pull request #21748: [SPARK-23146][K8S] Support client mode.

2018-07-23 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21748#discussion_r204635755 --- Diff: docs/running-on-kubernetes.md --- @@ -117,6 +117,45 @@ If the local proxy is running at localhost:8001, `--master k8s://http://127.0.0. s

[GitHub] spark pull request #21748: [SPARK-23146][K8S] Support client mode.

2018-07-23 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21748#discussion_r204635584 --- Diff: docs/running-on-kubernetes.md --- @@ -117,6 +117,45 @@ If the local proxy is running at localhost:8001, `--master k8s://http://127.0.0. s

[GitHub] spark pull request #21748: [SPARK-23146][K8S] Support client mode.

2018-07-23 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21748#discussion_r204636048 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterManager.scala --- @@ -35,26 +35,39 @

[GitHub] spark pull request #21546: [SPARK-23030][SQL][PYTHON] Use Arrow stream forma...

2018-07-23 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21546#discussion_r204635055 --- Diff: python/pyspark/serializers.py --- @@ -184,27 +184,67 @@ def loads(self, obj): raise NotImplementedError -class Arr

[GitHub] spark pull request #21809: [SPARK-24851] : Map a Stage ID to it's Associated...

2018-07-23 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/21809#discussion_r204634512 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -182,6 +184,12 @@ private[ui] class StagePage(parent: StagesTab, store:

[GitHub] spark pull request #21809: [SPARK-24851] : Map a Stage ID to it's Associated...

2018-07-23 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/21809#discussion_r204634172 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -115,6 +115,8 @@ private[ui] class StagePage(parent: StagesTab, store: A

[GitHub] spark issue #21788: [SPARK-24609][ML][DOC] PySpark/SparkR doc doesn't explai...

2018-07-23 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21788 what's changed? it was forced pushed so I can't tell what's updated --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.a

[GitHub] spark pull request #21809: [SPARK-24851] : Map a Stage ID to it's Associated...

2018-07-23 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/21809#discussion_r204633878 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -115,6 +115,8 @@ private[ui] class StagePage(parent: StagesTab, store: A

[GitHub] spark issue #21789: [SPARK-24829][STS]In Spark Thrift Server, CAST AS FLOAT ...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21789 **[Test build #93482 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93482/testReport)** for PR 21789 at commit [`88dff75`](https://github.com/apache/spark/commit/88

[GitHub] spark pull request #21835: [SPARK-24779]Add sequence / map_concat / map_from...

2018-07-23 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21835#discussion_r204633191 --- Diff: R/pkg/R/functions.R --- @@ -1986,15 +1998,20 @@ setMethod("levenshtein", signature(y = "Column"), #' are on the same day of month, or both

[GitHub] spark pull request #21835: [SPARK-24779]Add sequence / map_concat / map_from...

2018-07-23 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21835#discussion_r204633464 --- Diff: R/pkg/tests/fulltests/test_context.R --- @@ -21,10 +21,11 @@ test_that("Check masked functions", { # Check that we are not masking any n

[GitHub] spark pull request #21823: [SPARK-24870][SQL]Cache can't work normally if th...

2018-07-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21823 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21823: [SPARK-24870][SQL]Cache can't work normally if there are...

2018-07-23 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21823 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21103 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21103 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1259/

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21103 **[Test build #93481 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93481/testReport)** for PR 21103 at commit [`f099cbf`](https://github.com/apache/spark/commit/f0

[GitHub] spark issue #21850: [SPARK-24892] [SQL] Simplify `CaseWhen` to `If` when the...

2018-07-23 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21850 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revie

[GitHub] spark issue #21546: [SPARK-23030][SQL][PYTHON] Use Arrow stream format for c...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21546 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93465/ Test PASSed. ---

[GitHub] spark issue #21546: [SPARK-23030][SQL][PYTHON] Use Arrow stream format for c...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21546 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21546: [SPARK-23030][SQL][PYTHON] Use Arrow stream format for c...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21546 **[Test build #93465 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93465/testReport)** for PR 21546 at commit [`3224625`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #21823: [SPARK-24870][SQL]Cache can't work normally if there are...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21823 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21823: [SPARK-24870][SQL]Cache can't work normally if there are...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21823 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93467/ Test PASSed. ---

[GitHub] spark issue #21823: [SPARK-24870][SQL]Cache can't work normally if there are...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21823 **[Test build #93467 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93467/testReport)** for PR 21823 at commit [`f3a7963`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #21839: [SPARK-24339][SQL] Prunes the unused columns from child ...

2018-07-23 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/21839 Thanks for reviewing. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: re

[GitHub] spark issue #21772: [SPARK-24809] [SQL] Serializing LongToUnsafeRowMap in ex...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21772 **[Test build #93480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93480/testReport)** for PR 21772 at commit [`c9ebfd0`](https://github.com/apache/spark/commit/c9

[GitHub] spark issue #21854: [SPARK-24896][SQL] Uuid should produce different values ...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21854 **[Test build #93479 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93479/testReport)** for PR 21854 at commit [`c1ce69c`](https://github.com/apache/spark/commit/c1

[GitHub] spark issue #21854: [SPARK-24896][SQL] Uuid should produce different values ...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21854 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21854: [SPARK-24896][SQL] Uuid should produce different values ...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21854 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1258/

[GitHub] spark issue #21854: [SPARK-24896][SQL] Uuid should produce different values ...

2018-07-23 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21854 Actually I think `Rand` and `Randn` should also have the same issue. But I want to hear opinions first before dealing them. --- -

[GitHub] spark pull request #21758: [SPARK-24795][CORE] Implement barrier execution m...

2018-07-23 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/21758#discussion_r204624088 --- Diff: core/src/main/scala/org/apache/spark/BarrierTaskInfo.scala --- @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistics to improve ...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16677 **[Test build #93478 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93478/testReport)** for PR 16677 at commit [`d05c144`](https://github.com/apache/spark/commit/d0

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistics to improve ...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16677 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1257/

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistics to improve ...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16677 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21854: [SPARK-24896][SQL] Uuid should produce different ...

2018-07-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21854#discussion_r204622960 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1392,3 +1394,17 @@ object UpdateNullabilityInAttribute

[GitHub] spark issue #21853: [SPARK-23957][SQL] Sorts in subqueries are redundant and...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21853 **[Test build #93477 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93477/testReport)** for PR 21853 at commit [`a86cb9f`](https://github.com/apache/spark/commit/a8

[GitHub] spark issue #21853: [SPARK-23957][SQL] Sorts in subqueries are redundant and...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21853 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1256/

[GitHub] spark issue #21853: [SPARK-23957][SQL] Sorts in subqueries are redundant and...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21853 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #21854: [SPARK-24896][SQL] Uuid should produce different ...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21854#discussion_r204622410 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -1392,3 +1394,17 @@ object UpdateNullabilityInAttr

[GitHub] spark issue #21851: [SPARK-24891][SQL] Fix HandleNullInputsForUDF rule

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21851 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93469/ Test FAILed. ---

[GitHub] spark issue #21854: [SPARK-24896][SQL] Uuid should produce different values ...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21854 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21854: [SPARK-24896][SQL] Uuid should produce different values ...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21854 **[Test build #93476 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93476/testReport)** for PR 21854 at commit [`8ef299f`](https://github.com/apache/spark/commit/8e

[GitHub] spark issue #21851: [SPARK-24891][SQL] Fix HandleNullInputsForUDF rule

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21851 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21854: [SPARK-24896][SQL] Uuid should produce different values ...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21854 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1255/

[GitHub] spark issue #21851: [SPARK-24891][SQL] Fix HandleNullInputsForUDF rule

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21851 **[Test build #93469 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93469/testReport)** for PR 21851 at commit [`b499b97`](https://github.com/apache/spark/commit/b

[GitHub] spark pull request #21854: [SPARK-24896][SQL] Uuid should produce different ...

2018-07-23 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/21854 [SPARK-24896][SQL] Uuid should produce different values for each execution in streaming query ## What changes were proposed in this pull request? `Uuid`'s results depend on random seed give

[GitHub] spark issue #21850: [SPARK-24892] [SQL] Simplify `CaseWhen` to `If` when the...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21850 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93468/ Test FAILed. ---

[GitHub] spark issue #21850: [SPARK-24892] [SQL] Simplify `CaseWhen` to `If` when the...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21850 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21850: [SPARK-24892] [SQL] Simplify `CaseWhen` to `If` when the...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21850 **[Test build #93468 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93468/testReport)** for PR 21850 at commit [`a9c97ce`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #21772: [SPARK-24809] [SQL] Serializing LongHashedRelation in ex...

2018-07-23 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21772 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@sp

[GitHub] spark pull request #21772: [SPARK-24809] [SQL] Serializing LongHashedRelatio...

2018-07-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21772#discussion_r204618927 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/joins/HashedRelationSuite.scala --- @@ -278,6 +278,39 @@ class HashedRelationSuite extends

[GitHub] spark pull request #21772: [SPARK-24809] [SQL] Serializing LongHashedRelatio...

2018-07-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21772#discussion_r204618745 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/joins/HashedRelationSuite.scala --- @@ -278,6 +278,39 @@ class HashedRelationSuite extends

[GitHub] spark pull request #21608: [SPARK-24626] [SQL] Improve location size calcula...

2018-07-23 Thread Achuth17
Github user Achuth17 commented on a diff in the pull request: https://github.com/apache/spark/pull/21608#discussion_r204618663 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/StatisticsSuite.scala --- @@ -148,6 +148,19 @@ class StatisticsSuite extends StatisticsCollect

[GitHub] spark issue #21608: [SPARK-24626] [SQL] Improve location size calculation in...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21608 **[Test build #93475 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93475/testReport)** for PR 21608 at commit [`4c405c5`](https://github.com/apache/spark/commit/4c

[GitHub] spark pull request #21608: [SPARK-24626] [SQL] Improve location size calcula...

2018-07-23 Thread Achuth17
Github user Achuth17 commented on a diff in the pull request: https://github.com/apache/spark/pull/21608#discussion_r204618589 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Utils.scala --- @@ -55,4 +57,11 @@ private[sql] object DataSou

[GitHub] spark pull request #21772: [SPARK-24809] [SQL] Serializing LongHashedRelatio...

2018-07-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21772#discussion_r204618320 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/joins/HashedRelationSuite.scala --- @@ -278,6 +278,39 @@ class HashedRelationSuite extends

[GitHub] spark issue #21772: [SPARK-24809] [SQL] Serializing LongHashedRelation in ex...

2018-07-23 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21772 As you actually modify `LongToUnsafeRowMap`, is it better to update the PR title and description to reflect that? --- - To unsubs

[GitHub] spark issue #21653: [SPARK-13343] speculative tasks that didn't commit shoul...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21653 **[Test build #93474 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93474/testReport)** for PR 21653 at commit [`b6585da`](https://github.com/apache/spark/commit/b6

[GitHub] spark pull request #21772: [SPARK-24809] [SQL] Serializing LongHashedRelatio...

2018-07-23 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21772#discussion_r204617884 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala --- @@ -772,6 +772,8 @@ private[execution] final class LongToUns

[GitHub] spark issue #21653: [SPARK-13343] speculative tasks that didn't commit shoul...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21653 test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-

[GitHub] spark issue #21835: [SPARK-24779]Add sequence / map_concat / map_from_entrie...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21835 Seems fine except https://github.com/apache/spark/pull/21835#discussion_r204617441 --- - To unsubscribe, e-mail: reviews-uns

[GitHub] spark pull request #21835: [SPARK-24779]Add sequence / map_concat / map_from...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21835#discussion_r204617441 --- Diff: R/pkg/tests/fulltests/test_context.R --- @@ -21,10 +21,11 @@ test_that("Check masked functions", { # Check that we are not masking any n

[GitHub] spark pull request #21835: [SPARK-24779]Add sequence / map_concat / map_from...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21835#discussion_r204616561 --- Diff: R/pkg/R/functions.R --- @@ -1986,15 +1998,20 @@ setMethod("levenshtein", signature(y = "Column"), #' are on the same day of month, or both

[GitHub] spark issue #21772: [SPARK-24809] [SQL] Serializing LongHashedRelation in ex...

2018-07-23 Thread liutang123
Github user liutang123 commented on the issue: https://github.com/apache/spark/pull/21772 @viirya This case occurred in our cluster and we took a lot of time to find this bug. For some man-made reasons, the small table's max id has become abnormally large. The LongHasedRelation g

[GitHub] spark pull request #21848: [SPARK-24890] [SQL] Short circuiting the `if` con...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21848#discussion_r204615261 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -403,14 +404,14 @@ object SimplifyConditionals e

[GitHub] spark pull request #21848: [SPARK-24890] [SQL] Short circuiting the `if` con...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21848#discussion_r204615040 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala --- @@ -651,6 +652,7 @@ object SimplifyCaseConversionEx

[GitHub] spark pull request #21772: [SPARK-24809] [SQL] Serializing LongHashedRelatio...

2018-07-23 Thread liutang123
Github user liutang123 commented on a diff in the pull request: https://github.com/apache/spark/pull/21772#discussion_r204613880 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/joins/HashedRelationSuite.scala --- @@ -278,6 +278,39 @@ class HashedRelationSuite exte

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-23 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r204613831 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3799,331 @@ object ArrayUnion {

[GitHub] spark issue #21772: [SPARK-24809] [SQL] Serializing LongHashedRelation in ex...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21772 **[Test build #93473 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93473/testReport)** for PR 21772 at commit [`06a9547`](https://github.com/apache/spark/commit/06

[GitHub] spark issue #21853: [SPARK-23957][SQL] Sorts in subqueries are redundant and...

2018-07-23 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21853 Also, could you add `Closes #21049` in the description? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For add

[GitHub] spark issue #21788: [SPARK-24609][ML][DOC] PySpark/SparkR doc doesn't explai...

2018-07-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21788 **[Test build #93472 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93472/testReport)** for PR 21788 at commit [`22396b0`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #21853: [SPARK-23957][SQL] Sorts in subqueries are redundant and...

2018-07-23 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21853 LGTM except for minor comments --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21788: [SPARK-24609][ML][DOC] PySpark/SparkR doc doesn't explai...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21788 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #21788: [SPARK-24609][ML][DOC] PySpark/SparkR doc doesn't explai...

2018-07-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21788 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93472/ Test FAILed. ---

[GitHub] spark pull request #21853: [SPARK-23957][SQL] Sorts in subqueries are redund...

2018-07-23 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21853#discussion_r204612114 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SubquerySuite.scala --- @@ -970,4 +973,300 @@ class SubquerySuite extends QueryTest with SharedSQLCon

[GitHub] spark issue #21848: [SPARK-24890] [SQL] Short circuiting the `if` condition ...

2018-07-23 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21848 Hmm, seems we have limitation on where non deterministic expressions can be in. --- - To unsubscribe, e-mail: reviews-unsubscr...

[GitHub] spark pull request #21853: [SPARK-23957][SQL] Sorts in subqueries are redund...

2018-07-23 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21853#discussion_r204609653 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -164,10 +164,20 @@ abstract class Optimizer(sessionCata

[GitHub] spark pull request #21853: [SPARK-23957][SQL] Sorts in subqueries are redund...

2018-07-23 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21853#discussion_r204609622 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -164,10 +164,20 @@ abstract class Optimizer(sessionCata

[GitHub] spark pull request #21853: [SPARK-23957][SQL] Sorts in subqueries are redund...

2018-07-23 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21853#discussion_r204609532 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SubquerySuite.scala --- @@ -970,4 +973,300 @@ class SubquerySuite extends QueryTest with SharedSQLCon

[GitHub] spark pull request #21837: [SPARK-24881][SQL] New Avro options - compression...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21837#discussion_r204609385 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1430,6 +1431,18 @@ object SQLConf { "This only take

[GitHub] spark pull request #21837: [SPARK-24881][SQL] New Avro options - compression...

2018-07-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21837#discussion_r204609328 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1430,6 +1431,18 @@ object SQLConf { "This only take

[GitHub] spark pull request #21608: [SPARK-24626] [SQL] Improve location size calcula...

2018-07-23 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21608#discussion_r204609271 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Utils.scala --- @@ -55,4 +57,11 @@ private[sql] object DataSourc

  1   2   3   4   5   >