[GitHub] [spark] maropu commented on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

maropu commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-650637938 > Hi, All. So, this is a regression at 3.0.0 and we need this in branch-3.0 for Apache Spark 3.0.1? Yea, I think so, too: https://github.com/apache/spark/blob/branch-3.0/s

[GitHub] [spark] SparkQA commented on pull request #28923: [SPARK-32090][SQL] UserDefinedType.equal() should be symmetrical

SparkQA commented on pull request #28923: URL: https://github.com/apache/spark/pull/28923#issuecomment-650637926 **[Test build #124567 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124567/testReport)** for PR 28923 at commit [`b29cef0`](https://github.co

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r446499537 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,58 @@ class NestedColu

[GitHub] [spark] maropu commented on a change in pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

maropu commented on a change in pull request #28804: URL: https://github.com/apache/spark/pull/28804#discussion_r446571681 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -2196,6 +2196,13 @@ object SQLConf { .checkValue(bit =>

[GitHub] [spark] maropu edited a comment on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

maropu edited a comment on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-650552251 > it is more of a manual step and can be used only if the user knows the nature of data upfront.Like in my benchmark, where we expect the the all but few grouping keys to

[GitHub] [spark] maropu commented on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

maropu commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-650636794 > No, The Final aggregation will take care giving the right results. This is like more like setting the Aggregation mode to org.apache.spark.sql.catalyst.expressions.aggregate.

[GitHub] [spark] frankyin-factual commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

frankyin-factual commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r446568773 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,58 @@ class

[GitHub] [spark] AmplabJenkins commented on pull request #28934: [SPARK-32113][SQL] Avoid coalescing shuffle partitions if join condition has inequality predicate

AmplabJenkins commented on pull request #28934: URL: https://github.com/apache/spark/pull/28934#issuecomment-650629435 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28934: [SPARK-32113][SQL] Avoid coalescing shuffle partitions if join condition has inequality predicate

SparkQA removed a comment on pull request #28934: URL: https://github.com/apache/spark/pull/28934#issuecomment-650581756 **[Test build #124566 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124566/testReport)** for PR 28934 at commit [`1d88f08`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28934: [SPARK-32113][SQL] Avoid coalescing shuffle partitions if join condition has inequality predicate

AmplabJenkins removed a comment on pull request #28934: URL: https://github.com/apache/spark/pull/28934#issuecomment-650629435 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28934: [SPARK-32113][SQL] Avoid coalescing shuffle partitions if join condition has inequality predicate

SparkQA commented on pull request #28934: URL: https://github.com/apache/spark/pull/28934#issuecomment-650629035 **[Test build #124566 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124566/testReport)** for PR 28934 at commit [`1d88f08`](https://github.co

[GitHub] [spark] Fokko commented on pull request #28754: [SPARK-10520][SQL] Allow average out of DateType

Fokko commented on pull request #28754: URL: https://github.com/apache/spark/pull/28754#issuecomment-650624929 Hmm, weird that the test is failing. I've just pulled in the latest master to retrigger the tests. This is an aut

[GitHub] [spark] AmplabJenkins commented on pull request #28885: [SPARK-29375][SPARK-28940][SPARK-32041][SQL] Whole plan exchange and subquery reuse

AmplabJenkins commented on pull request #28885: URL: https://github.com/apache/spark/pull/28885#issuecomment-650623623 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28885: [SPARK-29375][SPARK-28940][SPARK-32041][SQL] Whole plan exchange and subquery reuse

AmplabJenkins removed a comment on pull request #28885: URL: https://github.com/apache/spark/pull/28885#issuecomment-650623623 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28885: [SPARK-29375][SPARK-28940][SPARK-32041][SQL] Whole plan exchange and subquery reuse

SparkQA commented on pull request #28885: URL: https://github.com/apache/spark/pull/28885#issuecomment-650623357 **[Test build #124572 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124572/testReport)** for PR 28885 at commit [`c49a0f9`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28754: [SPARK-10520][SQL] Allow average out of DateType

AmplabJenkins removed a comment on pull request #28754: URL: https://github.com/apache/spark/pull/28754#issuecomment-650608767 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124

[GitHub] [spark] AmplabJenkins commented on pull request #28754: [SPARK-10520][SQL] Allow average out of DateType

AmplabJenkins commented on pull request #28754: URL: https://github.com/apache/spark/pull/28754#issuecomment-650608763 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28754: [SPARK-10520][SQL] Allow average out of DateType

SparkQA removed a comment on pull request #28754: URL: https://github.com/apache/spark/pull/28754#issuecomment-650588843 **[Test build #124570 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124570/testReport)** for PR 28754 at commit [`533dd8d`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28754: [SPARK-10520][SQL] Allow average out of DateType

SparkQA commented on pull request #28754: URL: https://github.com/apache/spark/pull/28754#issuecomment-650608705 **[Test build #124570 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124570/testReport)** for PR 28754 at commit [`533dd8d`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28754: [SPARK-10520][SQL] Allow average out of DateType

AmplabJenkins removed a comment on pull request #28754: URL: https://github.com/apache/spark/pull/28754#issuecomment-650608763 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

AmplabJenkins removed a comment on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-650590610 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124

[GitHub] [spark] AmplabJenkins commented on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

AmplabJenkins commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-650590606 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

AmplabJenkins removed a comment on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-650590606 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

SparkQA removed a comment on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-650587304 **[Test build #124568 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124568/testReport)** for PR 28912 at commit [`d68e77b`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

SparkQA commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-650590552 **[Test build #124568 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124568/testReport)** for PR 28912 at commit [`d68e77b`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28629: [SPARK-31769][CORE] Add MDC support for driver threads

AmplabJenkins removed a comment on pull request #28629: URL: https://github.com/apache/spark/pull/28629#issuecomment-650589149 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

AmplabJenkins removed a comment on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-650589153 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124

[GitHub] [spark] AmplabJenkins commented on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

AmplabJenkins commented on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-650589150 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

AmplabJenkins removed a comment on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-650589150 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #28629: [SPARK-31769][CORE] Add MDC support for driver threads

SparkQA removed a comment on pull request #28629: URL: https://github.com/apache/spark/pull/28629#issuecomment-650588852 **[Test build #124571 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124571/testReport)** for PR 28629 at commit [`cc8d522`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

SparkQA removed a comment on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-650588854 **[Test build #124569 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124569/testReport)** for PR 28863 at commit [`e16b4a4`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28629: [SPARK-31769][CORE] Add MDC support for driver threads

AmplabJenkins removed a comment on pull request #28629: URL: https://github.com/apache/spark/pull/28629#issuecomment-650589148 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28541: [SPARK-31720][CORE] TaskMemoryManager allocate failed when new task coming

dongjoon-hyun commented on a change in pull request #28541: URL: https://github.com/apache/spark/pull/28541#discussion_r446547861 ## File path: core/src/main/scala/org/apache/spark/memory/ExecutionMemoryPool.scala ## @@ -105,11 +106,12 @@ private[memory] class ExecutionMemoryPo

[GitHub] [spark] SparkQA commented on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

SparkQA commented on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-650589145 **[Test build #124569 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124569/testReport)** for PR 28863 at commit [`e16b4a4`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #28629: [SPARK-31769][CORE] Add MDC support for driver threads

AmplabJenkins commented on pull request #28629: URL: https://github.com/apache/spark/pull/28629#issuecomment-650589148 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28629: [SPARK-31769][CORE] Add MDC support for driver threads

SparkQA commented on pull request #28629: URL: https://github.com/apache/spark/pull/28629#issuecomment-650589144 **[Test build #124571 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124571/testReport)** for PR 28629 at commit [`cc8d522`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

AmplabJenkins removed a comment on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-650589003 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

AmplabJenkins commented on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-650589003 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] dongjoon-hyun commented on pull request #28545: [WIP][SPARK-30090][SHELL] Adapt Spark REPL to Scala 2.13

dongjoon-hyun commented on pull request #28545: URL: https://github.com/apache/spark/pull/28545#issuecomment-650589010 Hi, @karolchmist . Thank you for contribution. Apache Spark 3.0.0 is released. Can we resume this work for Apache Spark 3.1.0 (December 2020)?

[GitHub] [spark] AmplabJenkins commented on pull request #28629: [SPARK-31769][CORE] Add MDC support for driver threads

AmplabJenkins commented on pull request #28629: URL: https://github.com/apache/spark/pull/28629#issuecomment-650588996 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28754: [SPARK-10520][SQL] Allow average out of DateType

AmplabJenkins removed a comment on pull request #28754: URL: https://github.com/apache/spark/pull/28754#issuecomment-650588999 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28754: [SPARK-10520][SQL] Allow average out of DateType

AmplabJenkins commented on pull request #28754: URL: https://github.com/apache/spark/pull/28754#issuecomment-650588999 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28629: [SPARK-31769][CORE] Add MDC support for driver threads

AmplabJenkins removed a comment on pull request #28629: URL: https://github.com/apache/spark/pull/28629#issuecomment-650588996 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

SparkQA commented on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-650588854 **[Test build #124569 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124569/testReport)** for PR 28863 at commit [`e16b4a4`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28629: [SPARK-31769][CORE] Add MDC support for driver threads

SparkQA commented on pull request #28629: URL: https://github.com/apache/spark/pull/28629#issuecomment-650588852 **[Test build #124571 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124571/testReport)** for PR 28629 at commit [`cc8d522`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28629: [SPARK-31769][CORE] Add MDC support for driver threads

AmplabJenkins removed a comment on pull request #28629: URL: https://github.com/apache/spark/pull/28629#issuecomment-633186963 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] SparkQA commented on pull request #28754: [SPARK-10520][SQL] Allow average out of DateType

SparkQA commented on pull request #28754: URL: https://github.com/apache/spark/pull/28754#issuecomment-650588843 **[Test build #124570 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124570/testReport)** for PR 28754 at commit [`533dd8d`](https://github.com

[GitHub] [spark] dongjoon-hyun commented on pull request #28629: [SPARK-31769][CORE] Add MDC support for driver threads

dongjoon-hyun commented on pull request #28629: URL: https://github.com/apache/spark/pull/28629#issuecomment-650588725 Retest this please This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28754: [SPARK-10520][SQL] Allow average out of DateType

AmplabJenkins removed a comment on pull request #28754: URL: https://github.com/apache/spark/pull/28754#issuecomment-640557338 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] dongjoon-hyun commented on pull request #28754: [SPARK-10520][SQL] Allow average out of DateType

dongjoon-hyun commented on pull request #28754: URL: https://github.com/apache/spark/pull/28754#issuecomment-650588530 Retest this please. This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [spark] dongjoon-hyun commented on pull request #28863: [SPARK-31336][SQL] Support Oracle Kerberos login in JDBC connector

dongjoon-hyun commented on pull request #28863: URL: https://github.com/apache/spark/pull/28863#issuecomment-650588395 Retest this please This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28865: [SPARK-32026][CORE] Support consistency on Prometheus driver and executor metrics format

dongjoon-hyun commented on a change in pull request #28865: URL: https://github.com/apache/spark/pull/28865#discussion_r446547164 ## File path: core/src/test/scala/org/apache/spark/metrics/sink/PrometheusServletSuite.scala ## @@ -0,0 +1,64 @@ +/* + * Licensed to the Apache Sof

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28865: [SPARK-32026][CORE] Support consistency on Prometheus driver and executor metrics format

dongjoon-hyun edited a comment on pull request #28865: URL: https://github.com/apache/spark/pull/28865#issuecomment-650588035 No. Please test on Apache Spark 3.0.0. DropWizard 4 itself introduced type attribute(e.g: type="gauges") and `PrometheusSevlet` just follows that inevitably. > P

[GitHub] [spark] dongjoon-hyun commented on pull request #28865: [SPARK-32026][CORE] Support consistency on Prometheus driver and executor metrics format

dongjoon-hyun commented on pull request #28865: URL: https://github.com/apache/spark/pull/28865#issuecomment-650588035 Please test on Apache Spark 3.0.0. DropWizard 4 itself introduced type attribute(e.g: type="gauges") and `PrometheusSevlet` just follows that inevitably. ---

[GitHub] [spark] dongjoon-hyun commented on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

dongjoon-hyun commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-650587607 Hi, All. So, this is a regression at 3.0.0 and we need this in `branch-3.0` for Apache Spark 3.0.1? This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

AmplabJenkins removed a comment on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-650587448 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

AmplabJenkins commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-650587448 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

SparkQA commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-650587304 **[Test build #124568 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124568/testReport)** for PR 28912 at commit [`d68e77b`](https://github.com

[GitHub] [spark] dongjoon-hyun commented on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

dongjoon-hyun commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-650586938 Retest this please. This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [spark] AmplabJenkins commented on pull request #28923: [SPARK-32090][SQL] UserDefinedType.equal() should be symmetrical

AmplabJenkins commented on pull request #28923: URL: https://github.com/apache/spark/pull/28923#issuecomment-650586572 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28923: [SPARK-32090][SQL] UserDefinedType.equal() should be symmetrical

AmplabJenkins removed a comment on pull request #28923: URL: https://github.com/apache/spark/pull/28923#issuecomment-650586572 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28923: [SPARK-32090][SQL] UserDefinedType.equal() should be symmetrical

SparkQA commented on pull request #28923: URL: https://github.com/apache/spark/pull/28923#issuecomment-650586478 **[Test build #124567 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124567/testReport)** for PR 28923 at commit [`b29cef0`](https://github.com

[GitHub] [spark] dongjoon-hyun commented on pull request #28923: [SPARK-32090][SQL] UserDefinedType.equal() should be symmetrical

dongjoon-hyun commented on pull request #28923: URL: https://github.com/apache/spark/pull/28923#issuecomment-650586329 Retest this please. This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [spark] karuppayya commented on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

karuppayya commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-650585266 > > it is more of a manual step and can be used only if the user knows the nature of data upfront.Like in my benchmark, where we expect the the all but few grouping keys to b

[GitHub] [spark] wangyum commented on pull request #28934: [SPARK-32113][SQL] Avoid coalescing shuffle partitions if join condition has inequality predicate

wangyum commented on pull request #28934: URL: https://github.com/apache/spark/pull/28934#issuecomment-650582509 Another case: Default | Avoid coalescing shuffle partitions --- | --- https://user-images.githubusercontent.com/5399861/85927019-f3707600-b8d5-11ea-8f61-d2456037d02b.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28934: [SPARK-32113][SQL] Avoid coalescing shuffle partitions if join condition has inequality predicate

AmplabJenkins removed a comment on pull request #28934: URL: https://github.com/apache/spark/pull/28934#issuecomment-650580251 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28934: [SPARK-32113][SQL] Avoid coalescing shuffle partitions if join condition has inequality predicate

SparkQA commented on pull request #28934: URL: https://github.com/apache/spark/pull/28934#issuecomment-650581756 **[Test build #124566 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124566/testReport)** for PR 28934 at commit [`1d88f08`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #28934: [SPARK-32113][SQL] Avoid coalescing shuffle partitions if join condition has inequality predicate

AmplabJenkins commented on pull request #28934: URL: https://github.com/apache/spark/pull/28934#issuecomment-650580251 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] wangyum opened a new pull request #28934: [SPARK-32113][SQL] Avoid coalescing shuffle partitions if join condition has inequality predicate

wangyum opened a new pull request #28934: URL: https://github.com/apache/spark/pull/28934 ### What changes were proposed in this pull request? The data usually expand if joining event-based table(Chinese named 拉链表). This PR makes it avoid coalescing shuffle partitions if joining even

[GitHub] [spark] viirya commented on a change in pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

viirya commented on a change in pull request #27690: URL: https://github.com/apache/spark/pull/27690#discussion_r446540549 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/SaveAsHiveFile.scala ## @@ -97,12 +99,38 @@ private[hive] trait SaveAsHiveFile e

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

AmplabJenkins removed a comment on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650572743 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

AmplabJenkins commented on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650572743 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

SparkQA commented on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650572247 **[Test build #124565 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124565/testReport)** for PR 28676 at commit [`488e051`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

SparkQA removed a comment on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650536600 **[Test build #124565 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124565/testReport)** for PR 28676 at commit [`488e051`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28841: [SPARK-31962][SQL][SS] Provide option to load files after a specified date when reading from a folder path

AmplabJenkins removed a comment on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-650558195 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28841: [SPARK-31962][SQL][SS] Provide option to load files after a specified date when reading from a folder path

AmplabJenkins commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-650558195 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28841: [SPARK-31962][SQL][SS] Provide option to load files after a specified date when reading from a folder path

SparkQA commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-650557976 **[Test build #124564 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124564/testReport)** for PR 28841 at commit [`7ee667d`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #28841: [SPARK-31962][SQL][SS] Provide option to load files after a specified date when reading from a folder path

SparkQA removed a comment on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-650519071 **[Test build #124564 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124564/testReport)** for PR 28841 at commit [`7ee667d`](https://gi

[GitHub] [spark] maropu commented on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

maropu commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-650552251 > it is more of a manual step and can be used only if the user knows the nature of data upfront.Like in my benchmark, where we expect the the all but few grouping keys to be diff

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28919: [SPARK-32038][SQL][FOLLOWUP] Make the alias name pretty after float/double normalization

AmplabJenkins removed a comment on pull request #28919: URL: https://github.com/apache/spark/pull/28919#issuecomment-650551433 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28919: [SPARK-32038][SQL][FOLLOWUP] Make the alias name pretty after float/double normalization

AmplabJenkins commented on pull request #28919: URL: https://github.com/apache/spark/pull/28919#issuecomment-650551433 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28919: [SPARK-32038][SQL][FOLLOWUP] Make the alias name pretty after float/double normalization

SparkQA removed a comment on pull request #28919: URL: https://github.com/apache/spark/pull/28919#issuecomment-650512191 **[Test build #124563 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124563/testReport)** for PR 28919 at commit [`c1d4321`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28919: [SPARK-32038][SQL][FOLLOWUP] Make the alias name pretty after float/double normalization

SparkQA commented on pull request #28919: URL: https://github.com/apache/spark/pull/28919#issuecomment-650551206 **[Test build #124563 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124563/testReport)** for PR 28919 at commit [`c1d4321`](https://github.co

[GitHub] [spark] dbaliafroozeh commented on a change in pull request #28885: [SPARK-29375][SPARK-28940][SPARK-32041][SQL] Whole plan exchange and subquery reuse

dbaliafroozeh commented on a change in pull request #28885: URL: https://github.com/apache/spark/pull/28885#discussion_r446516852 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/reuse/Reuse.scala ## @@ -0,0 +1,95 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] moomindani commented on a change in pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

moomindani commented on a change in pull request #27690: URL: https://github.com/apache/spark/pull/27690#discussion_r446516892 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/SaveAsHiveFile.scala ## @@ -97,12 +99,38 @@ private[hive] trait SaveAsHiveFi

[GitHub] [spark] dbaliafroozeh commented on a change in pull request #28885: [SPARK-29375][SPARK-28940][SPARK-32041][SQL] Whole plan exchange and subquery reuse

dbaliafroozeh commented on a change in pull request #28885: URL: https://github.com/apache/spark/pull/28885#discussion_r446514163 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala ## @@ -326,7 +327,8 @@ object QueryExecution { */

[GitHub] [spark] dbaliafroozeh commented on a change in pull request #28885: [SPARK-29375][SPARK-28940][SPARK-32041][SQL] Whole plan exchange and subquery reuse

dbaliafroozeh commented on a change in pull request #28885: URL: https://github.com/apache/spark/pull/28885#discussion_r446514163 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala ## @@ -326,7 +327,8 @@ object QueryExecution { */

[GitHub] [spark] maropu commented on pull request #28912: [SPARK-32057][SQL] ExecuteStatement: cancel and close should not transiently ERROR

maropu commented on pull request #28912: URL: https://github.com/apache/spark/pull/28912#issuecomment-650537977 hm..., I'm not sure about the root cause, but I think the simplest way to fix the issue is that we just remove the mockito part if its possible to test this PR without it.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

AmplabJenkins removed a comment on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650536783 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] maropu commented on pull request #28425: [SPARK-31480][SQL] Improve the EXPLAIN FORMATTED's output for DSV2's Scan Node

maropu commented on pull request #28425: URL: https://github.com/apache/spark/pull/28425#issuecomment-650536856 kindly ping @gengliangwang @cloud-fan This is an automated message from the Apache Git Service. To respond to th

[GitHub] [spark] maropu commented on a change in pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

maropu commented on a change in pull request #28852: URL: https://github.com/apache/spark/pull/28852#discussion_r446509317 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileIndexSuite.scala ## @@ -488,6 +489,25 @@ class FileIndexSuite extends

[GitHub] [spark] AmplabJenkins commented on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

AmplabJenkins commented on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650536783 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

SparkQA commented on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650536600 **[Test build #124565 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124565/testReport)** for PR 28676 at commit [`488e051`](https://github.com

[GitHub] [spark] maropu commented on a change in pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

maropu commented on a change in pull request #28852: URL: https://github.com/apache/spark/pull/28852#discussion_r446509229 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveMetadataCacheSuite.scala ## @@ -126,4 +129,39 @@ class HiveMetadataCacheSuite extends

[GitHub] [spark] maropu commented on pull request #28676: [WIP][SPARK-31869][SQL] BroadcastHashJoinExec can utilize the build side for its output partitioning

maropu commented on pull request #28676: URL: https://github.com/apache/spark/pull/28676#issuecomment-650535975 retest this please This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r446508750 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -39,6 +39,22 @@ object NestedColumnAlia

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r446502299 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -39,6 +39,22 @@ object NestedColumnAlia

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r446500230 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/SchemaPruningSuite.scala ## @@ -460,6 +460,40 @@ abstract class SchemaPru

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r446499982 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/SchemaPruningSuite.scala ## @@ -460,6 +460,40 @@ abstract class SchemaPru

[GitHub] [spark] maropu commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

maropu commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r446499910 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,58 @@ class NestedColu

<    1   2   3   4   >