[GitHub] [spark] SparkQA commented on pull request #32388: [SPARK-35258][SHUFFLE][YARN] Add new metrics to ExternalShuffleService for better monitoring

2021-05-10 Thread GitBox
SparkQA commented on pull request #32388: URL: https://github.com/apache/spark/pull/32388#issuecomment-837178915 **[Test build #138340 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138340/testReport)** for PR 32388 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32389: [SPARK-35263] [TEST] Refactor ShuffleBlockFetcherIteratorSuite to reduce duplicated code

2021-05-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32389: URL: https://github.com/apache/spark/pull/32389#issuecomment-837177734 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138338/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

2021-05-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-837177729 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42863/

[GitHub] [spark] AmplabJenkins commented on pull request #32389: [SPARK-35263] [TEST] Refactor ShuffleBlockFetcherIteratorSuite to reduce duplicated code

2021-05-10 Thread GitBox
AmplabJenkins commented on pull request #32389: URL: https://github.com/apache/spark/pull/32389#issuecomment-837177734 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138338/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

2021-05-10 Thread GitBox
AmplabJenkins commented on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-837177729 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42863/ --

[GitHub] [spark] viirya commented on a change in pull request #32448: [SPARK-35290][SQL] Use StructType merging for unionByName with null filling

2021-05-10 Thread GitBox
viirya commented on a change in pull request #32448: URL: https://github.com/apache/spark/pull/32448#discussion_r629618659 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SchemaPruning.scala ## @@ -38,7 +38,7 @@ object SchemaPruning extends

[GitHub] [spark] viirya commented on a change in pull request #32448: [SPARK-35290][SQL] Use StructType merging for unionByName with null filling

2021-05-10 Thread GitBox
viirya commented on a change in pull request #32448: URL: https://github.com/apache/spark/pull/32448#discussion_r629616627 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/StructType.scala ## @@ -483,8 +483,8 @@ case class StructType(fields:

[GitHub] [spark] SparkQA commented on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

2021-05-10 Thread GitBox
SparkQA commented on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-837171174 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42863/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

2021-05-10 Thread GitBox
SparkQA commented on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-837164157 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42863/ -- This is an automated message from the Apache

[GitHub] [spark] viirya commented on a change in pull request #32457: [SPARK-35329][SQL] Split generated switch code into pieces in ExpandExec

2021-05-10 Thread GitBox
viirya commented on a change in pull request #32457: URL: https://github.com/apache/spark/pull/32457#discussion_r629613414 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/ExpandExec.scala ## @@ -152,38 +152,66 @@ case class ExpandExec( // This

[GitHub] [spark] SparkQA removed a comment on pull request #32389: [SPARK-35263] [TEST] Refactor ShuffleBlockFetcherIteratorSuite to reduce duplicated code

2021-05-10 Thread GitBox
SparkQA removed a comment on pull request #32389: URL: https://github.com/apache/spark/pull/32389#issuecomment-836917482 **[Test build #138338 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138338/testReport)** for PR 32389 at commit

[GitHub] [spark] SparkQA commented on pull request #32389: [SPARK-35263] [TEST] Refactor ShuffleBlockFetcherIteratorSuite to reduce duplicated code

2021-05-10 Thread GitBox
SparkQA commented on pull request #32389: URL: https://github.com/apache/spark/pull/32389#issuecomment-837148852 **[Test build #138338 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138338/testReport)** for PR 32389 at commit

[GitHub] [spark] SparkQA commented on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

2021-05-10 Thread GitBox
SparkQA commented on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-837081829 **[Test build #138341 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138341/testReport)** for PR 32303 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32415: [SPARK-35295][ML] Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0

2021-05-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32415: URL: https://github.com/apache/spark/pull/32415#issuecomment-837079970 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42861/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32493: [SPARK-34246][FOLLOWUP] Change the definition of `findTightestCommonType` for backward compatibility

2021-05-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32493: URL: https://github.com/apache/spark/pull/32493#issuecomment-837079963 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138335/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32388: [SPARK-35258][SHUFFLE][YARN] Add new metrics to ExternalShuffleService for better monitoring

2021-05-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32388: URL: https://github.com/apache/spark/pull/32388#issuecomment-837079972 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42862/

[GitHub] [spark] AmplabJenkins commented on pull request #32493: [SPARK-34246][FOLLOWUP] Change the definition of `findTightestCommonType` for backward compatibility

2021-05-10 Thread GitBox
AmplabJenkins commented on pull request #32493: URL: https://github.com/apache/spark/pull/32493#issuecomment-837079963 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138335/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32388: [SPARK-35258][SHUFFLE][YARN] Add new metrics to ExternalShuffleService for better monitoring

2021-05-10 Thread GitBox
AmplabJenkins commented on pull request #32388: URL: https://github.com/apache/spark/pull/32388#issuecomment-837079972 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42862/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32415: [SPARK-35295][ML] Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0

2021-05-10 Thread GitBox
AmplabJenkins commented on pull request #32415: URL: https://github.com/apache/spark/pull/32415#issuecomment-837079970 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42861/ --

[GitHub] [spark] sarutak commented on pull request #32317: [SPARK-33195][UI] Fix stages/stage UI page fails because of uri parameters encoded twice

2021-05-10 Thread GitBox
sarutak commented on pull request #32317: URL: https://github.com/apache/spark/pull/32317#issuecomment-837075235 @mdianjun Sorry for the late reply. This PR is for the case that `spark.ui.proxyRedirectUri` is `true` right? If so, should we rather fix `ProxyRedirectHandler` ? I

[GitHub] [spark] SparkQA commented on pull request #32388: [SPARK-35258][SHUFFLE][YARN] Add new metrics to ExternalShuffleService for better monitoring

2021-05-10 Thread GitBox
SparkQA commented on pull request #32388: URL: https://github.com/apache/spark/pull/32388#issuecomment-837069568 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] sarutak commented on a change in pull request #32317: [SPARK-33195][UI] Fix stages/stage UI page fails because of uri parameters encoded twice

2021-05-10 Thread GitBox
sarutak commented on a change in pull request #32317: URL: https://github.com/apache/spark/pull/32317#discussion_r629564122 ## File path: core/src/main/scala/org/apache/spark/status/api/v1/StagesResource.scala ## @@ -152,10 +153,11 @@ private[v1] class StagesResource extends

[GitHub] [spark] SparkQA commented on pull request #32415: [SPARK-35295][ML] Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0

2021-05-10 Thread GitBox
SparkQA commented on pull request #32415: URL: https://github.com/apache/spark/pull/32415#issuecomment-837057448 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] SparkQA removed a comment on pull request #32493: [SPARK-34246][FOLLOWUP] Change the definition of `findTightestCommonType` for backward compatibility

2021-05-10 Thread GitBox
SparkQA removed a comment on pull request #32493: URL: https://github.com/apache/spark/pull/32493#issuecomment-836697327 **[Test build #138335 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138335/testReport)** for PR 32493 at commit

[GitHub] [spark] SparkQA commented on pull request #32493: [SPARK-34246][FOLLOWUP] Change the definition of `findTightestCommonType` for backward compatibility

2021-05-10 Thread GitBox
SparkQA commented on pull request #32493: URL: https://github.com/apache/spark/pull/32493#issuecomment-837032862 **[Test build #138335 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138335/testReport)** for PR 32493 at commit

[GitHub] [spark] SparkQA commented on pull request #32388: [SPARK-35258][SHUFFLE][YARN] Add new metrics to ExternalShuffleService for better monitoring

2021-05-10 Thread GitBox
SparkQA commented on pull request #32388: URL: https://github.com/apache/spark/pull/32388#issuecomment-836992973 **[Test build #138340 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138340/testReport)** for PR 32388 at commit

[GitHub] [spark] SparkQA commented on pull request #32415: [SPARK-35295][ML] Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0

2021-05-10 Thread GitBox
SparkQA commented on pull request #32415: URL: https://github.com/apache/spark/pull/32415#issuecomment-836992955 **[Test build #138339 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138339/testReport)** for PR 32415 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32287: [SPARK-27991][CORE] Defer the fetch request on Netty OOM

2021-05-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32287: URL: https://github.com/apache/spark/pull/32287#issuecomment-836989736 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138337/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32389: [SPARK-35263] [TEST] Refactor ShuffleBlockFetcherIteratorSuite to reduce duplicated code

2021-05-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32389: URL: https://github.com/apache/spark/pull/32389#issuecomment-836989725 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42860/

[GitHub] [spark] AmplabJenkins commented on pull request #32287: [SPARK-27991][CORE] Defer the fetch request on Netty OOM

2021-05-10 Thread GitBox
AmplabJenkins commented on pull request #32287: URL: https://github.com/apache/spark/pull/32287#issuecomment-836989736 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138337/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32389: [SPARK-35263] [TEST] Refactor ShuffleBlockFetcherIteratorSuite to reduce duplicated code

2021-05-10 Thread GitBox
AmplabJenkins commented on pull request #32389: URL: https://github.com/apache/spark/pull/32389#issuecomment-836989725 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42860/ --

[GitHub] [spark] SparkQA commented on pull request #32389: [SPARK-35263] [TEST] Refactor ShuffleBlockFetcherIteratorSuite to reduce duplicated code

2021-05-10 Thread GitBox
SparkQA commented on pull request #32389: URL: https://github.com/apache/spark/pull/32389#issuecomment-836979529 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] attilapiros closed pull request #31829: [SPARK-34736][K8S][TESTS] Kubernetes and Minikube version upgrade for integration tests

2021-05-10 Thread GitBox
attilapiros closed pull request #31829: URL: https://github.com/apache/spark/pull/31829 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] sigmod commented on a change in pull request #32298: [SPARK-34079][SQL] Merge non-correlated scalar subqueries to multi-column scalar subqueries for better reuse

2021-05-10 Thread GitBox
sigmod commented on a change in pull request #32298: URL: https://github.com/apache/spark/pull/32298#discussion_r629516965 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/MergeScalarSubqueries.scala ## @@ -0,0 +1,184 @@ +/* + * Licensed to

[GitHub] [spark] attilapiros commented on pull request #31829: [SPARK-34736][K8S][TESTS] Kubernetes and Minikube version upgrade for integration tests

2021-05-10 Thread GitBox
attilapiros commented on pull request #31829: URL: https://github.com/apache/spark/pull/31829#issuecomment-836953989 Thanks @dongjoon-hyun. Tested locally: ``` [INFO] --- scalatest-maven-plugin:2.0.0:test (integration-test) @ spark-kubernetes-integration-tests_2.12 ---

[GitHub] [spark] SparkQA removed a comment on pull request #32287: [SPARK-27991][CORE] Defer the fetch request on Netty OOM

2021-05-10 Thread GitBox
SparkQA removed a comment on pull request #32287: URL: https://github.com/apache/spark/pull/32287#issuecomment-836758152 **[Test build #138337 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138337/testReport)** for PR 32287 at commit

[GitHub] [spark] srowen commented on pull request #32415: [SPARK-35295][ML] Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0

2021-05-10 Thread GitBox
srowen commented on pull request #32415: URL: https://github.com/apache/spark/pull/32415#issuecomment-836946172 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries

[GitHub] [spark] SparkQA commented on pull request #32287: [SPARK-27991][CORE] Defer the fetch request on Netty OOM

2021-05-10 Thread GitBox
SparkQA commented on pull request #32287: URL: https://github.com/apache/spark/pull/32287#issuecomment-836946002 **[Test build #138337 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138337/testReport)** for PR 32287 at commit

[GitHub] [spark] xkrogen commented on pull request #32388: [SPARK-35258][SHUFFLE][YARN] Add new metrics to ExternalShuffleService for better monitoring

2021-05-10 Thread GitBox
xkrogen commented on pull request #32388: URL: https://github.com/apache/spark/pull/32388#issuecomment-836943054 Thanks for the comments @dongjoon-hyun ! I have pushed up new commits addressing them. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] xkrogen commented on a change in pull request #32388: [SPARK-35258][SHUFFLE][YARN] Add new metrics to ExternalShuffleService for better monitoring

2021-05-10 Thread GitBox
xkrogen commented on a change in pull request #32388: URL: https://github.com/apache/spark/pull/32388#discussion_r629508528 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalBlockHandler.java ## @@ -291,9 +294,18 @@ public

[GitHub] [spark] xkrogen commented on a change in pull request #32388: [SPARK-35258][SHUFFLE][YARN] Add new metrics to ExternalShuffleService for better monitoring

2021-05-10 Thread GitBox
xkrogen commented on a change in pull request #32388: URL: https://github.com/apache/spark/pull/32388#discussion_r629507805 ## File path: common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleServiceMetrics.java ## @@ -78,7 +82,17 @@ public static void

[GitHub] [spark] luhenry commented on pull request #32415: [SPARK-35295][ML] Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0

2021-05-10 Thread GitBox
luhenry commented on pull request #32415: URL: https://github.com/apache/spark/pull/32415#issuecomment-836932214 @srowen the profile is already back at https://github.com/apache/spark/pull/32415/files#diff-9c5fb3d1b7e3b0f54bc5c4182965c4fe1f9023d449017cece3005d3f90e8e4d8R3499. I think it's

[GitHub] [spark] SparkQA commented on pull request #32389: [SPARK-35263] [TEST] Refactor ShuffleBlockFetcherIteratorSuite to reduce duplicated code

2021-05-10 Thread GitBox
SparkQA commented on pull request #32389: URL: https://github.com/apache/spark/pull/32389#issuecomment-836917482 **[Test build #138338 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138338/testReport)** for PR 32389 at commit

[GitHub] [spark] xkrogen commented on pull request #32389: [SPARK-35263] [TEST] Refactor ShuffleBlockFetcherIteratorSuite to reduce duplicated code

2021-05-10 Thread GitBox
xkrogen commented on pull request #32389: URL: https://github.com/apache/spark/pull/32389#issuecomment-836915607 Great comments @Ngone51 ! Pushed up a new set of commits addressing your comments. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] xkrogen commented on a change in pull request #32389: [SPARK-35263] [TEST] Refactor ShuffleBlockFetcherIteratorSuite to reduce duplicated code

2021-05-10 Thread GitBox
xkrogen commented on a change in pull request #32389: URL: https://github.com/apache/spark/pull/32389#discussion_r629498336 ## File path: core/src/test/scala/org/apache/spark/storage/ShuffleBlockFetcherIteratorSuite.scala ## @@ -123,6 +131,54 @@ class

[GitHub] [spark] viirya commented on pull request #32487: [SPARK-35358][BUILD] Increase maximum Java heap used for release build to avoid OOM

2021-05-10 Thread GitBox
viirya commented on pull request #32487: URL: https://github.com/apache/spark/pull/32487#issuecomment-836905485 Thanks all! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32470: [WIP] Simplify ResolveAggregateFunctions

2021-05-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32470: URL: https://github.com/apache/spark/pull/32470#issuecomment-836904086 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138336/

[GitHub] [spark] AmplabJenkins commented on pull request #32470: [WIP] Simplify ResolveAggregateFunctions

2021-05-10 Thread GitBox
AmplabJenkins commented on pull request #32470: URL: https://github.com/apache/spark/pull/32470#issuecomment-836904086 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138336/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #32470: [WIP] Simplify ResolveAggregateFunctions

2021-05-10 Thread GitBox
SparkQA removed a comment on pull request #32470: URL: https://github.com/apache/spark/pull/32470#issuecomment-836757567 **[Test build #138336 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138336/testReport)** for PR 32470 at commit

[GitHub] [spark] SparkQA commented on pull request #32470: [WIP] Simplify ResolveAggregateFunctions

2021-05-10 Thread GitBox
SparkQA commented on pull request #32470: URL: https://github.com/apache/spark/pull/32470#issuecomment-836890194 **[Test build #138336 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138336/testReport)** for PR 32470 at commit

[GitHub] [spark] xkrogen commented on a change in pull request #32389: [SPARK-35263] [TEST] Refactor ShuffleBlockFetcherIteratorSuite to reduce duplicated code

2021-05-10 Thread GitBox
xkrogen commented on a change in pull request #32389: URL: https://github.com/apache/spark/pull/32389#discussion_r629467040 ## File path: core/src/test/scala/org/apache/spark/storage/ShuffleBlockFetcherIteratorSuite.scala ## @@ -228,117 +272,72 @@ class

[GitHub] [spark] gengliangwang closed pull request #32493: [SPARK-34246][FOLLOWUP] Change the definition of `findTightestCommonType` for backward compatibility

2021-05-10 Thread GitBox
gengliangwang closed pull request #32493: URL: https://github.com/apache/spark/pull/32493 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] gengliangwang commented on pull request #32493: [SPARK-34246][FOLLOWUP] Change the definition of `findTightestCommonType` for backward compatibility

2021-05-10 Thread GitBox
gengliangwang commented on pull request #32493: URL: https://github.com/apache/spark/pull/32493#issuecomment-836837534 Thanks, merging to master -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] cloud-fan commented on a change in pull request #32482: [SPARK-35332][SQL] Make cache plan disable configs configurable

2021-05-10 Thread GitBox
cloud-fan commented on a change in pull request #32482: URL: https://github.com/apache/spark/pull/32482#discussion_r629452119 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -1090,6 +1090,17 @@ object SQLConf { .booleanConf

[GitHub] [spark] cloud-fan commented on a change in pull request #25854: [SPARK-29145][SQL] Support sub-queries in join conditions

2021-05-10 Thread GitBox
cloud-fan commented on a change in pull request #25854: URL: https://github.com/apache/spark/pull/25854#discussion_r629451135 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1697,6 +1697,8 @@ class Analyzer( //

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32287: [SPARK-27991][CORE] Defer the fetch request on Netty OOM

2021-05-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32287: URL: https://github.com/apache/spark/pull/32287#issuecomment-836823212 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42859/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32470: [WIP] Simplify ResolveAggregateFunctions

2021-05-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32470: URL: https://github.com/apache/spark/pull/32470#issuecomment-836823210 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42858/

[GitHub] [spark] AmplabJenkins commented on pull request #32470: [WIP] Simplify ResolveAggregateFunctions

2021-05-10 Thread GitBox
AmplabJenkins commented on pull request #32470: URL: https://github.com/apache/spark/pull/32470#issuecomment-836823210 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42858/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32287: [SPARK-27991][CORE] Defer the fetch request on Netty OOM

2021-05-10 Thread GitBox
AmplabJenkins commented on pull request #32287: URL: https://github.com/apache/spark/pull/32287#issuecomment-836823212 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42859/ --

[GitHub] [spark] cloud-fan edited a comment on pull request #32493: [SPARK-34246][FOLLOWUP] Change the definition of `findTightestCommonType` for backward compatibility

2021-05-10 Thread GitBox
cloud-fan edited a comment on pull request #32493: URL: https://github.com/apache/spark/pull/32493#issuecomment-836822576 Generally speaking, we don't need to keep backward compatibility for internal APIs. But this one is just a small change to avoid breaking a MongoDB connector, which

[GitHub] [spark] cloud-fan commented on pull request #32493: [SPARK-34246][FOLLOWUP] Change the definition of `findTightestCommonType` for backward compatibility

2021-05-10 Thread GitBox
cloud-fan commented on pull request #32493: URL: https://github.com/apache/spark/pull/32493#issuecomment-836822576 Generally speaking, we don't need to keep backward compatibility for internal APIs. But this one is just a small change to avoid breaking a MongoDB connector, which seems

[GitHub] [spark] SparkQA commented on pull request #32287: [SPARK-27991][CORE] Defer the fetch request on Netty OOM

2021-05-10 Thread GitBox
SparkQA commented on pull request #32287: URL: https://github.com/apache/spark/pull/32287#issuecomment-836813317 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] SparkQA commented on pull request #32470: [WIP] Simplify ResolveAggregateFunctions

2021-05-10 Thread GitBox
SparkQA commented on pull request #32470: URL: https://github.com/apache/spark/pull/32470#issuecomment-836806244 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] cloud-fan commented on a change in pull request #31756: [SPARK-34637] [SQL] Support DPP + AQE when the broadcast exchange can be reused

2021-05-10 Thread GitBox
cloud-fan commented on a change in pull request #31756: URL: https://github.com/apache/spark/pull/31756#discussion_r629432669 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DynamicPartitionPruningSuite.scala ## @@ -1463,6 +1474,37 @@ abstract class

[GitHub] [spark] cloud-fan commented on a change in pull request #31756: [SPARK-34637] [SQL] Support DPP + AQE when the broadcast exchange can be reused

2021-05-10 Thread GitBox
cloud-fan commented on a change in pull request #31756: URL: https://github.com/apache/spark/pull/31756#discussion_r629430388 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DynamicPartitionPruningSuite.scala ## @@ -1463,6 +1474,37 @@ abstract class

[GitHub] [spark] cloud-fan commented on a change in pull request #31756: [SPARK-34637] [SQL] Support DPP + AQE when the broadcast exchange can be reused

2021-05-10 Thread GitBox
cloud-fan commented on a change in pull request #31756: URL: https://github.com/apache/spark/pull/31756#discussion_r629426795 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DynamicPartitionPruningSuite.scala ## @@ -1463,6 +1474,37 @@ abstract class

[GitHub] [spark] cloud-fan commented on a change in pull request #31756: [SPARK-34637] [SQL] Support DPP + AQE when the broadcast exchange can be reused

2021-05-10 Thread GitBox
cloud-fan commented on a change in pull request #31756: URL: https://github.com/apache/spark/pull/31756#discussion_r629426175 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DynamicPartitionPruningSuite.scala ## @@ -1463,6 +1474,37 @@ abstract class

[GitHub] [spark] cloud-fan commented on a change in pull request #31756: [SPARK-34637] [SQL] Support DPP + AQE when the broadcast exchange can be reused

2021-05-10 Thread GitBox
cloud-fan commented on a change in pull request #31756: URL: https://github.com/apache/spark/pull/31756#discussion_r629425441 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/PlanAdaptiveDynamicPruningFilters.scala ## @@ -44,12 +42,23 @@ case

[GitHub] [spark] hezuojiao commented on pull request #32417: [SPARK-35289][SQL] Make CatalogString contains nullable information

2021-05-10 Thread GitBox
hezuojiao commented on pull request #32417: URL: https://github.com/apache/spark/pull/32417#issuecomment-836789769 New messages about `MapType` and `StructType` has been updated in the description of this PR. @wzhfy -- This is an automated message from the Apache Git Service. To

[GitHub] [spark] yikf commented on pull request #32486: [SPARK-35356][SQL] Fix issue of the createTable when externalCatalog is InMemoryCatalog

2021-05-10 Thread GitBox
yikf commented on pull request #32486: URL: https://github.com/apache/spark/pull/32486#issuecomment-836770138 @HyukjinKwon could you help take a look when you have time? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] yikf commented on pull request #32483: [SPARK-35355][SQL] In the case of insert with limit, improve execution performance

2021-05-10 Thread GitBox
yikf commented on pull request #32483: URL: https://github.com/apache/spark/pull/32483#issuecomment-836769786 @HyukjinKwon could you help take a look when you have time? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #32287: [SPARK-27991][CORE] Defer the fetch request on Netty OOM

2021-05-10 Thread GitBox
SparkQA commented on pull request #32287: URL: https://github.com/apache/spark/pull/32287#issuecomment-836758152 **[Test build #138337 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138337/testReport)** for PR 32287 at commit

[GitHub] [spark] SparkQA commented on pull request #32470: [WIP] Simplify ResolveAggregateFunctions

2021-05-10 Thread GitBox
SparkQA commented on pull request #32470: URL: https://github.com/apache/spark/pull/32470#issuecomment-836757567 **[Test build #138336 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138336/testReport)** for PR 32470 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32485: [SPARK-35357][GRAPHX] Allow to turn off the normalization applied by static PageRank utilities

2021-05-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32485: URL: https://github.com/apache/spark/pull/32485#issuecomment-836752923 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138334/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32492: [SPARK-35088][SQL][FOLLOWUP] Improve the error message for Sequence expression

2021-05-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32492: URL: https://github.com/apache/spark/pull/32492#issuecomment-836752920 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138331/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32493: [SPARK-34246][FOLLOWUP] Change the definition of `findTightestCommonType` for backward compatibility

2021-05-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32493: URL: https://github.com/apache/spark/pull/32493#issuecomment-836752924 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42857/

[GitHub] [spark] AmplabJenkins commented on pull request #32493: [SPARK-34246][FOLLOWUP] Change the definition of `findTightestCommonType` for backward compatibility

2021-05-10 Thread GitBox
AmplabJenkins commented on pull request #32493: URL: https://github.com/apache/spark/pull/32493#issuecomment-836752924 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42857/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32485: [SPARK-35357][GRAPHX] Allow to turn off the normalization applied by static PageRank utilities

2021-05-10 Thread GitBox
AmplabJenkins commented on pull request #32485: URL: https://github.com/apache/spark/pull/32485#issuecomment-836752923 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138334/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32492: [SPARK-35088][SQL][FOLLOWUP] Improve the error message for Sequence expression

2021-05-10 Thread GitBox
AmplabJenkins commented on pull request #32492: URL: https://github.com/apache/spark/pull/32492#issuecomment-836752920 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138331/ -- This

[GitHub] [spark] SparkQA commented on pull request #32493: [SPARK-34246][FOLLOWUP] Change the definition of `findTightestCommonType` for backward compatibility

2021-05-10 Thread GitBox
SparkQA commented on pull request #32493: URL: https://github.com/apache/spark/pull/32493#issuecomment-836750549 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] Ngone51 commented on pull request #32287: [SPARK-27991][CORE] Defer the fetch request on Netty OOM

2021-05-10 Thread GitBox
Ngone51 commented on pull request #32287: URL: https://github.com/apache/spark/pull/32287#issuecomment-836736375 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AngersZhuuuu commented on pull request #30057: [SPARK-32838][SQL]Check DataSource insert command path with actual path

2021-05-10 Thread GitBox
AngersZh commented on pull request #30057: URL: https://github.com/apache/spark/pull/30057#issuecomment-836726551 Gentle ping -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] srowen commented on pull request #32415: [SPARK-35295][ML] Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0

2021-05-10 Thread GitBox
srowen commented on pull request #32415: URL: https://github.com/apache/spark/pull/32415#issuecomment-836724744 Just catching up on the state here - so we need to put back the netlib-lgpl profile? anything else pending? -- This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #32492: [SPARK-35088][SQL][FOLLOWUP] Improve the error message for Sequence expression

2021-05-10 Thread GitBox
SparkQA removed a comment on pull request #32492: URL: https://github.com/apache/spark/pull/32492#issuecomment-836458314 **[Test build #138331 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138331/testReport)** for PR 32492 at commit

[GitHub] [spark] SparkQA commented on pull request #32492: [SPARK-35088][SQL][FOLLOWUP] Improve the error message for Sequence expression

2021-05-10 Thread GitBox
SparkQA commented on pull request #32492: URL: https://github.com/apache/spark/pull/32492#issuecomment-836722708 **[Test build #138331 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138331/testReport)** for PR 32492 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #32485: [SPARK-35357][GRAPHX] Allow to turn off the normalization applied by static PageRank utilities

2021-05-10 Thread GitBox
SparkQA removed a comment on pull request #32485: URL: https://github.com/apache/spark/pull/32485#issuecomment-836685637 **[Test build #138334 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138334/testReport)** for PR 32485 at commit

[GitHub] [spark] SparkQA commented on pull request #32485: [SPARK-35357][GRAPHX] Allow to turn off the normalization applied by static PageRank utilities

2021-05-10 Thread GitBox
SparkQA commented on pull request #32485: URL: https://github.com/apache/spark/pull/32485#issuecomment-836705741 **[Test build #138334 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138334/testReport)** for PR 32485 at commit

[GitHub] [spark] SparkQA commented on pull request #32493: [SPARK-34246][FOLLOWUP] Change the definition of `findTightestCommonType` for backward compatibility

2021-05-10 Thread GitBox
SparkQA commented on pull request #32493: URL: https://github.com/apache/spark/pull/32493#issuecomment-836697327 **[Test build #138335 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138335/testReport)** for PR 32493 at commit

[GitHub] [spark] gengliangwang opened a new pull request #32493: [SPARK-34246][FOLLOWUP] Change the definition of `findTightestCommonType` for backward compatibility

2021-05-10 Thread GitBox
gengliangwang opened a new pull request #32493: URL: https://github.com/apache/spark/pull/32493 ### What changes were proposed in this pull request? Change the definition of `findTightestCommonType` from ``` def findTightestCommonType(t1: DataType, t2: DataType):

[GitHub] [spark] SparkQA commented on pull request #32485: [SPARK-35357][GRAPHX] Allow to turn off the normalization applied by static PageRank utilities

2021-05-10 Thread GitBox
SparkQA commented on pull request #32485: URL: https://github.com/apache/spark/pull/32485#issuecomment-836691512 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42856/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #32485: [SPARK-35357][GRAPHX] Allow to turn off the normalization applied by static PageRank utilities

2021-05-10 Thread GitBox
AmplabJenkins commented on pull request #32485: URL: https://github.com/apache/spark/pull/32485#issuecomment-836691567 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42856/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32485: [SPARK-35357][GRAPHX] Allow to turn off the normalization applied by static PageRank utilities

2021-05-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32485: URL: https://github.com/apache/spark/pull/32485#issuecomment-836691567 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42856/

[GitHub] [spark] SparkQA commented on pull request #32485: [SPARK-35357][GRAPHX] Allow to turn off the normalization applied by static PageRank utilities

2021-05-10 Thread GitBox
SparkQA commented on pull request #32485: URL: https://github.com/apache/spark/pull/32485#issuecomment-836686820 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42856/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32485: [SPARK-35357][GRAPHX] Allow to turn off the normalization applied by static PageRank utilities

2021-05-10 Thread GitBox
SparkQA commented on pull request #32485: URL: https://github.com/apache/spark/pull/32485#issuecomment-836685637 **[Test build #138334 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138334/testReport)** for PR 32485 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #32287: [SPARK-27991][CORE] Defer the fetch request on Netty OOM

2021-05-10 Thread GitBox
AmplabJenkins commented on pull request #32287: URL: https://github.com/apache/spark/pull/32287#issuecomment-836684411 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138332/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32287: [SPARK-27991][CORE] Defer the fetch request on Netty OOM

2021-05-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32287: URL: https://github.com/apache/spark/pull/32287#issuecomment-836684411 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138332/

[GitHub] [spark] AmplabJenkins commented on pull request #32442: [SPARK-35283][SQL] Support query some DDL with CTES

2021-05-10 Thread GitBox
AmplabJenkins commented on pull request #32442: URL: https://github.com/apache/spark/pull/32442#issuecomment-836684406 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138333/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32442: [SPARK-35283][SQL] Support query some DDL with CTES

2021-05-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32442: URL: https://github.com/apache/spark/pull/32442#issuecomment-836684406 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138333/

[GitHub] [spark] AmplabJenkins commented on pull request #32491: avoid using deprecated `buildForBatch` and `buildForStreaming`

2021-05-10 Thread GitBox
AmplabJenkins commented on pull request #32491: URL: https://github.com/apache/spark/pull/32491#issuecomment-836684408 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138329/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32491: avoid using deprecated `buildForBatch` and `buildForStreaming`

2021-05-10 Thread GitBox
AmplabJenkins removed a comment on pull request #32491: URL: https://github.com/apache/spark/pull/32491#issuecomment-836684408 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138329/

<    1   2   3   4   5   >