[GitHub] [spark] SparkQA commented on pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

SparkQA commented on pull request #32527: URL: https://github.com/apache/spark/pull/32527#issuecomment-840236001 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43000/ -- This is an automated message from the Apache

[GitHub] [spark] Kimahriman commented on a change in pull request #32448: [SPARK-35290][SQL] Use StructType merging for unionByName with null filling

Kimahriman commented on a change in pull request #32448: URL: https://github.com/apache/spark/pull/32448#discussion_r631516355 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSetOperationsSuite.scala ## @@ -752,8 +786,50 @@ class DataFrameSetOperationsSuite

[GitHub] [spark] Kimahriman commented on a change in pull request #32448: [SPARK-35290][SQL] Use StructType merging for unionByName with null filling

Kimahriman commented on a change in pull request #32448: URL: https://github.com/apache/spark/pull/32448#discussion_r631516151 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/StructType.scala ## @@ -483,8 +483,8 @@ case class StructType(fields: Array[Struc

[GitHub] [spark] Kimahriman commented on a change in pull request #32448: [SPARK-35290][SQL] Use StructType merging for unionByName with null filling

Kimahriman commented on a change in pull request #32448: URL: https://github.com/apache/spark/pull/32448#discussion_r631516256 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveUnion.scala ## @@ -21,136 +21,55 @@ import scala.collection.mu

[GitHub] [spark] Kimahriman commented on a change in pull request #32448: [SPARK-35290][SQL] Use StructType merging for unionByName with null filling

Kimahriman commented on a change in pull request #32448: URL: https://github.com/apache/spark/pull/32448#discussion_r631516011 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveUnion.scala ## @@ -181,7 +100,12 @@ object ResolveUnion extend

[GitHub] [spark] Kimahriman commented on a change in pull request #32448: [SPARK-35290][SQL] Use StructType merging for unionByName with null filling

Kimahriman commented on a change in pull request #32448: URL: https://github.com/apache/spark/pull/32448#discussion_r631515982 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/StructType.scala ## @@ -555,32 +555,31 @@ object StructType extends AbstractDataT

[GitHub] [spark] ulysses-you commented on pull request #32468: [SPARK-35335][SQL] Improve CoalesceShufflePartitions to avoid generating small files

ulysses-you commented on pull request #32468: URL: https://github.com/apache/spark/pull/32468#issuecomment-840233602 Some random thoughts. We considered about supporting stage level config completly that means for every query stage we can use it's own config. Some options: * enhan

[GitHub] [spark] maropu commented on a change in pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

maropu commented on a change in pull request #32498: URL: https://github.com/apache/spark/pull/32498#discussion_r631512825 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala ## @@ -789,6 +793,36 @@ case class Range

[GitHub] [spark] viirya commented on a change in pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

viirya commented on a change in pull request #32527: URL: https://github.com/apache/spark/pull/32527#discussion_r631513984 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala ## @@ -127,13 +128,18 @@ trait InvokeLike extend

[GitHub] [spark] viirya commented on a change in pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

viirya commented on a change in pull request #32527: URL: https://github.com/apache/spark/pull/32527#discussion_r631513984 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala ## @@ -127,13 +128,18 @@ trait InvokeLike extend

[GitHub] [spark] SparkQA removed a comment on pull request #32496: [SPARK-35207][SQL] Normalize hash function behavior with negative zero

SparkQA removed a comment on pull request #32496: URL: https://github.com/apache/spark/pull/32496#issuecomment-840115141 **[Test build #138474 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138474/testReport)** for PR 32496 at commit [`641629d`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32496: [SPARK-35207][SQL] Normalize hash function behavior with negative zero

SparkQA commented on pull request #32496: URL: https://github.com/apache/spark/pull/32496#issuecomment-840230803 **[Test build #138474 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138474/testReport)** for PR 32496 at commit [`641629d`](https://github.co

[GitHub] [spark] HyukjinKwon commented on pull request #32524: [SPARK-35388][INFRA] Allow the PR source branch to include slashes.

HyukjinKwon commented on pull request #32524: URL: https://github.com/apache/spark/pull/32524#issuecomment-840230532 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [spark] HyukjinKwon closed pull request #32524: [SPARK-35388][INFRA] Allow the PR source branch to include slashes.

HyukjinKwon closed pull request #32524: URL: https://github.com/apache/spark/pull/32524 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, pl

[GitHub] [spark] ulysses-you commented on pull request #32468: [SPARK-35335][SQL] Improve CoalesceShufflePartitions to avoid generating small files

ulysses-you commented on pull request #32468: URL: https://github.com/apache/spark/pull/32468#issuecomment-840227088 Thank you for introducing this idea. But that's not only about `spark.sql.adaptive.coalescePartitions.minPartitionNum ` but also other adaptive configs, such as `spark.sql.a

[GitHub] [spark] YuzhouSun edited a comment on pull request #32207: [SPARK-35106] Avoid failing rename in HadoopMapReduceCommitProtocol with dynamic partition overwrite

YuzhouSun edited a comment on pull request #32207: URL: https://github.com/apache/spark/pull/32207#issuecomment-840224412 Hello, about “we should only run Block 2 in the dynamicPartitionOverwrite == false case”: the Block 2 is actually meant for custom partition paths (i.e. absolute partit

[GitHub] [spark] YuzhouSun commented on pull request #32207: [SPARK-35106] Avoid failing rename in HadoopMapReduceCommitProtocol with dynamic partition overwrite

YuzhouSun commented on pull request #32207: URL: https://github.com/apache/spark/pull/32207#issuecomment-840224412 Hello, about “we should only run Block 2 in the dynamicPartitionOverwrite == false case”: the Block 2 is actually meant for custom partition paths (i.e. absolute partitions),

[GitHub] [spark] YuzhouSun opened a new pull request #32529: [SPARK-35106][Core][SQL] Avoid failing rename caused by destination directory not exist

YuzhouSun opened a new pull request #32529: URL: https://github.com/apache/spark/pull/32529 ### What changes were proposed in this pull request? 1. In HadoopMapReduceCommitProtocol, create parent directory before renaming custom partition path staging files 2. In InMemoryC

[GitHub] [spark] AmplabJenkins commented on pull request #32529: [SPARK-35106][Core][SQL] Avoid failing rename caused by destination directory not exist

AmplabJenkins commented on pull request #32529: URL: https://github.com/apache/spark/pull/32529#issuecomment-840222871 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] maropu commented on a change in pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

maropu commented on a change in pull request #32494: URL: https://github.com/apache/spark/pull/32494#discussion_r631505644 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/UnionEstimation.scala ## @@ -111,6 +111,44 @@ object

[GitHub] [spark] SparkQA commented on pull request #32448: [SPARK-35290][SQL] Use StructType merging for unionByName with null filling

SparkQA commented on pull request #32448: URL: https://github.com/apache/spark/pull/32448#issuecomment-840218983 **[Test build #138481 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138481/testReport)** for PR 32448 at commit [`93b47d3`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

SparkQA commented on pull request #32527: URL: https://github.com/apache/spark/pull/32527#issuecomment-840217408 **[Test build #138480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138480/testReport)** for PR 32527 at commit [`2831f9c`](https://github.com

[GitHub] [spark] sunchao commented on a change in pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

sunchao commented on a change in pull request #32527: URL: https://github.com/apache/spark/pull/32527#discussion_r631501337 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala ## @@ -127,13 +128,19 @@ trait InvokeLike exten

[GitHub] [spark] maropu commented on a change in pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

maropu commented on a change in pull request #32527: URL: https://github.com/apache/spark/pull/32527#discussion_r631499051 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala ## @@ -127,13 +128,19 @@ trait InvokeLike extend

[GitHub] [spark] sunchao commented on a change in pull request #32488: [SPARK-35316][SQL] UnwrapCastInBinaryComparison support In/InSet predicate

sunchao commented on a change in pull request #32488: URL: https://github.com/apache/spark/pull/32488#discussion_r631498596 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/UnwrapCastInBinaryComparisonSuite.scala ## @@ -233,6 +233,15 @@ class U

[GitHub] [spark] SparkQA commented on pull request #32528: [SPARK-35350][SQL] Add code-gen for left semi sort merge join

SparkQA commented on pull request #32528: URL: https://github.com/apache/spark/pull/32528#issuecomment-840211154 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries

[GitHub] [spark] SparkQA commented on pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

SparkQA commented on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-840210720 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries

[GitHub] [spark] SparkQA removed a comment on pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

SparkQA removed a comment on pull request #32498: URL: https://github.com/apache/spark/pull/32498#issuecomment-840083265 **[Test build #138473 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138473/testReport)** for PR 32498 at commit [`fcf2705`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

SparkQA commented on pull request #32498: URL: https://github.com/apache/spark/pull/32498#issuecomment-840209168 **[Test build #138473 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138473/testReport)** for PR 32498 at commit [`fcf2705`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

SparkQA commented on pull request #32498: URL: https://github.com/apache/spark/pull/32498#issuecomment-840207182 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42998/ -- This

[GitHub] [spark] sunchao commented on pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

sunchao commented on pull request #32527: URL: https://github.com/apache/spark/pull/32527#issuecomment-840204966 cc @HyukjinKwon @cloud-fan @dongjoon-hyun @viirya @maropu -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] maropu edited a comment on pull request #32455: [SPARK-35253][SQL][BUILD] Bump up the janino version to v3.1.4

maropu edited a comment on pull request #32455: URL: https://github.com/apache/spark/pull/32455#issuecomment-840203738 Thank you, all the reviewers~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] maropu commented on pull request #32455: [SPARK-35253][SQL][BUILD] Bump up the janino version to v3.1.4

maropu commented on pull request #32455: URL: https://github.com/apache/spark/pull/32455#issuecomment-840203738 Thank you for all the reviewers~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [spark] maropu commented on a change in pull request #32476: [SPARK-35349][SQL] Add code-gen for left/right outer sort merge join

maropu commented on a change in pull request #32476: URL: https://github.com/apache/spark/pull/32476#discussion_r631492986 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala ## @@ -418,115 +443,140 @@ case class SortMergeJoinExec

[GitHub] [spark] maropu commented on pull request #32520: [SPARK-35385][SQL][TESTS] Skip duplicate queries in the TPCDS-related tests

maropu commented on pull request #32520: URL: https://github.com/apache/spark/pull/32520#issuecomment-840200018 The last commit is only for the comment update, so I merged to master. Thank you, @cloud-fan @dongjoon-hyun -- This is an automated message from the Apache Git Service. To res

[GitHub] [spark] maropu closed pull request #32520: [SPARK-35385][SQL][TESTS] Skip duplicate queries in the TPCDS-related tests

maropu closed pull request #32520: URL: https://github.com/apache/spark/pull/32520 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [spark] SparkQA commented on pull request #32520: [SPARK-35385][SQL][TESTS] Skip duplicate queries in the TPCDS-related tests

SparkQA commented on pull request #32520: URL: https://github.com/apache/spark/pull/32520#issuecomment-840197479 **[Test build #138479 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138479/testReport)** for PR 32520 at commit [`299abb5`](https://github.com

[GitHub] [spark] maropu commented on a change in pull request #32520: [SPARK-35385][SQL][TESTS] Skip duplicate queries in the TPCDS-related tests

maropu commented on a change in pull request #32520: URL: https://github.com/apache/spark/pull/32520#discussion_r631489998 ## File path: sql/core/src/test/scala/org/apache/spark/sql/TPCDSBase.scala ## @@ -36,6 +36,12 @@ trait TPCDSBase extends SharedSparkSession with TPCDSSchem

[GitHub] [spark] maropu commented on a change in pull request #32520: [SPARK-35385][SQL][TESTS] Skip duplicate queries in the TPCDS-related tests

maropu commented on a change in pull request #32520: URL: https://github.com/apache/spark/pull/32520#discussion_r631489676 ## File path: sql/core/src/test/scala/org/apache/spark/sql/TPCDSBase.scala ## @@ -36,6 +36,12 @@ trait TPCDSBase extends SharedSparkSession with TPCDSSchem

[GitHub] [spark] SparkQA commented on pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

SparkQA commented on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-840190295 **[Test build #138478 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138478/testReport)** for PR 32494 at commit [`c929124`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

SparkQA commented on pull request #32498: URL: https://github.com/apache/spark/pull/32498#issuecomment-840190243 **[Test build #138477 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138477/testReport)** for PR 32498 at commit [`0bb49b3`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32528: [SPARK-35350][SQL] Add code-gen for left semi sort merge join

SparkQA commented on pull request #32528: URL: https://github.com/apache/spark/pull/32528#issuecomment-840190201 **[Test build #138476 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138476/testReport)** for PR 32528 at commit [`8eb55c3`](https://github.com

[GitHub] [spark] maropu commented on pull request #32523: [SPARK-35382][PYTHON] Fix lambda variable name issues in nested DataFrame functions in Python APIs.

maropu commented on pull request #32523: URL: https://github.com/apache/spark/pull/32523#issuecomment-840189393 The fix itself looks fine. Thank you, @ueshin ~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

AmplabJenkins removed a comment on pull request #32527: URL: https://github.com/apache/spark/pull/32527#issuecomment-840187929 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42996/

[GitHub] [spark] AmplabJenkins commented on pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

AmplabJenkins commented on pull request #32527: URL: https://github.com/apache/spark/pull/32527#issuecomment-840187929 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42996/ -- T

[GitHub] [spark] maropu commented on a change in pull request #32496: [SPARK-35207][SQL] Normalize hash function behavior with negative zero

maropu commented on a change in pull request #32496: URL: https://github.com/apache/spark/pull/32496#discussion_r631478745 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/hash.scala ## @@ -369,11 +369,25 @@ abstract class HashExpression[E] e

[GitHub] [spark] SparkQA commented on pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

SparkQA commented on pull request #32527: URL: https://github.com/apache/spark/pull/32527#issuecomment-840182786 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries

[GitHub] [spark] c21 commented on pull request #32528: [SPARK-35350][SQL] Add code-gen for left semi sort merge join

c21 commented on pull request #32528: URL: https://github.com/apache/spark/pull/32528#issuecomment-840175988 cc @cloud-fan and @maropu could you help take a look when you have time? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] c21 opened a new pull request #32528: [SPARK-35350][SQL] Add code-gen for left semi sort merge join

c21 opened a new pull request #32528: URL: https://github.com/apache/spark/pull/32528 ### What changes were proposed in this pull request? As title. This PR is to add code-gen support for LEFT SEMI sort merge join. The main change is to add `semiJoin` code path in `SortMerge

[GitHub] [spark] maropu commented on pull request #32496: [SPARK-35207][SQL] Normalize hash function behavior with negative zero

maropu commented on pull request #32496: URL: https://github.com/apache/spark/pull/32496#issuecomment-840167901 Yea, it seems @ueshin 's working on it https://github.com/apache/spark/pull/32524?t=1620862145319 -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA commented on pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

SparkQA commented on pull request #32527: URL: https://github.com/apache/spark/pull/32527#issuecomment-840161171 **[Test build #138475 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138475/testReport)** for PR 32527 at commit [`9ce2542`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32524: [SPARK-35388][INFRA] Allow the PR source branch to include slashes.

AmplabJenkins removed a comment on pull request #32524: URL: https://github.com/apache/spark/pull/32524#issuecomment-840159449 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138472/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32525: [DO NOT MERGE][SPARK-35388][INFRA] Allow the PR source branch to include slashes.

AmplabJenkins removed a comment on pull request #32525: URL: https://github.com/apache/spark/pull/32525#issuecomment-840159448 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138471/ -

[GitHub] [spark] AmplabJenkins commented on pull request #32524: [SPARK-35388][INFRA] Allow the PR source branch to include slashes.

AmplabJenkins commented on pull request #32524: URL: https://github.com/apache/spark/pull/32524#issuecomment-840159449 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138472/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32525: [DO NOT MERGE][SPARK-35388][INFRA] Allow the PR source branch to include slashes.

AmplabJenkins commented on pull request #32525: URL: https://github.com/apache/spark/pull/32525#issuecomment-840159448 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138471/ -- This

[GitHub] [spark] sunchao opened a new pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

sunchao opened a new pull request #32527: URL: https://github.com/apache/spark/pull/32527 ### What changes were proposed in this pull request? Change `map` in `InvokeLike.invoke` to a while loop to improve performance, following Spark [style guide](https://github.com/dat

[GitHub] [spark] ueshin commented on pull request #32524: [SPARK-35388][INFRA] Allow the PR source branch to include slashes.

ueshin commented on pull request #32524: URL: https://github.com/apache/spark/pull/32524#issuecomment-840153097 cc @HyukjinKwon @dongjoon-hyun -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] SparkQA removed a comment on pull request #32524: [SPARK-35388][INFRA] Allow the PR source branch to include slashes.

SparkQA removed a comment on pull request #32524: URL: https://github.com/apache/spark/pull/32524#issuecomment-840074691 **[Test build #138472 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138472/testReport)** for PR 32524 at commit [`3a8c5af`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32524: [SPARK-35388][INFRA] Allow the PR source branch to include slashes.

SparkQA commented on pull request #32524: URL: https://github.com/apache/spark/pull/32524#issuecomment-840143253 **[Test build #138472 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138472/testReport)** for PR 32524 at commit [`3a8c5af`](https://github.co

[GitHub] [spark] planga82 commented on pull request #32496: [SPARK-35207][SQL] Normalize hash function behavior with negative zero

planga82 commented on pull request #32496: URL: https://github.com/apache/spark/pull/32496#issuecomment-840143154 Hi @maropu , Thank for your comments. I have looked at the weird problems of the GA tests and I think they were because of the name of the branch. This name "feature/spark35207

[GitHub] [spark] SparkQA removed a comment on pull request #32525: [DO NOT MERGE][SPARK-35388][INFRA] Allow the PR source branch to include slashes.

SparkQA removed a comment on pull request #32525: URL: https://github.com/apache/spark/pull/32525#issuecomment-840074647 **[Test build #138471 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138471/testReport)** for PR 32525 at commit [`732728f`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32525: [DO NOT MERGE][SPARK-35388][INFRA] Allow the PR source branch to include slashes.

SparkQA commented on pull request #32525: URL: https://github.com/apache/spark/pull/32525#issuecomment-840141488 **[Test build #138471 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138471/testReport)** for PR 32525 at commit [`732728f`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32496: [SPARK-35207][SQL] Normalize hash function behavior with negative zero

AmplabJenkins removed a comment on pull request #32496: URL: https://github.com/apache/spark/pull/32496#issuecomment-840135088 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42995/

[GitHub] [spark] AmplabJenkins commented on pull request #32496: [SPARK-35207][SQL] Normalize hash function behavior with negative zero

AmplabJenkins commented on pull request #32496: URL: https://github.com/apache/spark/pull/32496#issuecomment-840135088 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42995/ -- T

[GitHub] [spark] SparkQA commented on pull request #32496: [SPARK-35207][SQL] Normalize hash function behavior with negative zero

SparkQA commented on pull request #32496: URL: https://github.com/apache/spark/pull/32496#issuecomment-840135063 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

AmplabJenkins removed a comment on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-840132988 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138467/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

AmplabJenkins removed a comment on pull request #32498: URL: https://github.com/apache/spark/pull/32498#issuecomment-840132987 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32522: [SPARK-35361][SQL][FOLLOWUP] Switch to use while loop

AmplabJenkins removed a comment on pull request #32522: URL: https://github.com/apache/spark/pull/32522#issuecomment-840132986 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138469/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32292: [SPARK-35162][SQL] New SQL functions: TRY_ADD/TRY_DIVIDE

AmplabJenkins removed a comment on pull request #32292: URL: https://github.com/apache/spark/pull/32292#issuecomment-840132989 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138468/ -

[GitHub] [spark] AmplabJenkins commented on pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

AmplabJenkins commented on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-840132988 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138467/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32522: [SPARK-35361][SQL][FOLLOWUP] Switch to use while loop

AmplabJenkins commented on pull request #32522: URL: https://github.com/apache/spark/pull/32522#issuecomment-840132986 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138469/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32292: [SPARK-35162][SQL] New SQL functions: TRY_ADD/TRY_DIVIDE

AmplabJenkins commented on pull request #32292: URL: https://github.com/apache/spark/pull/32292#issuecomment-840132989 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138468/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

AmplabJenkins commented on pull request #32498: URL: https://github.com/apache/spark/pull/32498#issuecomment-840132990 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For q

[GitHub] [spark] SparkQA commented on pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

SparkQA commented on pull request #32498: URL: https://github.com/apache/spark/pull/32498#issuecomment-840129440 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries

[GitHub] [spark] xkrogen commented on a change in pull request #32388: [SPARK-35258][SHUFFLE][YARN] Add new metrics to ExternalShuffleService for better monitoring

xkrogen commented on a change in pull request #32388: URL: https://github.com/apache/spark/pull/32388#discussion_r631430645 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalBlockHandler.java ## @@ -98,10 +99,10 @@ public ExternalBlock

[GitHub] [spark] SparkQA removed a comment on pull request #32292: [SPARK-35162][SQL] New SQL functions: TRY_ADD/TRY_DIVIDE

SparkQA removed a comment on pull request #32292: URL: https://github.com/apache/spark/pull/32292#issuecomment-839961495 **[Test build #138468 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138468/testReport)** for PR 32292 at commit [`60b4514`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32292: [SPARK-35162][SQL] New SQL functions: TRY_ADD/TRY_DIVIDE

SparkQA commented on pull request #32292: URL: https://github.com/apache/spark/pull/32292#issuecomment-840122984 **[Test build #138468 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138468/testReport)** for PR 32292 at commit [`60b4514`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #32522: [SPARK-35361][SQL][FOLLOWUP] Switch to use while loop

SparkQA removed a comment on pull request #32522: URL: https://github.com/apache/spark/pull/32522#issuecomment-839967576 **[Test build #138469 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138469/testReport)** for PR 32522 at commit [`98153ab`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32522: [SPARK-35361][SQL][FOLLOWUP] Switch to use while loop

SparkQA commented on pull request #32522: URL: https://github.com/apache/spark/pull/32522#issuecomment-840120648 **[Test build #138469 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138469/testReport)** for PR 32522 at commit [`98153ab`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

SparkQA removed a comment on pull request #32498: URL: https://github.com/apache/spark/pull/32498#issuecomment-839958685 **[Test build #138466 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138466/testReport)** for PR 32498 at commit [`27ea38a`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

SparkQA commented on pull request #32498: URL: https://github.com/apache/spark/pull/32498#issuecomment-840117966 **[Test build #138466 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138466/testReport)** for PR 32498 at commit [`27ea38a`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

SparkQA removed a comment on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-839958738 **[Test build #138467 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138467/testReport)** for PR 32494 at commit [`b338d89`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

SparkQA commented on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-840116072 **[Test build #138467 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138467/testReport)** for PR 32494 at commit [`b338d89`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #32496: [SPARK-35207][SQL] Normalize hash function behavior with negative zero

SparkQA commented on pull request #32496: URL: https://github.com/apache/spark/pull/32496#issuecomment-840115141 **[Test build #138474 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138474/testReport)** for PR 32496 at commit [`641629d`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32524: [SPARK-35388][INFRA] Allow the PR source branch to include slashes.

AmplabJenkins removed a comment on pull request #32524: URL: https://github.com/apache/spark/pull/32524#issuecomment-840107876 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42993/

[GitHub] [spark] AmplabJenkins commented on pull request #32524: [SPARK-35388][INFRA] Allow the PR source branch to include slashes.

AmplabJenkins commented on pull request #32524: URL: https://github.com/apache/spark/pull/32524#issuecomment-840107876 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42993/ -- T

[GitHub] [spark] SparkQA commented on pull request #32524: [SPARK-35388][INFRA] Allow the PR source branch to include slashes.

SparkQA commented on pull request #32524: URL: https://github.com/apache/spark/pull/32524#issuecomment-840107840 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42993/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

AmplabJenkins removed a comment on pull request #32498: URL: https://github.com/apache/spark/pull/32498#issuecomment-840106070 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138462/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32525: [DO NOT MERGE][SPARK-35388][INFRA] Allow the PR source branch to include slashes.

AmplabJenkins removed a comment on pull request #32525: URL: https://github.com/apache/spark/pull/32525#issuecomment-840106068 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42992/

[GitHub] [spark] AmplabJenkins commented on pull request #32526: [SPARK-35391] Fix memory leak in ExecutorAllocationListener

AmplabJenkins commented on pull request #32526: URL: https://github.com/apache/spark/pull/32526#issuecomment-840106353 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] AmplabJenkins commented on pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

AmplabJenkins commented on pull request #32498: URL: https://github.com/apache/spark/pull/32498#issuecomment-840106070 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138462/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32525: [DO NOT MERGE][SPARK-35388][INFRA] Allow the PR source branch to include slashes.

AmplabJenkins commented on pull request #32525: URL: https://github.com/apache/spark/pull/32525#issuecomment-840106068 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42992/ -- T

[GitHub] [spark] SparkQA commented on pull request #32524: [SPARK-35388][INFRA] Allow the PR source branch to include slashes.

SparkQA commented on pull request #32524: URL: https://github.com/apache/spark/pull/32524#issuecomment-840104298 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42993/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32525: [DO NOT MERGE][SPARK-35388][INFRA] Allow the PR source branch to include slashes.

SparkQA commented on pull request #32525: URL: https://github.com/apache/spark/pull/32525#issuecomment-840099522 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries

[GitHub] [spark] VasilyKolpakov opened a new pull request #32526: [SPARK-35391] Fix memory leak in ExecutorAllocationListener

VasilyKolpakov opened a new pull request #32526: URL: https://github.com/apache/spark/pull/32526 ### What changes were proposed in this pull request? This PR fixes a memory leak in ExecutorAllocationListener. ### Why are the changes needed? Dynamic allocation stops working under

[GitHub] [spark] LucaCanali commented on pull request #32500: [SPARK-35369][DOC] Document ExecutorAllocationManager metrics

LucaCanali commented on pull request #32500: URL: https://github.com/apache/spark/pull/32500#issuecomment-840085856 Thanks @dongjoon-hyun -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[GitHub] [spark] shahidki31 commented on a change in pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

shahidki31 commented on a change in pull request #32498: URL: https://github.com/apache/spark/pull/32498#discussion_r631385071 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala ## @@ -759,6 +759,10 @@ case class R

[GitHub] [spark] shahidki31 commented on a change in pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

shahidki31 commented on a change in pull request #32498: URL: https://github.com/apache/spark/pull/32498#discussion_r631385071 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala ## @@ -759,6 +759,10 @@ case class R

[GitHub] [spark] SparkQA removed a comment on pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

SparkQA removed a comment on pull request #32498: URL: https://github.com/apache/spark/pull/32498#issuecomment-839918693 **[Test build #138462 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138462/testReport)** for PR 32498 at commit [`d314ba9`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

SparkQA commented on pull request #32498: URL: https://github.com/apache/spark/pull/32498#issuecomment-840084693 **[Test build #138462 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138462/testReport)** for PR 32498 at commit [`d314ba9`](https://github.co

<    1   2   3   4   5   6   7   8   >