[GitHub] [spark] SparkQA removed a comment on pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

SparkQA removed a comment on pull request #32527: URL: https://github.com/apache/spark/pull/32527#issuecomment-840161171 **[Test build #138475 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138475/testReport)** for PR 32527 at commit

[GitHub] [spark] SparkQA commented on pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

SparkQA commented on pull request #32527: URL: https://github.com/apache/spark/pull/32527#issuecomment-840272580 **[Test build #138475 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138475/testReport)** for PR 32527 at commit

[GitHub] [spark] cfmcgrady commented on a change in pull request #32488: [SPARK-35316][SQL] UnwrapCastInBinaryComparison support In/InSet predicate

cfmcgrady commented on a change in pull request #32488: URL: https://github.com/apache/spark/pull/32488#discussion_r631542180 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/UnwrapCastInBinaryComparisonSuite.scala ## @@ -233,6 +233,15 @@

[GitHub] [spark] SparkQA removed a comment on pull request #32528: [SPARK-35350][SQL] Add code-gen for left semi sort merge join

SparkQA removed a comment on pull request #32528: URL: https://github.com/apache/spark/pull/32528#issuecomment-840190201 **[Test build #138476 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138476/testReport)** for PR 32528 at commit

[GitHub] [spark] SparkQA commented on pull request #32528: [SPARK-35350][SQL] Add code-gen for left semi sort merge join

SparkQA commented on pull request #32528: URL: https://github.com/apache/spark/pull/32528#issuecomment-840271318 **[Test build #138476 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138476/testReport)** for PR 32528 at commit

[GitHub] [spark] maropu commented on pull request #31967: [SPARK-34819][SQL] MapType supports orderable semantics

maropu commented on pull request #31967: URL: https://github.com/apache/spark/pull/31967#issuecomment-840270023 @WangGuangxin If you cannot keep working on it, is it okay that I take this over? -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] sunchao commented on a change in pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

sunchao commented on a change in pull request #32527: URL: https://github.com/apache/spark/pull/32527#discussion_r631540561 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala ## @@ -127,13 +128,18 @@ trait InvokeLike

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

dongjoon-hyun commented on a change in pull request #32527: URL: https://github.com/apache/spark/pull/32527#discussion_r631540103 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala ## @@ -127,13 +128,18 @@ trait

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32515: [SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

AmplabJenkins removed a comment on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-840268531 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43004/

[GitHub] [spark] SparkQA commented on pull request #32515: [SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

SparkQA commented on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-840268507 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43004/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32515: [SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

AmplabJenkins commented on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-840268531 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43004/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32515: [SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

AmplabJenkins removed a comment on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-840267692 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138484/

[GitHub] [spark] SparkQA removed a comment on pull request #32515: [SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

SparkQA removed a comment on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-840264686 **[Test build #138484 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138484/testReport)** for PR 32515 at commit

[GitHub] [spark] SparkQA commented on pull request #32515: [SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

SparkQA commented on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-840267671 **[Test build #138484 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138484/testReport)** for PR 32515 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #32515: [SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

AmplabJenkins commented on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-840267692 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138484/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32523: [SPARK-35382][PYTHON] Fix lambda variable name issues in nested DataFrame functions in Python APIs.

AmplabJenkins removed a comment on pull request #32523: URL: https://github.com/apache/spark/pull/32523#issuecomment-840264870 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43003/

[GitHub] [spark] SparkQA commented on pull request #32199: [SPARK-35100][ML] Refactor AFT - support virtual centering

SparkQA commented on pull request #32199: URL: https://github.com/apache/spark/pull/32199#issuecomment-840264912 **[Test build #138487 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138487/testReport)** for PR 32199 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #32523: [SPARK-35382][PYTHON] Fix lambda variable name issues in nested DataFrame functions in Python APIs.

AmplabJenkins commented on pull request #32523: URL: https://github.com/apache/spark/pull/32523#issuecomment-840264870 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43003/ --

[GitHub] [spark] SparkQA commented on pull request #32523: [SPARK-35382][PYTHON] Fix lambda variable name issues in nested DataFrame functions in Python APIs.

SparkQA commented on pull request #32523: URL: https://github.com/apache/spark/pull/32523#issuecomment-840264838 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] SparkQA commented on pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

SparkQA commented on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-840264760 **[Test build #138486 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138486/testReport)** for PR 32494 at commit

[GitHub] [spark] SparkQA commented on pull request #32515: [SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

SparkQA commented on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-840264686 **[Test build #138484 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138484/testReport)** for PR 32515 at commit

[GitHub] [spark] SparkQA commented on pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

SparkQA commented on pull request #32498: URL: https://github.com/apache/spark/pull/32498#issuecomment-840264723 **[Test build #138485 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138485/testReport)** for PR 32498 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #32530: [SPARK-35106][Core][SQL] Avoid failing rename caused by destination directory not exist

AmplabJenkins commented on pull request #32530: URL: https://github.com/apache/spark/pull/32530#issuecomment-840264504 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32523: [SPARK-35382][PYTHON] Fix lambda variable name issues in nested DataFrame functions in Python APIs.

AmplabJenkins removed a comment on pull request #32523: URL: https://github.com/apache/spark/pull/32523#issuecomment-840264037 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138483/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32515: [SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

AmplabJenkins removed a comment on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-840264039 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43002/

[GitHub] [spark] AmplabJenkins commented on pull request #32515: [SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

AmplabJenkins commented on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-840264039 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43002/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32523: [SPARK-35382][PYTHON] Fix lambda variable name issues in nested DataFrame functions in Python APIs.

AmplabJenkins commented on pull request #32523: URL: https://github.com/apache/spark/pull/32523#issuecomment-840264037 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138483/ -- This

[GitHub] [spark] shahidki31 commented on a change in pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

shahidki31 commented on a change in pull request #32498: URL: https://github.com/apache/spark/pull/32498#discussion_r631536454 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala ## @@ -789,6 +793,36 @@ case class

[GitHub] [spark] shahidki31 commented on a change in pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

shahidki31 commented on a change in pull request #32498: URL: https://github.com/apache/spark/pull/32498#discussion_r631536413 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala ## @@ -774,13 +774,17 @@ case class

[GitHub] [spark] shahidki31 commented on a change in pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

shahidki31 commented on a change in pull request #32498: URL: https://github.com/apache/spark/pull/32498#discussion_r631536346 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala ## @@ -789,6 +793,36 @@ case class

[GitHub] [spark] SparkQA commented on pull request #32515: [SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

SparkQA commented on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-840261229 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] wangyum commented on pull request #29642: [SPARK-32792][SQL] Improve Parquet In filter pushdown

wangyum commented on pull request #29642: URL: https://github.com/apache/spark/pull/29642#issuecomment-840259632 @dongjoon-hyun Do you have more comments? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] viirya commented on a change in pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

viirya commented on a change in pull request #32527: URL: https://github.com/apache/spark/pull/32527#discussion_r631532550 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala ## @@ -127,13 +128,18 @@ trait InvokeLike

[GitHub] [spark] YuzhouSun closed pull request #32529: [SPARK-35106][Core][SQL] Avoid failing rename caused by destination directory not exist

YuzhouSun closed pull request #32529: URL: https://github.com/apache/spark/pull/32529 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] YuzhouSun commented on pull request #32529: [SPARK-35106][Core][SQL] Avoid failing rename caused by destination directory not exist

YuzhouSun commented on pull request #32529: URL: https://github.com/apache/spark/pull/32529#issuecomment-840257470 Seems branch name should not contain `/`, Creating a new PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] YuzhouSun opened a new pull request #32530: [SPARK-35106][Core][SQL] Avoid failing rename caused by destination directory not exist

YuzhouSun opened a new pull request #32530: URL: https://github.com/apache/spark/pull/32530 ### What changes were proposed in this pull request? 1. In HadoopMapReduceCommitProtocol, create parent directory before renaming custom partition path staging files 2. In

[GitHub] [spark] SparkQA removed a comment on pull request #32523: [SPARK-35382][PYTHON] Fix lambda variable name issues in nested DataFrame functions in Python APIs.

SparkQA removed a comment on pull request #32523: URL: https://github.com/apache/spark/pull/32523#issuecomment-840247014 **[Test build #138483 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138483/testReport)** for PR 32523 at commit

[GitHub] [spark] SparkQA commented on pull request #32523: [SPARK-35382][PYTHON] Fix lambda variable name issues in nested DataFrame functions in Python APIs.

SparkQA commented on pull request #32523: URL: https://github.com/apache/spark/pull/32523#issuecomment-840255768 **[Test build #138483 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138483/testReport)** for PR 32523 at commit

[GitHub] [spark] shahidki31 commented on a change in pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

shahidki31 commented on a change in pull request #32494: URL: https://github.com/apache/spark/pull/32494#discussion_r631530864 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala ## @@ -225,7 +225,7 @@

[GitHub] [spark] beliefer commented on pull request #32464: [SPARK-35062][SQL] Group exception messages in sql/streaming

beliefer commented on pull request #32464: URL: https://github.com/apache/spark/pull/32464#issuecomment-840254774 cc @cloud-fan -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] shahidki31 commented on a change in pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

shahidki31 commented on a change in pull request #32494: URL: https://github.com/apache/spark/pull/32494#discussion_r631528684 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/FilterEstimation.scala ## @@ -225,7 +225,7 @@

[GitHub] [spark] shahidki31 commented on a change in pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

shahidki31 commented on a change in pull request #32494: URL: https://github.com/apache/spark/pull/32494#discussion_r631528270 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/UnionEstimation.scala ## @@ -111,6 +111,44 @@

[GitHub] [spark] shahidki31 commented on a change in pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

shahidki31 commented on a change in pull request #32494: URL: https://github.com/apache/spark/pull/32494#discussion_r631528084 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/UnionEstimation.scala ## @@ -111,6 +111,44 @@

[GitHub] [spark] maropu commented on a change in pull request #32528: [SPARK-35350][SQL] Add code-gen for left semi sort merge join

maropu commented on a change in pull request #32528: URL: https://github.com/apache/spark/pull/32528#discussion_r631525737 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala ## @@ -724,9 +749,32 @@ case class SortMergeJoinExec(

[GitHub] [spark] SparkQA commented on pull request #32523: [SPARK-35382][PYTHON] Fix lambda variable name issues in nested DataFrame functions in Python APIs.

SparkQA commented on pull request #32523: URL: https://github.com/apache/spark/pull/32523#issuecomment-840247014 **[Test build #138483 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138483/testReport)** for PR 32523 at commit

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32515: [SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

HyukjinKwon commented on a change in pull request #32515: URL: https://github.com/apache/spark/pull/32515#discussion_r631524751 ## File path: sql/core/src/main/scala/org/apache/spark/sql/SparkSessionExtensionsProvider.scala ## @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32515: [SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

HyukjinKwon commented on a change in pull request #32515: URL: https://github.com/apache/spark/pull/32515#discussion_r631524651 ## File path: sql/core/src/main/scala/org/apache/spark/sql/SparkSessionExtensionsProvider.scala ## @@ -0,0 +1,83 @@ +/* + * Licensed to the Apache

[GitHub] [spark] sunchao commented on a change in pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

sunchao commented on a change in pull request #32527: URL: https://github.com/apache/spark/pull/32527#discussion_r631523221 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala ## @@ -127,13 +128,18 @@ trait InvokeLike

[GitHub] [spark] ueshin closed pull request #32525: [DO NOT MERGE][SPARK-35388][INFRA] Allow the PR source branch to include slashes.

ueshin closed pull request #32525: URL: https://github.com/apache/spark/pull/32525 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [spark] yaooqinn commented on pull request #32515: [SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

yaooqinn commented on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-840243173 > Ur, @yaooqinn . The R failure looks a little suspicious because it's consistent and relevant. It might be a side-effect in terms of test classes. Could you double-check it?

[GitHub] [spark] maropu commented on a change in pull request #32292: [SPARK-35162][SQL] New SQL functions: TRY_ADD/TRY_DIVIDE

maropu commented on a change in pull request #32292: URL: https://github.com/apache/spark/pull/32292#discussion_r631520466 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/TryEval.scala ## @@ -0,0 +1,107 @@ +/* + * Licensed to the Apache

[GitHub] [spark] SparkQA commented on pull request #32515: [SPARK-35380][SQL] Loading SparkSessionExtensions from ServiceLoader

SparkQA commented on pull request #32515: URL: https://github.com/apache/spark/pull/32515#issuecomment-840242617 **[Test build #138482 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138482/testReport)** for PR 32515 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32496: [SPARK-35207][SQL] Normalize hash function behavior with negative zero

AmplabJenkins removed a comment on pull request #32496: URL: https://github.com/apache/spark/pull/32496#issuecomment-840240208 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138474/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32448: [SPARK-35290][SQL] Use StructType merging for unionByName with null filling

AmplabJenkins removed a comment on pull request #32448: URL: https://github.com/apache/spark/pull/32448#issuecomment-840240209 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43001/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

AmplabJenkins removed a comment on pull request #32527: URL: https://github.com/apache/spark/pull/32527#issuecomment-840240206 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43000/

[GitHub] [spark] AmplabJenkins commented on pull request #32496: [SPARK-35207][SQL] Normalize hash function behavior with negative zero

AmplabJenkins commented on pull request #32496: URL: https://github.com/apache/spark/pull/32496#issuecomment-840240208 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138474/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32448: [SPARK-35290][SQL] Use StructType merging for unionByName with null filling

AmplabJenkins commented on pull request #32448: URL: https://github.com/apache/spark/pull/32448#issuecomment-840240209 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43001/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

AmplabJenkins commented on pull request #32527: URL: https://github.com/apache/spark/pull/32527#issuecomment-840240206 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43000/ --

[GitHub] [spark] SparkQA commented on pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

SparkQA commented on pull request #32527: URL: https://github.com/apache/spark/pull/32527#issuecomment-840238443 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43000/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #32448: [SPARK-35290][SQL] Use StructType merging for unionByName with null filling

SparkQA commented on pull request #32448: URL: https://github.com/apache/spark/pull/32448#issuecomment-840237394 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] SparkQA commented on pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

SparkQA commented on pull request #32527: URL: https://github.com/apache/spark/pull/32527#issuecomment-840236001 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43000/ -- This is an automated message from the Apache

[GitHub] [spark] Kimahriman commented on a change in pull request #32448: [SPARK-35290][SQL] Use StructType merging for unionByName with null filling

Kimahriman commented on a change in pull request #32448: URL: https://github.com/apache/spark/pull/32448#discussion_r631516355 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSetOperationsSuite.scala ## @@ -752,8 +786,50 @@ class

[GitHub] [spark] Kimahriman commented on a change in pull request #32448: [SPARK-35290][SQL] Use StructType merging for unionByName with null filling

Kimahriman commented on a change in pull request #32448: URL: https://github.com/apache/spark/pull/32448#discussion_r631516151 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/StructType.scala ## @@ -483,8 +483,8 @@ case class StructType(fields:

[GitHub] [spark] Kimahriman commented on a change in pull request #32448: [SPARK-35290][SQL] Use StructType merging for unionByName with null filling

Kimahriman commented on a change in pull request #32448: URL: https://github.com/apache/spark/pull/32448#discussion_r631516256 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveUnion.scala ## @@ -21,136 +21,55 @@ import

[GitHub] [spark] Kimahriman commented on a change in pull request #32448: [SPARK-35290][SQL] Use StructType merging for unionByName with null filling

Kimahriman commented on a change in pull request #32448: URL: https://github.com/apache/spark/pull/32448#discussion_r631516011 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveUnion.scala ## @@ -181,7 +100,12 @@ object ResolveUnion

[GitHub] [spark] Kimahriman commented on a change in pull request #32448: [SPARK-35290][SQL] Use StructType merging for unionByName with null filling

Kimahriman commented on a change in pull request #32448: URL: https://github.com/apache/spark/pull/32448#discussion_r631515982 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/StructType.scala ## @@ -555,32 +555,31 @@ object StructType extends

[GitHub] [spark] ulysses-you commented on pull request #32468: [SPARK-35335][SQL] Improve CoalesceShufflePartitions to avoid generating small files

ulysses-you commented on pull request #32468: URL: https://github.com/apache/spark/pull/32468#issuecomment-840233602 Some random thoughts. We considered about supporting stage level config completly that means for every query stage we can use it's own config. Some options: *

[GitHub] [spark] maropu commented on a change in pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

maropu commented on a change in pull request #32498: URL: https://github.com/apache/spark/pull/32498#discussion_r631512825 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala ## @@ -789,6 +793,36 @@ case class

[GitHub] [spark] viirya commented on a change in pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

viirya commented on a change in pull request #32527: URL: https://github.com/apache/spark/pull/32527#discussion_r631513984 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala ## @@ -127,13 +128,18 @@ trait InvokeLike

[GitHub] [spark] viirya commented on a change in pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

viirya commented on a change in pull request #32527: URL: https://github.com/apache/spark/pull/32527#discussion_r631513984 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala ## @@ -127,13 +128,18 @@ trait InvokeLike

[GitHub] [spark] SparkQA removed a comment on pull request #32496: [SPARK-35207][SQL] Normalize hash function behavior with negative zero

SparkQA removed a comment on pull request #32496: URL: https://github.com/apache/spark/pull/32496#issuecomment-840115141 **[Test build #138474 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138474/testReport)** for PR 32496 at commit

[GitHub] [spark] SparkQA commented on pull request #32496: [SPARK-35207][SQL] Normalize hash function behavior with negative zero

SparkQA commented on pull request #32496: URL: https://github.com/apache/spark/pull/32496#issuecomment-840230803 **[Test build #138474 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138474/testReport)** for PR 32496 at commit

[GitHub] [spark] HyukjinKwon commented on pull request #32524: [SPARK-35388][INFRA] Allow the PR source branch to include slashes.

HyukjinKwon commented on pull request #32524: URL: https://github.com/apache/spark/pull/32524#issuecomment-840230532 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] HyukjinKwon closed pull request #32524: [SPARK-35388][INFRA] Allow the PR source branch to include slashes.

HyukjinKwon closed pull request #32524: URL: https://github.com/apache/spark/pull/32524 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] ulysses-you commented on pull request #32468: [SPARK-35335][SQL] Improve CoalesceShufflePartitions to avoid generating small files

ulysses-you commented on pull request #32468: URL: https://github.com/apache/spark/pull/32468#issuecomment-840227088 Thank you for introducing this idea. But that's not only about `spark.sql.adaptive.coalescePartitions.minPartitionNum ` but also other adaptive configs, such as

[GitHub] [spark] YuzhouSun edited a comment on pull request #32207: [SPARK-35106] Avoid failing rename in HadoopMapReduceCommitProtocol with dynamic partition overwrite

YuzhouSun edited a comment on pull request #32207: URL: https://github.com/apache/spark/pull/32207#issuecomment-840224412 Hello, about “we should only run Block 2 in the dynamicPartitionOverwrite == false case”: the Block 2 is actually meant for custom partition paths (i.e. absolute

[GitHub] [spark] YuzhouSun commented on pull request #32207: [SPARK-35106] Avoid failing rename in HadoopMapReduceCommitProtocol with dynamic partition overwrite

YuzhouSun commented on pull request #32207: URL: https://github.com/apache/spark/pull/32207#issuecomment-840224412 Hello, about “we should only run Block 2 in the dynamicPartitionOverwrite == false case”: the Block 2 is actually meant for custom partition paths (i.e. absolute partitions),

[GitHub] [spark] YuzhouSun opened a new pull request #32529: [SPARK-35106][Core][SQL] Avoid failing rename caused by destination directory not exist

YuzhouSun opened a new pull request #32529: URL: https://github.com/apache/spark/pull/32529 ### What changes were proposed in this pull request? 1. In HadoopMapReduceCommitProtocol, create parent directory before renaming custom partition path staging files 2. In

[GitHub] [spark] AmplabJenkins commented on pull request #32529: [SPARK-35106][Core][SQL] Avoid failing rename caused by destination directory not exist

AmplabJenkins commented on pull request #32529: URL: https://github.com/apache/spark/pull/32529#issuecomment-840222871 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] maropu commented on a change in pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

maropu commented on a change in pull request #32494: URL: https://github.com/apache/spark/pull/32494#discussion_r631505644 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/UnionEstimation.scala ## @@ -111,6 +111,44 @@

[GitHub] [spark] SparkQA commented on pull request #32448: [SPARK-35290][SQL] Use StructType merging for unionByName with null filling

SparkQA commented on pull request #32448: URL: https://github.com/apache/spark/pull/32448#issuecomment-840218983 **[Test build #138481 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138481/testReport)** for PR 32448 at commit

[GitHub] [spark] SparkQA commented on pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

SparkQA commented on pull request #32527: URL: https://github.com/apache/spark/pull/32527#issuecomment-840217408 **[Test build #138480 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138480/testReport)** for PR 32527 at commit

[GitHub] [spark] sunchao commented on a change in pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

sunchao commented on a change in pull request #32527: URL: https://github.com/apache/spark/pull/32527#discussion_r631501337 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala ## @@ -127,13 +128,19 @@ trait InvokeLike

[GitHub] [spark] maropu commented on a change in pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

maropu commented on a change in pull request #32527: URL: https://github.com/apache/spark/pull/32527#discussion_r631499051 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala ## @@ -127,13 +128,19 @@ trait InvokeLike

[GitHub] [spark] sunchao commented on a change in pull request #32488: [SPARK-35316][SQL] UnwrapCastInBinaryComparison support In/InSet predicate

sunchao commented on a change in pull request #32488: URL: https://github.com/apache/spark/pull/32488#discussion_r631498596 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/UnwrapCastInBinaryComparisonSuite.scala ## @@ -233,6 +233,15 @@ class

[GitHub] [spark] SparkQA commented on pull request #32528: [SPARK-35350][SQL] Add code-gen for left semi sort merge join

SparkQA commented on pull request #32528: URL: https://github.com/apache/spark/pull/32528#issuecomment-840211154 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] SparkQA commented on pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

SparkQA commented on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-840210720 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] SparkQA removed a comment on pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

SparkQA removed a comment on pull request #32498: URL: https://github.com/apache/spark/pull/32498#issuecomment-840083265 **[Test build #138473 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138473/testReport)** for PR 32498 at commit

[GitHub] [spark] SparkQA commented on pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

SparkQA commented on pull request #32498: URL: https://github.com/apache/spark/pull/32498#issuecomment-840209168 **[Test build #138473 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138473/testReport)** for PR 32498 at commit

[GitHub] [spark] SparkQA commented on pull request #32498: [SPARK-35368][SQL] Update histogram statistics for RANGE operator for stats estimation

SparkQA commented on pull request #32498: URL: https://github.com/apache/spark/pull/32498#issuecomment-840207182 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42998/ --

[GitHub] [spark] sunchao commented on pull request #32527: [SPARK-35384][SQL] Improve performance for InvokeLike.invoke

sunchao commented on pull request #32527: URL: https://github.com/apache/spark/pull/32527#issuecomment-840204966 cc @HyukjinKwon @cloud-fan @dongjoon-hyun @viirya @maropu -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] maropu edited a comment on pull request #32455: [SPARK-35253][SQL][BUILD] Bump up the janino version to v3.1.4

maropu edited a comment on pull request #32455: URL: https://github.com/apache/spark/pull/32455#issuecomment-840203738 Thank you, all the reviewers~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] maropu commented on pull request #32455: [SPARK-35253][SQL][BUILD] Bump up the janino version to v3.1.4

maropu commented on pull request #32455: URL: https://github.com/apache/spark/pull/32455#issuecomment-840203738 Thank you for all the reviewers~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] maropu commented on a change in pull request #32476: [SPARK-35349][SQL] Add code-gen for left/right outer sort merge join

maropu commented on a change in pull request #32476: URL: https://github.com/apache/spark/pull/32476#discussion_r631492986 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala ## @@ -418,115 +443,140 @@ case class

[GitHub] [spark] maropu commented on pull request #32520: [SPARK-35385][SQL][TESTS] Skip duplicate queries in the TPCDS-related tests

maropu commented on pull request #32520: URL: https://github.com/apache/spark/pull/32520#issuecomment-840200018 The last commit is only for the comment update, so I merged to master. Thank you, @cloud-fan @dongjoon-hyun -- This is an automated message from the Apache Git Service. To

[GitHub] [spark] maropu closed pull request #32520: [SPARK-35385][SQL][TESTS] Skip duplicate queries in the TPCDS-related tests

maropu closed pull request #32520: URL: https://github.com/apache/spark/pull/32520 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [spark] SparkQA commented on pull request #32520: [SPARK-35385][SQL][TESTS] Skip duplicate queries in the TPCDS-related tests

SparkQA commented on pull request #32520: URL: https://github.com/apache/spark/pull/32520#issuecomment-840197479 **[Test build #138479 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138479/testReport)** for PR 32520 at commit

[GitHub] [spark] maropu commented on a change in pull request #32520: [SPARK-35385][SQL][TESTS] Skip duplicate queries in the TPCDS-related tests

maropu commented on a change in pull request #32520: URL: https://github.com/apache/spark/pull/32520#discussion_r631489998 ## File path: sql/core/src/test/scala/org/apache/spark/sql/TPCDSBase.scala ## @@ -36,6 +36,12 @@ trait TPCDSBase extends SharedSparkSession with

[GitHub] [spark] maropu commented on a change in pull request #32520: [SPARK-35385][SQL][TESTS] Skip duplicate queries in the TPCDS-related tests

maropu commented on a change in pull request #32520: URL: https://github.com/apache/spark/pull/32520#discussion_r631489676 ## File path: sql/core/src/test/scala/org/apache/spark/sql/TPCDSBase.scala ## @@ -36,6 +36,12 @@ trait TPCDSBase extends SharedSparkSession with

[GitHub] [spark] SparkQA commented on pull request #32494: [SPARK-35362][SQL] Update null count in the column stats for UNION operator stats estimation

SparkQA commented on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-840190295 **[Test build #138478 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138478/testReport)** for PR 32494 at commit

<    1   2   3   4   5   6   7   8   >