[GitHub] [spark] HyukjinKwon commented on pull request #29145: [SPARK-32346][SQL] Support filters pushdown in Avro datasource

HyukjinKwon commented on pull request #29145: URL: https://github.com/apache/spark/pull/29145#issuecomment-662258665 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort plans

dongjoon-hyun edited a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-662257917 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29170: [SPARK-30876][SQL] Optimizer fails to infer constraints within join

AmplabJenkins commented on pull request #29170: URL: https://github.com/apache/spark/pull/29170#issuecomment-662258378 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] adjordan commented on pull request #29080: [SPARK-32271][ML] Add option for k-fold cross-validation to CrossValidator

adjordan commented on pull request #29080: URL: https://github.com/apache/spark/pull/29080#issuecomment-662257891 @viirya I made the changes and added unit tests. Let me know what you think. This is an automated message from

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort plans

dongjoon-hyun edited a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-662257917 @frankyin-factual . Thank you for updating. In general, it's a nice improvement contribution. - For the test cases, you may have a different idea. - For the

[GitHub] [spark] dongjoon-hyun commented on pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort plans

dongjoon-hyun commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-662257917 @frankyin-factual . Thank you for updating. In general, it's a nice improvement contribution. - For the test cases, you may have a different idea. - For the

[GitHub] [spark] SparkQA removed a comment on pull request #29170: [SPARK-30876][SQL] Optimizer fails to infer constraints within join

SparkQA removed a comment on pull request #29170: URL: https://github.com/apache/spark/pull/29170#issuecomment-662168094 **[Test build #126285 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126285/testReport)** for PR 29170 at commit

[GitHub] [spark] SparkQA commented on pull request #29170: [SPARK-30876][SQL] Optimizer fails to infer constraints within join

SparkQA commented on pull request #29170: URL: https://github.com/apache/spark/pull/29170#issuecomment-662257654 **[Test build #126285 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126285/testReport)** for PR 29170 at commit

[GitHub] [spark] MaxGekk commented on pull request #29176: [MINOR][SQL][TESTS] Create tables once in JDBC tests

MaxGekk commented on pull request #29176: URL: https://github.com/apache/spark/pull/29176#issuecomment-662256851 ping @gengliangwang @HyukjinKwon @cloud-fan This is an automated message from the Apache Git Service. To

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort plans

dongjoon-hyun commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r458550545 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,144 @@ class

[GitHub] [spark] frankyin-factual commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort plans

frankyin-factual commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r458550558 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,144 @@

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort plans

dongjoon-hyun commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r458550049 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,144 @@ class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

AmplabJenkins removed a comment on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-662256175 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

AmplabJenkins commented on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-662256175 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] jovany-wang commented on pull request #29065: [WIP][SPARK-32268][SQL] Bloom Filter Join

jovany-wang commented on pull request #29065: URL: https://github.com/apache/spark/pull/29065#issuecomment-662255897 Hi @wangyum , This is a nice PR to me. But some issues in my mind should be thrown here. I didn't do more perf between MinMax and Bloom, but in my personal sense,

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort plans

dongjoon-hyun commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r458548850 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -39,6 +39,20 @@ object

[GitHub] [spark] SparkQA commented on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

SparkQA commented on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-662255813 **[Test build #126302 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126302/testReport)** for PR 29014 at commit

[GitHub] [spark] cloud-fan commented on pull request #29156: [SPARK-32347][SQL] Hint in CTE should be resolved in Hints batch rule

cloud-fan commented on pull request #29156: URL: https://github.com/apache/spark/pull/29156#issuecomment-662255460 > cannot resolve 'id' given input columns: [cte.id]; Do you know how this happens? The `id` column is there.

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort plans

dongjoon-hyun commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r458548850 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -39,6 +39,20 @@ object

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort plans

dongjoon-hyun commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r458548850 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -39,6 +39,20 @@ object

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort plans

dongjoon-hyun commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r458548850 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasing.scala ## @@ -39,6 +39,20 @@ object

[GitHub] [spark] frankyin-factual commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort plans

frankyin-factual commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r458548001 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,144 @@

[GitHub] [spark] frankyin-factual commented on pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort plans

frankyin-factual commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-662254203 > "Allow nested schema pruning thru window/sort/filter plans" looks like a little an over-claim to me. Technically, this PR doesn't support all general `Filter` plans,

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort plans

dongjoon-hyun commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r458547587 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,144 @@ class

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

dongjoon-hyun commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r458547587 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,144 @@ class

[GitHub] [spark] AmplabJenkins commented on pull request #29032: [SPARK-32217] Plumb whether a worker would also be decommissioned along with executor

AmplabJenkins commented on pull request #29032: URL: https://github.com/apache/spark/pull/29032#issuecomment-662253753 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29032: [SPARK-32217] Plumb whether a worker would also be decommissioned along with executor

AmplabJenkins removed a comment on pull request #29032: URL: https://github.com/apache/spark/pull/29032#issuecomment-662253753 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] frankyin-factual commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

frankyin-factual commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r458547437 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,144 @@

[GitHub] [spark] SparkQA commented on pull request #29032: [SPARK-32217] Plumb whether a worker would also be decommissioned along with executor

SparkQA commented on pull request #29032: URL: https://github.com/apache/spark/pull/29032#issuecomment-662253416 **[Test build #126301 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126301/testReport)** for PR 29032 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec

AmplabJenkins removed a comment on pull request #27019: URL: https://github.com/apache/spark/pull/27019#issuecomment-662252775 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] frankyin-factual commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

frankyin-factual commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r458546579 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,144 @@

[GitHub] [spark] AmplabJenkins commented on pull request #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec

AmplabJenkins commented on pull request #27019: URL: https://github.com/apache/spark/pull/27019#issuecomment-662252765 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec

AmplabJenkins removed a comment on pull request #27019: URL: https://github.com/apache/spark/pull/27019#issuecomment-662252765 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] frankyin-factual commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

frankyin-factual commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r458546414 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,144 @@

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

dongjoon-hyun commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r458546372 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,144 @@ class

[GitHub] [spark] SparkQA removed a comment on pull request #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec

SparkQA removed a comment on pull request #27019: URL: https://github.com/apache/spark/pull/27019#issuecomment-662172469 **[Test build #126286 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126286/testReport)** for PR 27019 at commit

[GitHub] [spark] SparkQA commented on pull request #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec

SparkQA commented on pull request #27019: URL: https://github.com/apache/spark/pull/27019#issuecomment-662252159 **[Test build #126286 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126286/testReport)** for PR 27019 at commit

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

dongjoon-hyun commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r458545107 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,144 @@ class

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

dongjoon-hyun commented on a change in pull request #28898: URL: https://github.com/apache/spark/pull/28898#discussion_r458544989 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/NestedColumnAliasingSuite.scala ## @@ -493,6 +491,144 @@ class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AmplabJenkins removed a comment on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662249329 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AmplabJenkins commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662249329 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] zhengruifeng commented on a change in pull request #29159: [SPARK-32310][ML][PySpark][3.0] ML params default value parity

zhengruifeng commented on a change in pull request #29159: URL: https://github.com/apache/spark/pull/29159#discussion_r458542336 ## File path: mllib/src/test/scala/org/apache/spark/ml/util/DefaultReadWriteTest.scala ## @@ -64,7 +64,12 @@ trait DefaultReadWriteTest extends

[GitHub] [spark] SparkQA commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

SparkQA commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662249043 **[Test build #126300 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126300/testReport)** for PR 29085 at commit

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r458540858 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/BaseScriptTransformationExec.scala ## @@ -87,17 +175,69 @@ trait

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

AmplabJenkins removed a comment on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-662247330 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

AmplabJenkins commented on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-662247330 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29182: [SPARK-32003][CORE][2.4] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

AmplabJenkins commented on pull request #29182: URL: https://github.com/apache/spark/pull/29182#issuecomment-662247271 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29182: [SPARK-32003][CORE][2.4] When external shuffle service is used, unregister outputs for executor on fetch failure after executor

AmplabJenkins removed a comment on pull request #29182: URL: https://github.com/apache/spark/pull/29182#issuecomment-662247271 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

SparkQA removed a comment on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-662208163 **[Test build #126293 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126293/testReport)** for PR 29014 at commit

[GitHub] [spark] SparkQA commented on pull request #29182: [SPARK-32003][CORE][2.4] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

SparkQA commented on pull request #29182: URL: https://github.com/apache/spark/pull/29182#issuecomment-662246904 **[Test build #126299 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126299/testReport)** for PR 29182 at commit

[GitHub] [spark] SparkQA commented on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

SparkQA commented on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-662246791 **[Test build #126293 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126293/testReport)** for PR 29014 at commit

[GitHub] [spark] wypoon commented on pull request #29182: [SPARK-32003][CORE][2.4] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

wypoon commented on pull request #29182: URL: https://github.com/apache/spark/pull/29182#issuecomment-662245932 This is a backport of https://github.com/apache/spark/pull/28848 to branch-2.4. There were two files that were changed. The backport of DAGScheduler.scala was

[GitHub] [spark] wypoon commented on pull request #28848: [SPARK-32003][CORE] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

wypoon commented on pull request #28848: URL: https://github.com/apache/spark/pull/28848#issuecomment-662246115 I created https://github.com/apache/spark/pull/29182 for the backport to branch-2.4. This is an automated

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AmplabJenkins removed a comment on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662245159 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] wypoon opened a new pull request #29182: [SPARK-32003][CORE][2.4] When external shuffle service is used, unregister outputs for executor on fetch failure after executor is lost

wypoon opened a new pull request #29182: URL: https://github.com/apache/spark/pull/29182 ### What changes were proposed in this pull request? If an executor is lost, the `DAGScheduler` handles the executor loss by removing the executor but does not unregister its outputs if the

[GitHub] [spark] AmplabJenkins commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AmplabJenkins commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662245159 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

SparkQA commented on pull request #29085: URL: https://github.com/apache/spark/pull/29085#issuecomment-662244867 **[Test build #126298 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126298/testReport)** for PR 29085 at commit

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r458537184 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/BaseScriptTransformationExec.scala ## @@ -56,10 +69,85 @@ trait

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29080: [WIP][SPARK-32271][ML] Add option for k-fold cross-validation to CrossValidator

AmplabJenkins removed a comment on pull request #29080: URL: https://github.com/apache/spark/pull/29080#issuecomment-662243131 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29080: [WIP][SPARK-32271][ML] Add option for k-fold cross-validation to CrossValidator

AmplabJenkins commented on pull request #29080: URL: https://github.com/apache/spark/pull/29080#issuecomment-662243131 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] baohe-zhang commented on pull request #29149: [SPARK-32350][CORE] Add batch-write on LevelDB to improve performance of HybridStore

baohe-zhang commented on pull request #29149: URL: https://github.com/apache/spark/pull/29149#issuecomment-662242851 Thanks for the review! This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] SparkQA commented on pull request #29080: [WIP][SPARK-32271][ML] Add option for k-fold cross-validation to CrossValidator

SparkQA commented on pull request #29080: URL: https://github.com/apache/spark/pull/29080#issuecomment-662242821 **[Test build #126297 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126297/testReport)** for PR 29080 at commit

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r458534534 ## File path: sql/core/src/test/resources/sql-tests/inputs/transform.sql ## @@ -0,0 +1,72 @@ +-- Test data. +CREATE OR REPLACE TEMPORARY VIEW t1 AS

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

AmplabJenkins removed a comment on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-662241141 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

AmplabJenkins commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-662241141 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

SparkQA commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-662240866 **[Test build #126296 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126296/testReport)** for PR 28898 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

dongjoon-hyun commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-662240643 Retest this please. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins removed a comment on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-662239900 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins removed a comment on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-662239897 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins commented on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-662239897 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

SparkQA removed a comment on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-662203969 **[Test build #126292 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126292/testReport)** for PR 28818 at commit

[GitHub] [spark] stijndehaes commented on pull request #28423: [SPARK-24266][k8s] Restart the watcher when we receive a version changed from k8s

stijndehaes commented on pull request #28423: URL: https://github.com/apache/spark/pull/28423#issuecomment-662239237 @holdenk my JIRA username if sdehaes This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r458532562 ## File path: sql/core/src/test/resources/sql-tests/inputs/transform.sql ## @@ -0,0 +1,72 @@ +-- Test data. +CREATE OR REPLACE TEMPORARY VIEW t1 AS

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r458532644 ## File path: sql/core/src/test/resources/sql-tests/inputs/transform.sql ## @@ -0,0 +1,72 @@ +-- Test data. +CREATE OR REPLACE TEMPORARY VIEW t1 AS

[GitHub] [spark] SparkQA commented on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

SparkQA commented on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-662239504 **[Test build #126292 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126292/testReport)** for PR 28818 at commit

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r458532129 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/BaseScriptTransformationSuite.scala ## @@ -0,0 +1,367 @@ +/* + * Licensed to

[GitHub] [spark] dongjoon-hyun commented on pull request #28898: [SPARK-32059][SQL] Allow nested schema pruning thru window/sort/filter plans

dongjoon-hyun commented on pull request #28898: URL: https://github.com/apache/spark/pull/28898#issuecomment-662237766 Thank you for pinging me, @frankyin-factual , @maropu , @viirya . I'll take a look at this PR. This is

[GitHub] [spark] SparkQA commented on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

SparkQA commented on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-662237102 **[Test build #126295 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126295/testReport)** for PR 29014 at commit

[GitHub] [spark] LantaoJin commented on a change in pull request #29062: [SPARK-32237][SQL] Resolve hint in CTE

LantaoJin commented on a change in pull request #29062: URL: https://github.com/apache/spark/pull/29062#discussion_r458529424 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -200,18 +200,18 @@ class Analyzer( val

[GitHub] [spark] LantaoJin edited a comment on pull request #29062: [SPARK-32237][SQL] Resolve hint in CTE

LantaoJin edited a comment on pull request #29062: URL: https://github.com/apache/spark/pull/29062#issuecomment-662233804 > I checked the order of analysis rules in branch-2.4. Substitution batch is after Hints too, like branch-3.0. Is there any change causing this issue? @viirya I

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

AmplabJenkins removed a comment on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-662235589 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

AmplabJenkins commented on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-662235589 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HyukjinKwon commented on pull request #29014: [SPARK-32199][SPARK-32198] Reduce job failures during decommissioning

HyukjinKwon commented on pull request #29014: URL: https://github.com/apache/spark/pull/29014#issuecomment-662235378 add to whitelist This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29178: [SPARK-32380][SQL] fixed spark3.0 access hive table while data in hbase problem

HyukjinKwon commented on a change in pull request #29178: URL: https://github.com/apache/spark/pull/29178#discussion_r458527958 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ## @@ -299,7 +299,9 @@ class HadoopTableReader( */ private

[GitHub] [spark] LantaoJin edited a comment on pull request #29062: [SPARK-32237][SQL] Resolve hint in CTE

LantaoJin edited a comment on pull request #29062: URL: https://github.com/apache/spark/pull/29062#issuecomment-662233804 > I checked the order of analysis rules in branch-2.4. Substitution batch is after Hints too, like branch-3.0. Is there any change causing this issue? @viirya I

[GitHub] [spark] HeartSaVioR closed pull request #29149: [SPARK-32350][CORE] Add batch-write on LevelDB to improve performance of HybridStore

HeartSaVioR closed pull request #29149: URL: https://github.com/apache/spark/pull/29149 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] LantaoJin edited a comment on pull request #29062: [SPARK-32237][SQL] Resolve hint in CTE

LantaoJin edited a comment on pull request #29062: URL: https://github.com/apache/spark/pull/29062#issuecomment-662233804 > I checked the order of analysis rules in branch-2.4. Substitution batch is after Hints too, like branch-3.0. Is there any change causing this issue? I found

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29149: [SPARK-32350][CORE] Add batch-write on LevelDB to improve performance of HybridStore

AmplabJenkins removed a comment on pull request #29149: URL: https://github.com/apache/spark/pull/29149#issuecomment-662233648 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29149: [SPARK-32350][CORE] Add batch-write on LevelDB to improve performance of HybridStore

AmplabJenkins commented on pull request #29149: URL: https://github.com/apache/spark/pull/29149#issuecomment-662233648 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] LantaoJin commented on pull request #29062: [SPARK-32237][SQL] Resolve hint in CTE

LantaoJin commented on pull request #29062: URL: https://github.com/apache/spark/pull/29062#issuecomment-662233804 > Can you keep the original PR template? Like "What changes were proposed in this pull request?" was removed. Please describe why the issue happened and how this fixes it in

[GitHub] [spark] HeartSaVioR commented on pull request #29149: [SPARK-32350][CORE] Add batch-write on LevelDB to improve performance of HybridStore

HeartSaVioR commented on pull request #29149: URL: https://github.com/apache/spark/pull/29149#issuecomment-662233722 Thanks! Merging to master. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA removed a comment on pull request #29149: [SPARK-32350][CORE] Add batch-write on LevelDB to improve performance of HybridStore

SparkQA removed a comment on pull request #29149: URL: https://github.com/apache/spark/pull/29149#issuecomment-662190043 **[Test build #126290 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126290/testReport)** for PR 29149 at commit

[GitHub] [spark] SparkQA commented on pull request #29149: [SPARK-32350][CORE] Add batch-write on LevelDB to improve performance of HybridStore

SparkQA commented on pull request #29149: URL: https://github.com/apache/spark/pull/29149#issuecomment-662233254 **[Test build #126290 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126290/testReport)** for PR 29149 at commit

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r458524392 ## File path: sql/core/src/test/resources/sql-tests/inputs/transform.sql ## @@ -0,0 +1,72 @@ +-- Test data. +CREATE OR REPLACE TEMPORARY VIEW t1 AS

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r458525381 ## File path: sql/core/src/test/resources/sql-tests/inputs/transform.sql ## @@ -0,0 +1,72 @@ +-- Test data. +CREATE OR REPLACE TEMPORARY VIEW t1 AS

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29107: [SPARK-32308][SQL] Move by-name resolution logic of unionByName from API code to analysis phase

AmplabJenkins removed a comment on pull request #29107: URL: https://github.com/apache/spark/pull/29107#issuecomment-662232123 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29107: [SPARK-32308][SQL] Move by-name resolution logic of unionByName from API code to analysis phase

AmplabJenkins commented on pull request #29107: URL: https://github.com/apache/spark/pull/29107#issuecomment-662232123 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29107: [SPARK-32308][SQL] Move by-name resolution logic of unionByName from API code to analysis phase

SparkQA commented on pull request #29107: URL: https://github.com/apache/spark/pull/29107#issuecomment-662231827 **[Test build #126294 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/126294/testReport)** for PR 29107 at commit

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r458524287 ## File path: sql/core/src/test/resources/sql-tests/inputs/transform.sql ## @@ -0,0 +1,72 @@ +-- Test data. +CREATE OR REPLACE TEMPORARY VIEW t1 AS

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29085: [SPARK-32106][SQL]Implement SparkScriptTransformationExec in sql/core

AngersZh commented on a change in pull request #29085: URL: https://github.com/apache/spark/pull/29085#discussion_r458524490 ## File path: sql/core/src/test/resources/sql-tests/inputs/transform.sql ## @@ -0,0 +1,72 @@ +-- Test data. +CREATE OR REPLACE TEMPORARY VIEW t1 AS

  1   2   3   4   5   6   7   8   9   10   >