[GitHub] [spark] HyukjinKwon commented on pull request #29465: [SPARK-32249][2.4] Run Github Actions builds in other branches as well

HyukjinKwon commented on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675830857 It should be ready to be reviewed or merged now. This is an automated message from the Apache Git Service.

[GitHub] [spark] HyukjinKwon closed pull request #29454: [SPARK-32645][INFRA] Upload unit-tests.log as an artifact

HyukjinKwon closed pull request #29454: URL: https://github.com/apache/spark/pull/29454 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29460: [SPARK-32249][3.0] Run Github Actions builds in other branches as well

AmplabJenkins removed a comment on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675830098 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29460: [SPARK-32249][3.0] Run Github Actions builds in other branches as well

AmplabJenkins commented on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675830098 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29465: [SPARK-32249][2.4] Run Github Actions builds in other branches as well

AmplabJenkins removed a comment on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675830101 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29465: [SPARK-32249][2.4] Run Github Actions builds in other branches as well

AmplabJenkins commented on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675830101 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HyukjinKwon commented on pull request #29454: [SPARK-32645][INFRA] Upload unit-tests.log as an artifact

HyukjinKwon commented on pull request #29454: URL: https://github.com/apache/spark/pull/29454#issuecomment-675830074 Thanks guys. Merged to master. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] Ngone51 commented on pull request #29468: [SPARK-32653][CORE] Decommissioned host/executor should be considered as inactive in TaskSchedulerImpl

Ngone51 commented on pull request #29468: URL: https://github.com/apache/spark/pull/29468#issuecomment-675830001 Have synced with @agrawaldevesh offline, I've updated the PR description according to his questions. This is

[GitHub] [spark] HyukjinKwon commented on pull request #29460: [SPARK-32249][3.0] Run Github Actions builds in other branches as well

HyukjinKwon commented on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675829483 I just turned this as single PR. This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] SparkQA commented on pull request #29465: [SPARK-32249][2.4] Run Github Actions builds in other branches as well

SparkQA commented on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675829674 **[Test build #127613 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127613/testReport)** for PR 29465 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #29469: [SPARK-28863][SQL] Introduce AlreadyPlanned to prevent reanalysis of V1FallbackWriters

cloud-fan commented on a change in pull request #29469: URL: https://github.com/apache/spark/pull/29469#discussion_r472639468 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/AlreadyPlannedSuite.scala ## @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache

[GitHub] [spark] SparkQA commented on pull request #29460: [SPARK-32249][3.0] Run Github Actions builds in other branches as well

SparkQA commented on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675829698 **[Test build #127614 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127614/testReport)** for PR 29460 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #29469: [SPARK-28863][SQL] Introduce AlreadyPlanned to prevent reanalysis of V1FallbackWriters

cloud-fan commented on a change in pull request #29469: URL: https://github.com/apache/spark/pull/29469#discussion_r472636665 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/AlreadyPlanned.scala ## @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] leanken commented on a change in pull request #29455: [SPARK-32644][SQL] NAAJ support for ShuffleHashJoin when AQE is on

leanken commented on a change in pull request #29455: URL: https://github.com/apache/spark/pull/29455#discussion_r472635946 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/EliminateNullAwareAntiJoin.scala ## @@ -20,22 +20,50 @@ package

[GitHub] [spark] AmplabJenkins commented on pull request #29460: [DO-NOT-MERGE][SPARK-32249][3.0] Run Github Actions builds in other branches as well

AmplabJenkins commented on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675828269 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29460: [DO-NOT-MERGE][SPARK-32249][3.0] Run Github Actions builds in other branches as well

AmplabJenkins removed a comment on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675828269 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29460: [DO-NOT-MERGE][SPARK-32249][3.0] Run Github Actions builds in other branches as well

SparkQA commented on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675827919 **[Test build #127612 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127612/testReport)** for PR 29460 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29465: [DO-NOT-MERGE][SPARK-32249][2.4] Run Github Actions builds in other branches as well

AmplabJenkins removed a comment on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675826456 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on pull request #29465: [DO-NOT-MERGE][SPARK-32249][2.4] Run Github Actions builds in other branches as well

SparkQA removed a comment on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675826013 **[Test build #127611 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127611/testReport)** for PR 29465 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29465: [DO-NOT-MERGE][SPARK-32249][2.4] Run Github Actions builds in other branches as well

AmplabJenkins removed a comment on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675826420 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29465: [DO-NOT-MERGE][SPARK-32249][2.4] Run Github Actions builds in other branches as well

AmplabJenkins commented on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675826420 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29465: [DO-NOT-MERGE][SPARK-32249][2.4] Run Github Actions builds in other branches as well

SparkQA commented on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675826440 **[Test build #127611 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127611/testReport)** for PR 29465 at commit

[GitHub] [spark] SparkQA commented on pull request #29465: [DO-NOT-MERGE][SPARK-32249][2.4] Run Github Actions builds in other branches as well

SparkQA commented on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675826013 **[Test build #127611 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127611/testReport)** for PR 29465 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29472: [SPARK-32655][K8S] Support appId/execId placeholder in K8s `spark.local.dir` conf

AmplabJenkins removed a comment on pull request #29472: URL: https://github.com/apache/spark/pull/29472#issuecomment-675825246 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on pull request #29472: [SPARK-32655][K8S] Support appId/execId placeholder in K8s `spark.local.dir` conf

SparkQA commented on pull request #29472: URL: https://github.com/apache/spark/pull/29472#issuecomment-675825229 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/32233/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29472: [SPARK-32655][K8S] Support appId/execId placeholder in K8s `spark.local.dir` conf

AmplabJenkins removed a comment on pull request #29472: URL: https://github.com/apache/spark/pull/29472#issuecomment-675825241 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29472: [SPARK-32655][K8S] Support appId/execId placeholder in K8s `spark.local.dir` conf

AmplabJenkins commented on pull request #29472: URL: https://github.com/apache/spark/pull/29472#issuecomment-675825241 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29082: [SPARK-32288][UI] Add exception summary for failed tasks in stage page

AmplabJenkins removed a comment on pull request #29082: URL: https://github.com/apache/spark/pull/29082#issuecomment-675822207 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29082: [SPARK-32288][UI] Add exception summary for failed tasks in stage page

AmplabJenkins commented on pull request #29082: URL: https://github.com/apache/spark/pull/29082#issuecomment-675822207 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29082: [SPARK-32288][UI] Add exception summary for failed tasks in stage page

SparkQA removed a comment on pull request #29082: URL: https://github.com/apache/spark/pull/29082#issuecomment-675784894 **[Test build #127605 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127605/testReport)** for PR 29082 at commit

[GitHub] [spark] SparkQA commented on pull request #29082: [SPARK-32288][UI] Add exception summary for failed tasks in stage page

SparkQA commented on pull request #29082: URL: https://github.com/apache/spark/pull/29082#issuecomment-675821746 **[Test build #127605 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127605/testReport)** for PR 29082 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29465: [DO-NOT-MERGE][SPARK-32249][2.4] Run Github Actions builds in other branches as well

AmplabJenkins commented on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675820455 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29465: [DO-NOT-MERGE][SPARK-32249][2.4] Run Github Actions builds in other branches as well

AmplabJenkins removed a comment on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675820455 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29472: [SPARK-32655][K8S] Support appId/execId placeholder in K8s `spark.local.dir` conf

SparkQA commented on pull request #29472: URL: https://github.com/apache/spark/pull/29472#issuecomment-675820285 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/32233/

[GitHub] [spark] SparkQA commented on pull request #29465: [DO-NOT-MERGE][SPARK-32249][2.4] Run Github Actions builds in other branches as well

SparkQA commented on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675820180 **[Test build #127604 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127604/testReport)** for PR 29465 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29465: [DO-NOT-MERGE][SPARK-32249][2.4] Run Github Actions builds in other branches as well

SparkQA removed a comment on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675783080 **[Test build #127604 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127604/testReport)** for PR 29465 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29472: [SPARK-32655][K8S] Support appId/execId placeholder in K8s `spark.local.dir` conf

AmplabJenkins commented on pull request #29472: URL: https://github.com/apache/spark/pull/29472#issuecomment-675812852 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/127610/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29472: [SPARK-32655][K8S] Support appId/execId placeholder in K8s `spark.local.dir` conf

AmplabJenkins removed a comment on pull request #29472: URL: https://github.com/apache/spark/pull/29472#issuecomment-675812852 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on pull request #29472: [SPARK-32655][K8S] Support appId/execId placeholder in K8s `spark.local.dir` conf

SparkQA commented on pull request #29472: URL: https://github.com/apache/spark/pull/29472#issuecomment-675812514 **[Test build #127610 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127610/testReport)** for PR 29472 at commit

[GitHub] [spark] maropu commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

maropu commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r472595471 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/PathFilterSuite.scala ## @@ -0,0 +1,501 @@ +/* + * Licensed to the

[GitHub] [spark] maropu commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

maropu commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r472595471 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/PathFilterSuite.scala ## @@ -0,0 +1,501 @@ +/* + * Licensed to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29418: [SPARK-32600][CORE] Unify task name in some logs between driver and executor

AmplabJenkins removed a comment on pull request #29418: URL: https://github.com/apache/spark/pull/29418#issuecomment-675810331 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29418: [SPARK-32600][CORE] Unify task name in some logs between driver and executor

AmplabJenkins commented on pull request #29418: URL: https://github.com/apache/spark/pull/29418#issuecomment-675810331 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] beliefer commented on a change in pull request #29228: [SPARK-31847][CORE][TESTS] DAGSchedulerSuite: Rewrite the test framework to support apply specified spark configurations.

beliefer commented on a change in pull request #29228: URL: https://github.com/apache/spark/pull/29228#discussion_r472594156 ## File path: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala ## @@ -295,7 +298,20 @@ class DAGSchedulerSuite extends

[GitHub] [spark] SparkQA removed a comment on pull request #29418: [SPARK-32600][CORE] Unify task name in some logs between driver and executor

SparkQA removed a comment on pull request #29418: URL: https://github.com/apache/spark/pull/29418#issuecomment-675770877 **[Test build #127603 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127603/testReport)** for PR 29418 at commit

[GitHub] [spark] SparkQA commented on pull request #29418: [SPARK-32600][CORE] Unify task name in some logs between driver and executor

SparkQA commented on pull request #29418: URL: https://github.com/apache/spark/pull/29418#issuecomment-675809789 **[Test build #127603 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127603/testReport)** for PR 29418 at commit

[GitHub] [spark] SparkQA commented on pull request #29472: [SPARK-32655][K8S] Support appId/execId placeholder in K8s `spark.local.dir` conf

SparkQA commented on pull request #29472: URL: https://github.com/apache/spark/pull/29472#issuecomment-675809644 **[Test build #127610 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127610/testReport)** for PR 29472 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #29472: [SPARK-32655][K8S] Support appId/execId placeholder in K8s `spark.local.dir` conf

dongjoon-hyun commented on pull request #29472: URL: https://github.com/apache/spark/pull/29472#issuecomment-675809298 The integration test failure is irrelevant to this one. ``` - Test basic decommissioning *** FAILED *** ``` cc @holdenk for the K8s IT failure.

[GitHub] [spark] leanken commented on a change in pull request #29455: [SPARK-32644][SQL] NAAJ support for ShuffleHashJoin when AQE is on

leanken commented on a change in pull request #29455: URL: https://github.com/apache/spark/pull/29455#discussion_r472592607 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/EliminateNullAwareAntiJoin.scala ## @@ -20,22 +20,50 @@ package

[GitHub] [spark] maropu commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

maropu commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r472592405 ## File path: docs/sql-data-sources-generic-options.md ## @@ -119,3 +119,48 @@ To load all files recursively, you can use: {% include_example

[GitHub] [spark] leanken commented on a change in pull request #29455: [SPARK-32644][SQL] NAAJ support for ShuffleHashJoin when AQE is on

leanken commented on a change in pull request #29455: URL: https://github.com/apache/spark/pull/29455#discussion_r472592192 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -317,4 +318,41 @@ case class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29472: [SPARK-32655][K8S] Support appId/execId placeholder in K8s executor SPARK_LOCAL_DIRS

AmplabJenkins removed a comment on pull request #29472: URL: https://github.com/apache/spark/pull/29472#issuecomment-675807816 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29472: [SPARK-32655][K8S] Support appId/execId placeholder in K8s executor SPARK_LOCAL_DIRS

AmplabJenkins removed a comment on pull request #29472: URL: https://github.com/apache/spark/pull/29472#issuecomment-675807809 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #29472: [SPARK-32655][K8S] Support appId/execId placeholder in K8s executor SPARK_LOCAL_DIRS

SparkQA commented on pull request #29472: URL: https://github.com/apache/spark/pull/29472#issuecomment-675807796 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/32230/

[GitHub] [spark] AmplabJenkins commented on pull request #29472: [SPARK-32655][K8S] Support appId/execId placeholder in K8s executor SPARK_LOCAL_DIRS

AmplabJenkins commented on pull request #29472: URL: https://github.com/apache/spark/pull/29472#issuecomment-675807809 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] maropu commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

maropu commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r472591099 ## File path: docs/sql-data-sources-generic-options.md ## @@ -119,3 +119,48 @@ To load all files recursively, you can use: {% include_example

[GitHub] [spark] maropu commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

maropu commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r472591183 ## File path: docs/sql-data-sources-generic-options.md ## @@ -119,3 +119,48 @@ To load all files recursively, you can use: {% include_example

[GitHub] [spark] maropu commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

maropu commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r472591099 ## File path: docs/sql-data-sources-generic-options.md ## @@ -119,3 +119,48 @@ To load all files recursively, you can use: {% include_example

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29455: [SPARK-32644][SQL] NAAJ support for ShuffleHashJoin when AQE is on

agrawaldevesh commented on a change in pull request #29455: URL: https://github.com/apache/spark/pull/29455#discussion_r472579634 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/EliminateNullAwareAntiJoin.scala ## @@ -20,22 +20,50 @@ package

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29469: [SPARK-28863][SQL] Introduce AlreadyPlanned to prevent reanalysis of V1FallbackWriters

AmplabJenkins removed a comment on pull request #29469: URL: https://github.com/apache/spark/pull/29469#issuecomment-675805079 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29469: [SPARK-28863][SQL] Introduce AlreadyPlanned to prevent reanalysis of V1FallbackWriters

AmplabJenkins commented on pull request #29469: URL: https://github.com/apache/spark/pull/29469#issuecomment-675805079 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29469: [SPARK-28863][SQL] Introduce AlreadyPlanned to prevent reanalysis of V1FallbackWriters

SparkQA removed a comment on pull request #29469: URL: https://github.com/apache/spark/pull/29469#issuecomment-675731761 **[Test build #127599 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127599/testReport)** for PR 29469 at commit

[GitHub] [spark] SparkQA commented on pull request #29469: [SPARK-28863][SQL] Introduce AlreadyPlanned to prevent reanalysis of V1FallbackWriters

SparkQA commented on pull request #29469: URL: https://github.com/apache/spark/pull/29469#issuecomment-675804597 **[Test build #127599 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127599/testReport)** for PR 29469 at commit

[GitHub] [spark] leanken commented on a change in pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

leanken commented on a change in pull request #29104: URL: https://github.com/apache/spark/pull/29104#discussion_r472587295 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala ## @@ -454,6 +490,40 @@ case class

[GitHub] [spark] leanken commented on a change in pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

leanken commented on a change in pull request #29104: URL: https://github.com/apache/spark/pull/29104#discussion_r472587295 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala ## @@ -454,6 +490,40 @@ case class

[GitHub] [spark] cchighman commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

cchighman commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r472586599 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/PathFilterSuite.scala ## @@ -0,0 +1,501 @@ +/* + * Licensed to the

[GitHub] [spark] SparkQA commented on pull request #29472: [SPARK-32655][K8S] Support appId/execId placeholder in K8s executor SPARK_LOCAL_DIRS

SparkQA commented on pull request #29472: URL: https://github.com/apache/spark/pull/29472#issuecomment-675802446 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/32230/

[GitHub] [spark] cchighman commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

cchighman commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r472585965 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/PathFilterSuite.scala ## @@ -0,0 +1,501 @@ +/* + * Licensed to the

[GitHub] [spark] maropu commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

maropu commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r472585785 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/PathFilterSuite.scala ## @@ -0,0 +1,501 @@ +/* + * Licensed to the

[GitHub] [spark] leanken commented on a change in pull request #29104: [SPARK-32290][SQL] SingleColumn Null Aware Anti Join Optimize

leanken commented on a change in pull request #29104: URL: https://github.com/apache/spark/pull/29104#discussion_r472585859 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala ## @@ -903,15 +910,61 @@ private[joins] object

[GitHub] [spark] maropu commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

maropu commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r472585442 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/PathFilterSuite.scala ## @@ -0,0 +1,501 @@ +/* + * Licensed to the

[GitHub] [spark] leanken commented on a change in pull request #29455: [SPARK-32644][SQL] NAAJ support for ShuffleHashJoin when AQE is on

leanken commented on a change in pull request #29455: URL: https://github.com/apache/spark/pull/29455#discussion_r472581213 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala ## @@ -235,8 +235,13 @@ abstract class SparkStrategies

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29454: [SPARK-32645][INFRA] Upload unit-tests.log as an artifact

HyukjinKwon commented on a change in pull request #29454: URL: https://github.com/apache/spark/pull/29454#discussion_r472584792 ## File path: .github/workflows/build_and_test.yml ## @@ -183,6 +183,12 @@ jobs: with: name: test-results-${{ matrix.modules }}-${{

[GitHub] [spark] maropu commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

maropu commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r472584156 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/PathFilterSuite.scala ## @@ -0,0 +1,501 @@ +/* + * Licensed to the

[GitHub] [spark] Karl-WangSK commented on pull request #29360: [SPARK-32542][SQL] Add an optimizer rule to split an Expand into multiple Expands for aggregates

Karl-WangSK commented on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-675801089 > > But shuffle is happened during Aggregate here, right? By splitting, the total amount of shuffled data is not changed, but split into several ones. Does it really result

[GitHub] [spark] c21 commented on a change in pull request #29455: [SPARK-32644][SQL] NAAJ support for ShuffleHashJoin when AQE is on

c21 commented on a change in pull request #29455: URL: https://github.com/apache/spark/pull/29455#discussion_r472583133 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala ## @@ -235,8 +235,13 @@ abstract class SparkStrategies extends

[GitHub] [spark] HyukjinKwon edited a comment on pull request #29460: [DO-NOT-MERGE][SPARK-32249][3.0] Run Github Actions builds in other branches as well

HyukjinKwon edited a comment on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675799645 Oh no. I will add one by one. I intentionally made corresponding changes to each appropriate commit. Does it make sense? If you prefer single commit way, I don't

[GitHub] [spark] HyukjinKwon edited a comment on pull request #29460: [DO-NOT-MERGE][SPARK-32249][3.0] Run Github Actions builds in other branches as well

HyukjinKwon edited a comment on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675799645 Oh no. I will add one by one. I intentionally made corresponding changes to each appropriate commit (by `git commit --amend`). Does it make sense? If you prefer

[GitHub] [spark] HyukjinKwon commented on pull request #29460: [DO-NOT-MERGE][SPARK-32249][3.0] Run Github Actions builds in other branches as well

HyukjinKwon commented on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675799645 Oh no. I will add one by one. I intentionally made corresponding changes to each appropriate commit. Does it make sense? If you prefer single commit way, I don't mind. I

[GitHub] [spark] leanken commented on a change in pull request #29455: [SPARK-32644][SQL] NAAJ support for ShuffleHashJoin when AQE is on

leanken commented on a change in pull request #29455: URL: https://github.com/apache/spark/pull/29455#discussion_r472581213 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkStrategies.scala ## @@ -235,8 +235,13 @@ abstract class SparkStrategies

[GitHub] [spark] AmplabJenkins commented on pull request #29473: [SPARK-32656][SQL] Repartition bucketed tables for sort merge join / shuffled hash join if applicable

AmplabJenkins commented on pull request #29473: URL: https://github.com/apache/spark/pull/29473#issuecomment-675797802 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29473: [SPARK-32656][SQL] Repartition bucketed tables for sort merge join / shuffled hash join if applicable

AmplabJenkins removed a comment on pull request #29473: URL: https://github.com/apache/spark/pull/29473#issuecomment-675797802 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] leanken commented on a change in pull request #29455: [SPARK-32644][SQL] NAAJ support for ShuffleHashJoin when AQE is on

leanken commented on a change in pull request #29455: URL: https://github.com/apache/spark/pull/29455#discussion_r472580755 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ShuffleExchangeExec.scala ## @@ -83,15 +83,18 @@ trait ShuffleExchangeLike

[GitHub] [spark] maropu commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

maropu commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r472580694 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/PathFilterSuite.scala ## @@ -0,0 +1,501 @@ +/* + * Licensed to the

[GitHub] [spark] SparkQA commented on pull request #29473: [SPARK-32656][SQL] Repartition bucketed tables for sort merge join / shuffled hash join if applicable

SparkQA commented on pull request #29473: URL: https://github.com/apache/spark/pull/29473#issuecomment-675797404 **[Test build #127609 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127609/testReport)** for PR 29473 at commit

[GitHub] [spark] maropu commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

maropu commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r472579334 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/pathFilters.scala ## @@ -0,0 +1,155 @@ +/* + * Licensed to the Apache

[GitHub] [spark] leanken commented on a change in pull request #29455: [SPARK-32644][SQL] NAAJ support for ShuffleHashJoin when AQE is on

leanken commented on a change in pull request #29455: URL: https://github.com/apache/spark/pull/29455#discussion_r472579516 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ShuffleExchangeExec.scala ## @@ -83,15 +83,18 @@ trait ShuffleExchangeLike

[GitHub] [spark] maropu commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

maropu commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r472579334 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/pathFilters.scala ## @@ -0,0 +1,155 @@ +/* + * Licensed to the Apache

[GitHub] [spark] maropu commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

maropu commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r472579411 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/pathFilters.scala ## @@ -0,0 +1,155 @@ +/* + * Licensed to the Apache

[GitHub] [spark] maropu commented on a change in pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

maropu commented on a change in pull request #28841: URL: https://github.com/apache/spark/pull/28841#discussion_r472578754 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/PathFilterSuite.scala ## @@ -0,0 +1,501 @@ +/* + * Licensed to the

[GitHub] [spark] imback82 opened a new pull request #29473: [WIP][SPARK-32656][SQL] Repartition bucketed tables for sort merge join / shuffled hash join if applicable

imback82 opened a new pull request #29473: URL: https://github.com/apache/spark/pull/29473 ### What changes were proposed in this pull request? #28123 and #29079 introduced coalescing bucketed tables for sort merge join / shuffled hash join. This PR proposes to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29472: [SPARK-32655][K8S] Support appId/execId placeholder in K8s executor SPARK_LOCAL_DIRS

AmplabJenkins removed a comment on pull request #29472: URL: https://github.com/apache/spark/pull/29472#issuecomment-675795289 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29472: [SPARK-32655][K8S] Support appId/execId placeholder in K8s executor SPARK_LOCAL_DIRS

AmplabJenkins commented on pull request #29472: URL: https://github.com/apache/spark/pull/29472#issuecomment-675795289 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29472: [SPARK-32655][K8S] Support appId/execId placeholder in K8s executor SPARK_LOCAL_DIRS

SparkQA removed a comment on pull request #29472: URL: https://github.com/apache/spark/pull/29472#issuecomment-675791745 **[Test build #127606 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127606/testReport)** for PR 29472 at commit

[GitHub] [spark] SparkQA commented on pull request #29472: [SPARK-32655][K8S] Support appId/execId placeholder in K8s executor SPARK_LOCAL_DIRS

SparkQA commented on pull request #29472: URL: https://github.com/apache/spark/pull/29472#issuecomment-675795175 **[Test build #127606 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127606/testReport)** for PR 29472 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29465: [DO-NOT-MERGE][SPARK-32249][2.4] Run Github Actions builds in other branches as well

AmplabJenkins removed a comment on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675793960 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29465: [DO-NOT-MERGE][SPARK-32249][2.4] Run Github Actions builds in other branches as well

AmplabJenkins commented on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675793960 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29465: [DO-NOT-MERGE][SPARK-32249][2.4] Run Github Actions builds in other branches as well

SparkQA commented on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675793629 **[Test build #127608 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127608/testReport)** for PR 29465 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #29460: [DO-NOT-MERGE][SPARK-32249][3.0] Run Github Actions builds in other branches as well

dongjoon-hyun commented on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675792926 This goes as a single commit. Did I understand correctly? This is an automated message from the Apache

[GitHub] [spark] HyukjinKwon commented on pull request #29460: [DO-NOT-MERGE][SPARK-32249][3.0] Run Github Actions builds in other branches as well

HyukjinKwon commented on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675792453 Now it should pass all tests. I will port back once I get a green. (except SparkR tests fixed at #29462).

<    1   2   3   4   5   6   7   8   9   10   >