[GitHub] [spark] LuciferYang commented on a change in pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

LuciferYang commented on a change in pull request #29000: URL: https://github.com/apache/spark/pull/29000#discussion_r472737943 ## File path: sql/core/src/test/scala/org/apache/spark/sql/sources/PartitionedWriteSuite.scala ## @@ -164,4 +165,48 @@ class PartitionedWriteSuite

[GitHub] [spark] HyukjinKwon removed a comment on pull request #29460: [SPARK-32249][INFRA][3.0] Run Github Actions builds in branch-3.0

HyukjinKwon removed a comment on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675390620 All tests passed (I am re-running to verify flakiness) except two tests: - `ThriftServerQueryTestSuite`: ``` [info]

[GitHub] [spark] SparkQA commented on pull request #29473: [SPARK-32656][SQL] Repartition bucketed tables for sort merge join / shuffled hash join if applicable

SparkQA commented on pull request #29473: URL: https://github.com/apache/spark/pull/29473#issuecomment-675875114 **[Test build #127609 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127609/testReport)** for PR 29473 at commit

[GitHub] [spark] HyukjinKwon edited a comment on pull request #29456: [SPARK-32647][INFRA] Report SparkR test results with JUnit reporter

HyukjinKwon edited a comment on pull request #29456: URL: https://github.com/apache/spark/pull/29456#issuecomment-675874159 I merged into branch-3.0 and branch-2.4 as well. cc @zhengruifeng and @ScrapCodes . this is test-only so should be fine.

[GitHub] [spark] cloud-fan commented on a change in pull request #29469: [SPARK-28863][SQL] Introduce AlreadyPlanned to prevent reanalysis of V1FallbackWriters

cloud-fan commented on a change in pull request #29469: URL: https://github.com/apache/spark/pull/29469#discussion_r472758250 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/AlreadyPlanned.scala ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] cloud-fan commented on a change in pull request #29469: [SPARK-28863][SQL] Introduce AlreadyPlanned to prevent reanalysis of V1FallbackWriters

cloud-fan commented on a change in pull request #29469: URL: https://github.com/apache/spark/pull/29469#discussion_r472758250 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/AlreadyPlanned.scala ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] SparkQA commented on pull request #29452: [SPARK-32643][CORE] Consolidate state decommissioning in the TaskSchedulerImpl realm

SparkQA commented on pull request #29452: URL: https://github.com/apache/spark/pull/29452#issuecomment-675880439 **[Test build #127625 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127625/testReport)** for PR 29452 at commit

[GitHub] [spark] agrawaldevesh commented on pull request #29468: [SPARK-32653][CORE] Decommissioned host/executor should be considered as inactive in TaskSchedulerImpl

agrawaldevesh commented on pull request #29468: URL: https://github.com/apache/spark/pull/29468#issuecomment-675883231 Thanks for updating the PR description. Now all we need is a real test to validate that the PR is actually fixing the delay scheduling issue you pointed out. The current

[GitHub] [spark] cloud-fan commented on a change in pull request #29452: [SPARK-32643][CORE] Consolidate state decommissioning in the TaskSchedulerImpl realm

cloud-fan commented on a change in pull request #29452: URL: https://github.com/apache/spark/pull/29452#discussion_r472767373 ## File path: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ## @@ -926,18 +926,21 @@ private[spark] class TaskSchedulerImpl(

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29452: [SPARK-32643][CORE] Consolidate state decommissioning in the TaskSchedulerImpl realm

AmplabJenkins removed a comment on pull request #29452: URL: https://github.com/apache/spark/pull/29452#issuecomment-675891103 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29437: [SPARK-32621][SQL] 'path' option can cause issues while inferring schema in CSV/JSON datasources

AmplabJenkins removed a comment on pull request #29437: URL: https://github.com/apache/spark/pull/29437#issuecomment-675893274 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29474: [SPARK-32658][CORE] Fix `PartitionWriterStream` partition length overflow

AmplabJenkins removed a comment on pull request #29474: URL: https://github.com/apache/spark/pull/29474#issuecomment-675893141 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #29465: [SPARK-32249][INFRA][2.4] Run Github Actions builds in branch-2.4

SparkQA removed a comment on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675858327 **[Test build #127622 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127622/testReport)** for PR 29465 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

SparkQA removed a comment on pull request #29434: URL: https://github.com/apache/spark/pull/29434#issuecomment-675838022 **[Test build #127618 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127618/testReport)** for PR 29434 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29465: [SPARK-32249][INFRA][2.4] Run Github Actions builds in branch-2.4

AmplabJenkins commented on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675893553 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28953: [SPARK-32013][SQL] Support query execution before reading DataFrame and before/after writing DataFrame over JDBC

AmplabJenkins removed a comment on pull request #28953: URL: https://github.com/apache/spark/pull/28953#issuecomment-675893267 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29453: [SPARK-31999][SQL][FOLLOWUP] Adds negative test cases with typos

AmplabJenkins commented on pull request #29453: URL: https://github.com/apache/spark/pull/29453#issuecomment-675893548 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28953: [SPARK-32013][SQL] Support query execution before reading DataFrame and before/after writing DataFrame over JDBC

SparkQA removed a comment on pull request #28953: URL: https://github.com/apache/spark/pull/28953#issuecomment-675844652 **[Test build #127620 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127620/testReport)** for PR 28953 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29469: [SPARK-28863][SQL] Introduce AlreadyPlanned to prevent reanalysis of V1FallbackWriters

AmplabJenkins removed a comment on pull request #29469: URL: https://github.com/apache/spark/pull/29469#issuecomment-675893281 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #29453: [SPARK-31999][SQL][FOLLOWUP] Adds negative test cases with typos

SparkQA removed a comment on pull request #29453: URL: https://github.com/apache/spark/pull/29453#issuecomment-675841543 **[Test build #127619 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127619/testReport)** for PR 29453 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29452: [SPARK-32643][CORE] Consolidate state decommissioning in the TaskSchedulerImpl realm

AmplabJenkins removed a comment on pull request #29452: URL: https://github.com/apache/spark/pull/29452#issuecomment-675893163 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #29437: [SPARK-32621][SQL] 'path' option can cause issues while inferring schema in CSV/JSON datasources

SparkQA removed a comment on pull request #29437: URL: https://github.com/apache/spark/pull/29437#issuecomment-675862392 **[Test build #127624 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127624/testReport)** for PR 29437 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29469: [SPARK-28863][SQL] Introduce AlreadyPlanned to prevent reanalysis of V1FallbackWriters

SparkQA removed a comment on pull request #29469: URL: https://github.com/apache/spark/pull/29469#issuecomment-675862342 **[Test build #127623 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127623/testReport)** for PR 29469 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29474: [SPARK-32658][CORE] Fix `PartitionWriterStream` partition length overflow

SparkQA removed a comment on pull request #29474: URL: https://github.com/apache/spark/pull/29474#issuecomment-675890659 **[Test build #127626 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127626/testReport)** for PR 29474 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29474: [SPARK-32658][CORE] Fix `PartitionWriterStream` partition length overflow

AmplabJenkins commented on pull request #29474: URL: https://github.com/apache/spark/pull/29474#issuecomment-675893141 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29437: [SPARK-32621][SQL] 'path' option can cause issues while inferring schema in CSV/JSON datasources

SparkQA commented on pull request #29437: URL: https://github.com/apache/spark/pull/29437#issuecomment-675893102 **[Test build #127624 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127624/testReport)** for PR 29437 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29460: [SPARK-32249][INFRA][3.0] Run Github Actions builds in branch-3.0

SparkQA removed a comment on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675829698 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29469: [SPARK-28863][SQL] Introduce AlreadyPlanned to prevent reanalysis of V1FallbackWriters

AmplabJenkins commented on pull request #29469: URL: https://github.com/apache/spark/pull/29469#issuecomment-675893281 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28953: [SPARK-32013][SQL] Support query execution before reading DataFrame and before/after writing DataFrame over JDBC

AmplabJenkins commented on pull request #28953: URL: https://github.com/apache/spark/pull/28953#issuecomment-675893267 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29465: [SPARK-32249][INFRA][2.4] Run Github Actions builds in branch-2.4

SparkQA commented on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675893100 **[Test build #127622 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127622/testReport)** for PR 29465 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29437: [SPARK-32621][SQL] 'path' option can cause issues while inferring schema in CSV/JSON datasources

AmplabJenkins commented on pull request #29437: URL: https://github.com/apache/spark/pull/29437#issuecomment-675893274 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29452: [SPARK-32643][CORE] Consolidate state decommissioning in the TaskSchedulerImpl realm

AmplabJenkins commented on pull request #29452: URL: https://github.com/apache/spark/pull/29452#issuecomment-675893163 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28953: [SPARK-32013][SQL] Support query execution before reading DataFrame and before/after writing DataFrame over JDBC

SparkQA commented on pull request #28953: URL: https://github.com/apache/spark/pull/28953#issuecomment-675893114 **[Test build #127620 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127620/testReport)** for PR 28953 at commit

[GitHub] [spark] SparkQA commented on pull request #29474: [SPARK-32658][CORE] Fix `PartitionWriterStream` partition length overflow

SparkQA commented on pull request #29474: URL: https://github.com/apache/spark/pull/29474#issuecomment-675893108 **[Test build #127626 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127626/testReport)** for PR 29474 at commit

[GitHub] [spark] SparkQA commented on pull request #29452: [SPARK-32643][CORE] Consolidate state decommissioning in the TaskSchedulerImpl realm

SparkQA commented on pull request #29452: URL: https://github.com/apache/spark/pull/29452#issuecomment-675893109 **[Test build #127627 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127627/testReport)** for PR 29452 at commit

[GitHub] [spark] SparkQA commented on pull request #29460: [SPARK-32249][INFRA][3.0] Run Github Actions builds in branch-3.0

SparkQA commented on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675893019 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] SparkQA commented on pull request #29453: [SPARK-31999][SQL][FOLLOWUP] Adds negative test cases with typos

SparkQA commented on pull request #29453: URL: https://github.com/apache/spark/pull/29453#issuecomment-675893110 **[Test build #127619 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127619/testReport)** for PR 29453 at commit

[GitHub] [spark] SparkQA commented on pull request #29469: [SPARK-28863][SQL] Introduce AlreadyPlanned to prevent reanalysis of V1FallbackWriters

SparkQA commented on pull request #29469: URL: https://github.com/apache/spark/pull/29469#issuecomment-675893101 **[Test build #127623 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127623/testReport)** for PR 29469 at commit

[GitHub] [spark] SparkQA commented on pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

SparkQA commented on pull request #29434: URL: https://github.com/apache/spark/pull/29434#issuecomment-675893104 **[Test build #127618 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127618/testReport)** for PR 29434 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29452: [SPARK-32643][CORE] Consolidate state decommissioning in the TaskSchedulerImpl realm

SparkQA removed a comment on pull request #29452: URL: https://github.com/apache/spark/pull/29452#issuecomment-675890700 **[Test build #127627 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127627/testReport)** for PR 29452 at commit

[GitHub] [spark] HyukjinKwon edited a comment on pull request #29460: [SPARK-32249][INFRA][3.0] Run Github Actions builds in branch-3.0

HyukjinKwon edited a comment on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675831293 Should be ready to be reviewed or go merging. Arrow related SparkR tests which will be fixed at #29462.

[GitHub] [spark] AmplabJenkins commented on pull request #29473: [SPARK-32656][SQL] Repartition bucketed tables for sort merge join / shuffled hash join if applicable

AmplabJenkins commented on pull request #29473: URL: https://github.com/apache/spark/pull/29473#issuecomment-675876095 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29473: [SPARK-32656][SQL] Repartition bucketed tables for sort merge join / shuffled hash join if applicable

AmplabJenkins removed a comment on pull request #29473: URL: https://github.com/apache/spark/pull/29473#issuecomment-675876095 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29468: [SPARK-32653][CORE] Decommissioned host/executor should be considered as inactive in TaskSchedulerImpl

agrawaldevesh commented on a change in pull request #29468: URL: https://github.com/apache/spark/pull/29468#discussion_r472762189 ## File path: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ## @@ -1062,25 +1062,36 @@ private[spark] class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29452: [SPARK-32643][CORE] Consolidate state decommissioning in the TaskSchedulerImpl realm

AmplabJenkins removed a comment on pull request #29452: URL: https://github.com/apache/spark/pull/29452#issuecomment-675881350 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] cloud-fan commented on a change in pull request #29452: [SPARK-32643][CORE] Consolidate state decommissioning in the TaskSchedulerImpl realm

cloud-fan commented on a change in pull request #29452: URL: https://github.com/apache/spark/pull/29452#discussion_r472769129 ## File path: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala ## @@ -1051,15 +1049,21 @@ private[spark] class TaskSetManager(

[GitHub] [spark] cloud-fan closed pull request #29466: [SPARK-32651][CORE] Decommission switch configuration should have the highest hierarchy

cloud-fan closed pull request #29466: URL: https://github.com/apache/spark/pull/29466 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] cloud-fan commented on pull request #29466: [SPARK-32651][CORE] Decommission switch configuration should have the highest hierarchy

cloud-fan commented on pull request #29466: URL: https://github.com/apache/spark/pull/29466#issuecomment-675888359 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29455: [SPARK-32644][SQL] NAAJ support for ShuffleHashJoin when AQE is on

agrawaldevesh commented on a change in pull request #29455: URL: https://github.com/apache/spark/pull/29455#discussion_r472772355 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ShuffleExchangeExec.scala ## @@ -83,15 +83,18 @@ trait

[GitHub] [spark] jiangxb1987 commented on pull request #29474: [SPARK-32658][CORE] Fix `PartitionWriterStream` partition length overflow

jiangxb1987 commented on pull request #29474: URL: https://github.com/apache/spark/pull/29474#issuecomment-675889531 cc @mccheah @cloud-fan @Ngone51 @zhengruifeng This is an automated message from the Apache Git Service. To

[GitHub] [spark] HyukjinKwon commented on pull request #29456: [SPARK-32647][INFRA] Report SparkR test results with JUnit reporter

HyukjinKwon commented on pull request #29456: URL: https://github.com/apache/spark/pull/29456#issuecomment-675874159 I merged into branch-3.0 as well. cc @zhengruifeng. this is test-only so should be fine. This is an

[GitHub] [spark] SparkQA removed a comment on pull request #29473: [SPARK-32656][SQL] Repartition bucketed tables for sort merge join / shuffled hash join if applicable

SparkQA removed a comment on pull request #29473: URL: https://github.com/apache/spark/pull/29473#issuecomment-675797404 **[Test build #127609 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127609/testReport)** for PR 29473 at commit

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29452: [SPARK-32643][CORE] Consolidate state decommissioning in the TaskSchedulerImpl realm

agrawaldevesh commented on a change in pull request #29452: URL: https://github.com/apache/spark/pull/29452#discussion_r472756046 ## File path: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala ## @@ -1051,15 +1049,19 @@ private[spark] class TaskSetManager(

[GitHub] [spark] SparkQA commented on pull request #29452: [SPARK-32643][CORE] Consolidate state decommissioning in the TaskSchedulerImpl realm

SparkQA commented on pull request #29452: URL: https://github.com/apache/spark/pull/29452#issuecomment-675881324 **[Test build #127625 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127625/testReport)** for PR 29452 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29452: [SPARK-32643][CORE] Consolidate state decommissioning in the TaskSchedulerImpl realm

AmplabJenkins removed a comment on pull request #29452: URL: https://github.com/apache/spark/pull/29452#issuecomment-675881340 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] agrawaldevesh commented on a change in pull request #29468: [SPARK-32653][CORE] Decommissioned host/executor should be considered as inactive in TaskSchedulerImpl

agrawaldevesh commented on a change in pull request #29468: URL: https://github.com/apache/spark/pull/29468#discussion_r472760587 ## File path: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ## @@ -1062,25 +1062,36 @@ private[spark] class

[GitHub] [spark] AmplabJenkins commented on pull request #29452: [SPARK-32643][CORE] Consolidate state decommissioning in the TaskSchedulerImpl realm

AmplabJenkins commented on pull request #29452: URL: https://github.com/apache/spark/pull/29452#issuecomment-675881340 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29452: [SPARK-32643][CORE] Consolidate state decommissioning in the TaskSchedulerImpl realm

SparkQA removed a comment on pull request #29452: URL: https://github.com/apache/spark/pull/29452#issuecomment-675880439 **[Test build #127625 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127625/testReport)** for PR 29452 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29452: [SPARK-32643][CORE] Consolidate state decommissioning in the TaskSchedulerImpl realm

AmplabJenkins removed a comment on pull request #29452: URL: https://github.com/apache/spark/pull/29452#issuecomment-675880787 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29452: [SPARK-32643][CORE] Consolidate state decommissioning in the TaskSchedulerImpl realm

AmplabJenkins commented on pull request #29452: URL: https://github.com/apache/spark/pull/29452#issuecomment-675880787 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] jiangxb1987 opened a new pull request #29474: [SPARK-32658][CORE] Fix `PartitionWriterStream` partition length overflow

jiangxb1987 opened a new pull request #29474: URL: https://github.com/apache/spark/pull/29474 # What changes were proposed in this pull request? The `count` in `PartitionWriterStream` should be a long value, instead of int. The issue is introduced by apache/spark@abef84a . When the

[GitHub] [spark] AmplabJenkins commented on pull request #29452: [SPARK-32643][CORE] Consolidate state decommissioning in the TaskSchedulerImpl realm

AmplabJenkins commented on pull request #29452: URL: https://github.com/apache/spark/pull/29452#issuecomment-675891086 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29474: [SPARK-32658][CORE] Fix `PartitionWriterStream` partition length overflow

AmplabJenkins commented on pull request #29474: URL: https://github.com/apache/spark/pull/29474#issuecomment-675891005 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29452: [SPARK-32643][CORE] Consolidate state decommissioning in the TaskSchedulerImpl realm

AmplabJenkins removed a comment on pull request #29452: URL: https://github.com/apache/spark/pull/29452#issuecomment-675891086 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29474: [SPARK-32658][CORE] Fix `PartitionWriterStream` partition length overflow

AmplabJenkins removed a comment on pull request #29474: URL: https://github.com/apache/spark/pull/29474#issuecomment-675891005 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29474: [SPARK-32658][CORE] Fix `PartitionWriterStream` partition length overflow

SparkQA commented on pull request #29474: URL: https://github.com/apache/spark/pull/29474#issuecomment-675890659 **[Test build #127626 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127626/testReport)** for PR 29474 at commit

[GitHub] [spark] SparkQA commented on pull request #29452: [SPARK-32643][CORE] Consolidate state decommissioning in the TaskSchedulerImpl realm

SparkQA commented on pull request #29452: URL: https://github.com/apache/spark/pull/29452#issuecomment-675890700 **[Test build #127627 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127627/testReport)** for PR 29452 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

AmplabJenkins commented on pull request #29434: URL: https://github.com/apache/spark/pull/29434#issuecomment-675893688 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

AmplabJenkins removed a comment on pull request #29434: URL: https://github.com/apache/spark/pull/29434#issuecomment-675893688 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28953: [SPARK-32013][SQL] Support query execution before reading DataFrame and before/after writing DataFrame over JDBC

AmplabJenkins removed a comment on pull request #28953: URL: https://github.com/apache/spark/pull/28953#issuecomment-675893272 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #29460: [SPARK-32249][INFRA][3.0] Run Github Actions builds in branch-3.0

AmplabJenkins commented on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675893985 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29465: [SPARK-32249][INFRA][2.4] Run Github Actions builds in branch-2.4

AmplabJenkins removed a comment on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675893553 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29474: [SPARK-32658][CORE] Fix `PartitionWriterStream` partition length overflow

AmplabJenkins removed a comment on pull request #29474: URL: https://github.com/apache/spark/pull/29474#issuecomment-675893147 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29469: [SPARK-28863][SQL] Introduce AlreadyPlanned to prevent reanalysis of V1FallbackWriters

AmplabJenkins removed a comment on pull request #29469: URL: https://github.com/apache/spark/pull/29469#issuecomment-675893286 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29437: [SPARK-32621][SQL] 'path' option can cause issues while inferring schema in CSV/JSON datasources

AmplabJenkins removed a comment on pull request #29437: URL: https://github.com/apache/spark/pull/29437#issuecomment-675893283 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29460: [SPARK-32249][INFRA][3.0] Run Github Actions builds in branch-3.0

AmplabJenkins removed a comment on pull request #29460: URL: https://github.com/apache/spark/pull/29460#issuecomment-675893985 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan commented on a change in pull request #29468: [SPARK-32653][CORE] Decommissioned host/executor should be considered as inactive in TaskSchedulerImpl

cloud-fan commented on a change in pull request #29468: URL: https://github.com/apache/spark/pull/29468#discussion_r472786561 ## File path: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ## @@ -1062,25 +1062,36 @@ private[spark] class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29465: [SPARK-32249][INFRA][2.4] Run Github Actions builds in branch-2.4

AmplabJenkins removed a comment on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675893566 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29453: [SPARK-31999][SQL][FOLLOWUP] Adds negative test cases with typos

AmplabJenkins removed a comment on pull request #29453: URL: https://github.com/apache/spark/pull/29453#issuecomment-675893555 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

AmplabJenkins removed a comment on pull request #29434: URL: https://github.com/apache/spark/pull/29434#issuecomment-675893692 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] cloud-fan closed pull request #29418: [SPARK-32600][CORE] Unify task name in some logs between driver and executor

cloud-fan closed pull request #29418: URL: https://github.com/apache/spark/pull/29418 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] cloud-fan commented on pull request #29418: [SPARK-32600][CORE] Unify task name in some logs between driver and executor

cloud-fan commented on pull request #29418: URL: https://github.com/apache/spark/pull/29418#issuecomment-675962204 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] LuciferYang commented on a change in pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

LuciferYang commented on a change in pull request #29434: URL: https://github.com/apache/spark/pull/29434#discussion_r472861979 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/StarJoinCostBasedReorderSuite.scala ## @@ -329,7 +329,7 @@ class

[GitHub] [spark] wangyum opened a new pull request #29475: [SPARK-32659][SQL] Replace Array with Set in InSubqueryExec

wangyum opened a new pull request #29475: URL: https://github.com/apache/spark/pull/29475 ### What changes were proposed in this pull request? Replace `Array` with `Set` in `InSubqueryExec`. ### Why are the changes needed? Set has better performance:

[GitHub] [spark] Ngone51 commented on pull request #29418: [SPARK-32600][CORE] Unify task name in some logs between driver and executor

Ngone51 commented on pull request #29418: URL: https://github.com/apache/spark/pull/29418#issuecomment-675971368 thanks all!! This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29475: [SPARK-32659][SQL] Replace Array with Set in InSubqueryExec

SparkQA commented on pull request #29475: URL: https://github.com/apache/spark/pull/29475#issuecomment-675978958 **[Test build #127637 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127637/testReport)** for PR 29475 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29475: [SPARK-32659][SQL] Replace Array with Set in InSubqueryExec

AmplabJenkins commented on pull request #29475: URL: https://github.com/apache/spark/pull/29475#issuecomment-675980722 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29475: [SPARK-32659][SQL] Replace Array with Set in InSubqueryExec

AmplabJenkins removed a comment on pull request #29475: URL: https://github.com/apache/spark/pull/29475#issuecomment-675980722 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] LuciferYang commented on a change in pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

LuciferYang commented on a change in pull request #29434: URL: https://github.com/apache/spark/pull/29434#discussion_r472861979 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/StarJoinCostBasedReorderSuite.scala ## @@ -329,7 +329,7 @@ class

[GitHub] [spark] LuciferYang commented on a change in pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

LuciferYang commented on a change in pull request #29434: URL: https://github.com/apache/spark/pull/29434#discussion_r472861979 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/StarJoinCostBasedReorderSuite.scala ## @@ -329,7 +329,7 @@ class

[GitHub] [spark] cloud-fan commented on a change in pull request #29438: [SPARK-32607][SQL] Script Transformation ROW FORMAT DELIMITED `TOK_TABLEROWFORMATLINES` only support '\n'

cloud-fan commented on a change in pull request #29438: URL: https://github.com/apache/spark/pull/29438#discussion_r472878919 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/SparkSqlParserSuite.scala ## @@ -330,4 +330,42 @@ class SparkSqlParserSuite

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29438: [SPARK-32607][SQL] Script Transformation ROW FORMAT DELIMITED `TOK_TABLEROWFORMATLINES` only support '\n'

AngersZh commented on a change in pull request #29438: URL: https://github.com/apache/spark/pull/29438#discussion_r472881479 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/SparkSqlParserSuite.scala ## @@ -330,4 +330,42 @@ class SparkSqlParserSuite

[GitHub] [spark] cloud-fan commented on a change in pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

cloud-fan commented on a change in pull request #29434: URL: https://github.com/apache/spark/pull/29434#discussion_r472881899 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/StarJoinCostBasedReorderSuite.scala ## @@ -329,7 +329,7 @@ class

[GitHub] [spark] cloud-fan commented on pull request #29438: [SPARK-32607][SQL] Script Transformation ROW FORMAT DELIMITED `TOK_TABLEROWFORMATLINES` only support '\n'

cloud-fan commented on pull request #29438: URL: https://github.com/apache/spark/pull/29438#issuecomment-676005317 the fix looks good, can you resolve conflicts? This is an automated message from the Apache Git Service. To

[GitHub] [spark] HyukjinKwon commented on pull request #29465: [SPARK-32249][INFRA][2.4] Run Github Actions builds in branch-2.4

HyukjinKwon commented on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-676006546 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] HyukjinKwon removed a comment on pull request #29465: [SPARK-32249][INFRA][2.4] Run Github Actions builds in branch-2.4

HyukjinKwon removed a comment on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-675905000 retest this please This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA commented on pull request #29465: [SPARK-32249][INFRA][2.4] Run Github Actions builds in branch-2.4

SparkQA commented on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-676008617 **[Test build #127638 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127638/testReport)** for PR 29465 at commit

[GitHub] [spark] LuciferYang commented on a change in pull request #29434: [SPARK-32526][SQL] Pass all test of sql/catalyst module in Scala 2.13

LuciferYang commented on a change in pull request #29434: URL: https://github.com/apache/spark/pull/29434#discussion_r472885119 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/StarJoinCostBasedReorderSuite.scala ## @@ -329,7 +329,7 @@ class

[GitHub] [spark] AmplabJenkins commented on pull request #29465: [SPARK-32249][INFRA][2.4] Run Github Actions builds in branch-2.4

AmplabJenkins commented on pull request #29465: URL: https://github.com/apache/spark/pull/29465#issuecomment-676010471 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] cloud-fan commented on pull request #29421: [SPARK-32388][SQL] TRANSFORM with schema-less mode should keep the same with hive

cloud-fan commented on pull request #29421: URL: https://github.com/apache/spark/pull/29421#issuecomment-676012804 can you list the behaviors of both w/ serde and w/o serde? We need to understand why we only need to fix the w/ serde case.

  1   2   3   4   5   6   7   >