[GitHub] [spark] LuciferYang commented on a diff in pull request #41487: [SPARK-43648][CONNECT][TESTS] Move `interrupt all` related test to a new test file to pass Maven test

2023-06-06 Thread via GitHub
LuciferYang commented on code in PR #41487: URL: https://github.com/apache/spark/pull/41487#discussion_r1220904149 ## connector/connect/client/jvm/src/test/scala/org/apache/spark/sql/SparkSessionE2ESuite.scala: ## @@ -0,0 +1,94 @@ +/* + * Licensed to the Apache Software Foundati

[GitHub] [spark] LuciferYang commented on pull request #41253: [SPARK-43988][INFRA] Add maven testing GitHub Action task for connect client module

2023-06-06 Thread via GitHub
LuciferYang commented on PR #41253: URL: https://github.com/apache/spark/pull/41253#issuecomment-1579978443 wait https://github.com/apache/spark/pull/41487 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] shuwang21 opened a new pull request, #41489: [SPARK-43987][Shuffle] Separate finalizeShuffleMerge Processing to Dedicated Thread Pools

2023-06-06 Thread via GitHub
shuwang21 opened a new pull request, #41489: URL: https://github.com/apache/spark/pull/41489 ### What changes were proposed in this pull request? In this PR, we proposed to separate finalizeShuffleMerge processing into dedicated thread pools. 1. We introduce `ShuffleTranspo

[GitHub] [spark] LuciferYang closed pull request #41264: [SPARK-43717][CONNECT] Scala client reduce agg cannot handle null partitions for scala primitive inputs

2023-06-06 Thread via GitHub
LuciferYang closed pull request #41264: [SPARK-43717][CONNECT] Scala client reduce agg cannot handle null partitions for scala primitive inputs URL: https://github.com/apache/spark/pull/41264 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] LuciferYang commented on pull request #41264: [SPARK-43717][CONNECT] Scala client reduce agg cannot handle null partitions for scala primitive inputs

2023-06-06 Thread via GitHub
LuciferYang commented on PR #41264: URL: https://github.com/apache/spark/pull/41264#issuecomment-1579984673 Merged into master. Thanks @zhenlineo @cloud-fan @hvanhovell @amaliujia -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] cloud-fan commented on pull request #41475: [SPARK-43979][SQL] CollectedMetrics should be treated as the same one for self-join

2023-06-06 Thread via GitHub
cloud-fan commented on PR #41475: URL: https://github.com/apache/spark/pull/41475#issuecomment-1580004404 The GA is known to be flaky locally, and I verified that the new test passes locally. I'm merging it to master. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] cloud-fan closed pull request #41475: [SPARK-43979][SQL] CollectedMetrics should be treated as the same one for self-join

2023-06-06 Thread via GitHub
cloud-fan closed pull request #41475: [SPARK-43979][SQL] CollectedMetrics should be treated as the same one for self-join URL: https://github.com/apache/spark/pull/41475 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [spark] pengzhon-db commented on pull request #41318: [SPARK-43803] [SS] [CONNECT] Improve awaitTermination() to handle client disconnects

2023-06-06 Thread via GitHub
pengzhon-db commented on PR #41318: URL: https://github.com/apache/spark/pull/41318#issuecomment-1580008365 @HyukjinKwon Could u review this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [spark] shuwang21 commented on pull request #41489: [SPARK-43987][Shuffle] Separate finalizeShuffleMerge Processing to Dedicated Thread Pools

2023-06-06 Thread via GitHub
shuwang21 commented on PR #41489: URL: https://github.com/apache/spark/pull/41489#issuecomment-1580011361 cc: @otterc @mridulm @zhouyejoe Please take a look. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] cloud-fan commented on a diff in pull request #41348: [SPARK-43203][SQL] Move all Drop Table case to DataSource V2

2023-06-06 Thread via GitHub
cloud-fan commented on code in PR #41348: URL: https://github.com/apache/spark/pull/41348#discussion_r1220983601 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala: ## @@ -849,6 +849,7 @@ class SessionCatalog( tempViews.remove(tabl

<    1   2   3