[GitHub] [spark] JkSelf commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution

JkSelf commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution URL: https://github.com/apache/spark/pull/25308#issuecomment-517590838 @cloud-fan I rebased master to commit(4e7a4cd20e2f6e19b45fee6bfd24071ce4c88dd8). And when I run your test

[GitHub] [spark] JkSelf commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution

JkSelf commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution URL: https://github.com/apache/spark/pull/25308#issuecomment-517552574 @cloud-fan Got it, thanks. But it seems the main query does not update the subquery in my test with [PR#25

[GitHub] [spark] JkSelf commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution

JkSelf commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution URL: https://github.com/apache/spark/pull/25308#issuecomment-517547021 @cloud-fan @maryannxue Thanks for your detailed explanations. And I am curious about when the subquery thre

[GitHub] [spark] JkSelf commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution

JkSelf commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution URL: https://github.com/apache/spark/pull/25308#issuecomment-517251794 @cloud-fan yes, the metrics of subquery is missing. And [PR#25316](https://github.com/apache/spark/pull/253

[GitHub] [spark] JkSelf commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution

JkSelf commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution URL: https://github.com/apache/spark/pull/25308#issuecomment-517188190 @maryannxue I still have a question why the main thread execute the [e.canonicalized](https://github.com

[GitHub] [spark] JkSelf commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution

JkSelf commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution URL: https://github.com/apache/spark/pull/25308#issuecomment-517175518 @maryannxue I have verified [PR#25316](https://github.com/apache/spark/pull/25316) and it can fix the dead

[GitHub] [spark] JkSelf commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution

JkSelf commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution URL: https://github.com/apache/spark/pull/25308#issuecomment-517155965 @cloud-fan the dead lock is as below from the jstack.log. Broadcast-exchange-0 thread lock the FilterExe

[GitHub] [spark] JkSelf commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution

JkSelf commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution URL: https://github.com/apache/spark/pull/25308#issuecomment-516722956 Broadcast-exchange-0 thread lock the FilterExec object(0x0006c2098e08) and then execute the subquer

[GitHub] [spark] JkSelf commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution

JkSelf commented on issue #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution URL: https://github.com/apache/spark/pull/25308#issuecomment-516707537 @maryannxue @carsonwang Please help me review If you have available time. Thanks for your help. -