JkSelf opened a new pull request #25308: [SPARK-28576][SQL] fix the dead lock issue when enable new adaptive execution URL: https://github.com/apache/spark/pull/25308 ## What changes were proposed in this pull request? In our local test 1TB TPC-DS, Q6 is hang when enable new AE. The issue is same with [PR#80](https://github.com/Intel-bigdata/spark-adaptive/pull/80). The root cause is that when execute the subquery the [synchronized](https://github.com/apache/spark/blob/a745381b9d3dd290057ef3089de7fdb9264f1f8b/sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlan.scala#L245) and [canonicalized lazy operator](https://github.com/apache/spark/blob/a745381b9d3dd290057ef3089de7fdb9264f1f8b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala#L213) are all lock the same FilterExec. And this PR change the synchronized to ReentrantLock to fix the hang issue. ## How was this patch tested? Manual test Q6 1TB TPC-DS
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
