Ziqi Liu created SPARK-48610:
--------------------------------
Summary: Remove ExplainUtils.processPlan synchronize
Key: SPARK-48610
URL: https://issues.apache.org/jira/browse/SPARK-48610
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 4.0.0
Reporter: Ziqi Liu
[https://github.com/apache/spark/pull/45282] introduced synchronize to
`ExplainUtils.processPlan` to avoid race condition when multiple queries
refers to same cached plan.
The granularity of lock is too large. We can try to fix the root cause of this
concurrency issue by refactoring the usage of mutable OP_ID_TAG, which is not a
good practice in terms of immutable nature of SparkPlan. Instead, we can use
an auxiliary id map, with object identity as the key.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
