[
https://issues.apache.org/jira/browse/SPARK-54620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-54620.
-----------------------------------
Fix Version/s: 4.1.0
Resolution: Fixed
Issue resolved by pull request 53358
[https://github.com/apache/spark/pull/53358]
> Add safety check in ObservationManager to avoid Observation blocking
> --------------------------------------------------------------------
>
> Key: SPARK-54620
> URL: https://issues.apache.org/jira/browse/SPARK-54620
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.0.1, 3.5.7
> Reporter: L. C. Hsieh
> Assignee: L. C. Hsieh
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.1.0
>
>
> We got reports that for some corner cases Observation.get will be blocked
> forever. It is not deadlock case after investigation. If the
> CollectMetricsExec operator was optimized away, e.g., the executed plan was
> optimized to have some empty relation propagation on top of plan tree of
> CollectMetricsExec, Spark won't fulfill the promise in Observation and get
> calls will be blocked forever.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
