L. C. Hsieh created SPARK-54620:
-----------------------------------
Summary: Add safety check in ObservationManager to avoid
Observation blocking
Key: SPARK-54620
URL: https://issues.apache.org/jira/browse/SPARK-54620
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.5.7, 4.0.1
Reporter: L. C. Hsieh
We got reports that for some corner cases Observation.get will be blocked
forever. It is not deadlock case after investigation. If the CollectMetricsExec
operator was optimized away, e.g., the executed plan was optimized to have some
empty relation propagation on top of plan tree of CollectMetricsExec, Spark
won't fulfill the promise in Observation and get calls will be blocked forever.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
