Shixiong Zhu created SPARK-30927:
------------------------------------
Summary: StreamingQueryManager should avoid keeping reference to
terminated StreamingQuery
Key: SPARK-30927
URL: https://issues.apache.org/jira/browse/SPARK-30927
Project: Spark
Issue Type: Improvement
Components: Structured Streaming
Affects Versions: 3.0.0
Reporter: Shixiong Zhu
Right now StreamingQueryManager will keep the last terminated query until
"resetTerminated" is called. When the last terminated query has lots of states
(a large sql plan, cached RDDs, etc.), it will waste these memory unnecessarily.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
