Xi Lyu created SPARK-47819:
------------------------------
Summary: Use asynchronous callback for execution cleanup
Key: SPARK-47819
URL: https://issues.apache.org/jira/browse/SPARK-47819
Project: Spark
Issue Type: Improvement
Components: Connect
Affects Versions: 4.0.0
Reporter: Xi Lyu
Fix For: 4.0.0
Expired sessions are regularly checked and cleaned up by a maintenance thread.
However, currently, this process is synchronous. Therefore, in occasional
cases, interrupting the execution thread of a query in a session can take
hours, causing the entire maintenance process to stall, resulting in a large
amount of memory not being cleared.
We address this by introducing asynchronous callbacks for execution cleanup,
avoiding synchronous joins of execution threads, and preventing the maintenance
thread from stalling in the above occasional scenarios.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
