Jungtaek Lim created SPARK-29301:
------------------------------------
Summary: Removing block is not reflected to the driver/executor's
storage memory
Key: SPARK-29301
URL: https://issues.apache.org/jira/browse/SPARK-29301
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 2.4.4, 3.0.0
Reporter: Jungtaek Lim
While investigating SPARK-29055 I've found that heap memory in driver doesn't
increase which would denote memory leak happen, but storage memory doesn't
decrease even there're some broadcast blocks getting removed - these values
keep increasing.
Please refer SPARK-29055 to see the step to reproduce. Very easy to see it, as
running simple query repeatedly would reproduce the issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
