Ruslan Dautkhanov created SPARK-25814:
-----------------------------------------
Summary: spark driver runs out of memory on
org.apache.spark.util.kvstore.InMemoryStore
Key: SPARK-25814
URL: https://issues.apache.org/jira/browse/SPARK-25814
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 2.3.2, 2.2.2
Reporter: Ruslan Dautkhanov
Attachments: image-2018-10-23-14-06-53-722.png
We're looking into issue when even huge spark driver memory gets eventually
exhausted and GC makes driver stop responding.
Used jxray.com tool and found that most of driver heap is used by
{noformat}
org.apache.spark.status.AppStatusStore
-> org.apache.spark.status.ElementTrackingStore
-> org.apache.spark.util.kvstore.InMemoryStore
{noformat}
Is there is a way to tune this particular spark driver's memory region down?
!image-2018-10-23-14-03-12-258.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
