David C Navas created SPARK-27726:
-------------------------------------
Summary: Performance of InMemoryStore suffers under load
Key: SPARK-27726
URL: https://issues.apache.org/jira/browse/SPARK-27726
Project: Spark
Issue Type: Umbrella
Components: Spark Core
Affects Versions: 2.3.3
Reporter: David C Navas
When our Spark system has been under load for an extended period of time, GC
remains highly active and the jobs page becomes unresponsive even when load
eases. Please see the attached GCRateIssues for more details regarding the
problem definition.
We found a number of separate issues which are detailed in the subtasks. I
anticipate committing a single PR for all subtasks whose commits roughly align
with the descriptions in the subtasks.
The performance of the code is measured before and after the change and is
attached in the document PerformanceBeforeAndAfter. tl;dr in our use case, we
saw about five (computed) orders of magnitude improvement.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
