Jungtaek Lim created SPARK-28870:
------------------------------------
Summary: Snapshot old event log files to support compaction
Key: SPARK-28870
URL: https://issues.apache.org/jira/browse/SPARK-28870
Project: Spark
Issue Type: Sub-task
Components: Spark Core
Affects Versions: 3.0.0
Reporter: Jungtaek Lim
This issue tracks the effort on compacting old event log files into snapshot
and achieve both goals, 1) reduce overall event log file size 2) speed up
replaying event logs. It also deals with cleaning event log files as snapshot
will provide the safe way to clean up old event log files without losing
ability to replay whole event logs.
This issue will be on top of SPARK-28869 as SPARK-28869 will create rolled
event log files.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
