[
https://issues.apache.org/jira/browse/SPARK-5210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14633548#comment-14633548
]
Tao Wang commented on SPARK-5210:
---------------------------------
Except the OOM caused by replaying the entire log, do you have other concerns
about long-running application's event log?
Any better way to replace log rolling? Ićcouldn't think of one now :(
> Support log rolling in EventLogger
> ----------------------------------
>
> Key: SPARK-5210
> URL: https://issues.apache.org/jira/browse/SPARK-5210
> Project: Spark
> Issue Type: New Feature
> Components: Spark Core, Web UI
> Reporter: Josh Rosen
>
> For long-running Spark applications (e.g. running for days / weeks), the
> Spark event log may grow to be very large.
> As a result, it would be useful if EventLoggingListener supported log file
> rolling / rotation. Adding this feature will involve changes to the
> HistoryServer in order to be able to load event logs from a sequence of files
> instead of a single file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
