[
https://issues.apache.org/jira/browse/SPARK-6751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves resolved SPARK-6751.
----------------------------------
Resolution: Duplicate
Dup of SPARK-4705
> Spark History Server support multiple application attempts
> ----------------------------------------------------------
>
> Key: SPARK-6751
> URL: https://issues.apache.org/jira/browse/SPARK-6751
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core, YARN
> Affects Versions: 1.3.0
> Reporter: Thomas Graves
>
> Spark on Yarn supports running multiple application attempts (configurable
> number) in case the first (or second..) attempts fail. The Spark History
> server only supports one history file though. Under the default configs it
> keeps the first attempts history file. You can set the undocumented config
> spark.eventLog.overwrite to allow the follow on attempts to overwrite the
> first attempts history file.
> Note that in spark 1.2 not having the overwrite config set causes any
> following attempts to actually fail to run, in spark 1.3 they run and you
> just see a warning at the end of the attempts.
> It would be really nice to have an option that keeps all the attempts history
> files. This way a user can go back and look at each one individually.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
