[
https://issues.apache.org/jira/browse/SPARK-20582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996013#comment-15996013
]
zhoukang commented on SPARK-20582:
----------------------------------
You are right [~vanzin],i searched into you implementation.Thanks for your time.
> Speed up the restart of HistoryServer using ApplicationAttemptInfo
> checkpointing
> --------------------------------------------------------------------------------
>
> Key: SPARK-20582
> URL: https://issues.apache.org/jira/browse/SPARK-20582
> Project: Spark
> Issue Type: Improvement
> Components: Deploy
> Affects Versions: 2.1.0
> Reporter: zhoukang
> Priority: Critical
>
> In the current code of HistoryServer,jetty server will be started
> after all the logs fetch from yarn has been replayed.However,when the number
> of logs is becoming larger,the start time of jetty will be too long.
> Here,we implement a solution which using ApplicationAttemptInfo
> checkpointing to speed up the start of historyserver.When historyserver is
> starting,it will load ApplicationAttemptInfo from checkpoint file first if
> exists which is faster then replaying one by one.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
