[
https://issues.apache.org/jira/browse/SPARK-20582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15996013#comment-15996013
]
zhoukang edited comment on SPARK-20582 at 5/4/17 1:44 AM:
----------------------------------------------------------
You are right [~vanzin],i have looked into you implementation.Thanks for your
time.
was (Author: cane):
You are right [~vanzin],i searched into you implementation.Thanks for your time.
> Speed up the restart of HistoryServer using ApplicationAttemptInfo
> checkpointing
> --------------------------------------------------------------------------------
>
> Key: SPARK-20582
> URL: https://issues.apache.org/jira/browse/SPARK-20582
> Project: Spark
> Issue Type: Improvement
> Components: Deploy
> Affects Versions: 2.1.0
> Reporter: zhoukang
> Priority: Critical
>
> In the current code of HistoryServer,jetty server will be started
> after all the logs fetch from yarn has been replayed.However,when the number
> of logs is becoming larger,the start time of jetty will be too long.
> Here,we implement a solution which using ApplicationAttemptInfo
> checkpointing to speed up the start of historyserver.When historyserver is
> starting,it will load ApplicationAttemptInfo from checkpoint file first if
> exists which is faster then replaying one by one.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
