[
https://issues.apache.org/jira/browse/AURORA-1837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15746961#comment-15746961
]
Mehrdad Nurolahzade edited comment on AURORA-1837 at 2/11/17 7:59 PM:
----------------------------------------------------------------------
-This becomes a problem on a scheduler that has been down longer than
{{HISTORY_PRUNE_THRESHOLD}} and an attempt is made to relaunch it. If the
scheduler was used for load testing, for example, all/many tasks will be dead
at relaunch time.-
@StephanErb's suggestion above should improve performance. However we need to
come up with a design around breaking apart instance-based and job-based
pruning.
was (Author: mnurolahzade):
This becomes a problem on a scheduler that has been down longer than
{{HISTORY_PRUNE_THRESHOLD}} and an attempt is made to relaunch it. If the
scheduler was used for load testing, for example, all/many tasks will be dead
at relaunch time.
@StephanErb's suggestion above should improve performance. However we need to
come up with a design around breaking apart instance-based and job-based
pruning.
> Improve task history pruning
> ----------------------------
>
> Key: AURORA-1837
> URL: https://issues.apache.org/jira/browse/AURORA-1837
> Project: Aurora
> Issue Type: Task
> Reporter: Reza Motamedi
> Assignee: Mehrdad Nurolahzade
> Priority: Minor
> Labels: scheduler
>
> Current implementation of {{TaskHistoryPrunner}} registers all inactive tasks
> upon terminal _state_ change for pruning.
> {{TaskHistoryPrunner::registerInactiveTask()}} uses a delay executor to
> schedule the process of pruning _task_s. However, we have noticed most of
> pruning takes place after scheduler recovers from a fail-over.
> Modify {{TaskHistoryPruner}} to a design similar to
> {{JobUpdateHistoryPruner}}:
> # Instead of registering delay executor's upon terminal task state
> transitions, have it wake up on preconfigured intervals, find all terminal
> state tasks that meet pruning criteria and delete them.
> # Make the initial task history pruning delay configurable so that it does
> not hamper scheduler upon start.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
