[
https://issues.apache.org/jira/browse/YARN-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13935128#comment-13935128
]
Billie Rinaldi commented on YARN-1717:
--------------------------------------
bq. We do deletion according to entity's TS and at the entity's granularity,
thus, the events that are still alive are likely to be deleted as well.
I believe this is the desired behavior. For example, in the case where we have
a job entity that starts several shorter-lived task entities, we would not want
to remove task entities before the job entity is removed. With the current
behavior, the job entity would be removed at the same time or earlier than the
task entities.
We don't yet have a good understanding of how applications with long-lived
entities would want to use the timeline store, so it's hard to design for them.
Perhaps an option for the future would be to have a configurable deletion
strategy, if some applications have different requirements.
> Enable offline deletion of entries in leveldb timeline store
> ------------------------------------------------------------
>
> Key: YARN-1717
> URL: https://issues.apache.org/jira/browse/YARN-1717
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Billie Rinaldi
> Assignee: Billie Rinaldi
> Attachments: YARN-1717.1.patch, YARN-1717.10.patch,
> YARN-1717.11.patch, YARN-1717.2.patch, YARN-1717.3.patch, YARN-1717.4.patch,
> YARN-1717.5.patch, YARN-1717.6-extra.patch, YARN-1717.6.patch,
> YARN-1717.7.patch, YARN-1717.8.patch, YARN-1717.9.patch
>
>
> The leveldb timeline store implementation needs the following:
> * better documentation of its internal structures
> * internal changes to enable deleting entities
> ** never overwrite existing primary filter entries
> ** add hidden reverse pointers to related entities
--
This message was sent by Atlassian JIRA
(v6.2#6252)
