[
https://issues.apache.org/jira/browse/YARN-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13932437#comment-13932437
]
Zhijie Shen commented on YARN-1717:
-----------------------------------
After the aging mechanism works, there may be some use cases that are affected.
I can come up with two:
1. Say I have long running service, and it records all the containers that it
has launched as an "entity". Since the service is running for such a long time
that the deletion thread has already discarded the early finished containers'
entity data. Then, if users want to query by giving entity type "Container",
the old containers will no longer show up, and users may be misled by
considering the given list has all containers.
2. Another case is related, but is a bit different. Given the service set all
its containers is related to the service entity. If I query the entity, I'm
supposed to get all the containers in the related entity section. However, if
some containers have been discarded, the section will in fact have an
incomplete list.
To some extent, it makes sense to discard entity at some granularity. For
example, w.r.t the generic history information, it should be fine if I delete
the history file which contains all the information about an application,
including those of attempts and containers. However, if we only delete parts,
say containers, users will be confused that the application haven't launched
any containers.
I'm just thinking it out loudly, in case the aging mechanism will affect some
usage of timeline service.
> Enable offline deletion of entries in leveldb timeline store
> ------------------------------------------------------------
>
> Key: YARN-1717
> URL: https://issues.apache.org/jira/browse/YARN-1717
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Billie Rinaldi
> Assignee: Billie Rinaldi
> Attachments: YARN-1717.1.patch, YARN-1717.2.patch, YARN-1717.3.patch,
> YARN-1717.4.patch, YARN-1717.5.patch, YARN-1717.6-extra.patch,
> YARN-1717.6.patch, YARN-1717.7.patch, YARN-1717.8.patch, YARN-1717.9.patch
>
>
> The leveldb timeline store implementation needs the following:
> * better documentation of its internal structures
> * internal changes to enable deleting entities
> ** never overwrite existing primary filter entries
> ** add hidden reverse pointers to related entities
--
This message was sent by Atlassian JIRA
(v6.2#6252)
