[
https://issues.apache.org/jira/browse/YARN-1717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13935409#comment-13935409
]
Billie Rinaldi commented on YARN-1717:
--------------------------------------
In testing of writing and age off at the same time, the deletion thread did not
seem to adversely affect the write rate. With a single writer, I saw about 450
single-entity puts per second, which is comparable to what I had observed
previously. I configured the deletion thread to age data off after 90 seconds,
and also set the deletion cycle interval to 90 seconds. It was able to age off
data around 4500 entities per second. With these settings, it typically aged
off on the order of 36,000 entities per cycle in less than 8 seconds.
> Enable offline deletion of entries in leveldb timeline store
> ------------------------------------------------------------
>
> Key: YARN-1717
> URL: https://issues.apache.org/jira/browse/YARN-1717
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Billie Rinaldi
> Assignee: Billie Rinaldi
> Attachments: YARN-1717.1.patch, YARN-1717.10.patch,
> YARN-1717.11.patch, YARN-1717.2.patch, YARN-1717.3.patch, YARN-1717.4.patch,
> YARN-1717.5.patch, YARN-1717.6-extra.patch, YARN-1717.6.patch,
> YARN-1717.7.patch, YARN-1717.8.patch, YARN-1717.9.patch
>
>
> The leveldb timeline store implementation needs the following:
> * better documentation of its internal structures
> * internal changes to enable deleting entities
> ** never overwrite existing primary filter entries
> ** add hidden reverse pointers to related entities
--
This message was sent by Atlassian JIRA
(v6.2#6252)
