[
https://issues.apache.org/jira/browse/YARN-4376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Eagles updated YARN-4376:
----------------------------------
Attachment: YARN-4376.2.patch
Posting a patch that addresses the issues above. It's not easy to reproduce
this error since the iteration order is implementation dependent. I went with
adding the TreeSet as secondary view into the entities. As far as extra memory
requirements needed, documentation states 40 bytes * capacity. For 1,000,000
entities the extra memory requirements will be 8MB. The insert time is
increased for 1,000,000 entries from 3.5 seconds to 11 seconds. Looking into
whether this is significant on amortization. Will test with a huge entity set
to validate the performance before and after.
> Memory Timeline Store return incorrect results on fromId paging
> ---------------------------------------------------------------
>
> Key: YARN-4376
> URL: https://issues.apache.org/jira/browse/YARN-4376
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Jonathan Eagles
> Assignee: Jonathan Eagles
> Attachments: YARN-4376.2.patch
>
>
> As pointed out correctly by [~jlowe].
> https://issues.apache.org/jira/browse/TEZ-2628?focusedCommentId=14715831&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14715831
> The MemoryTimelineStore cannot page correctly when using fromId. This is due
> switching between data structures that apparently have different natural
> sorting. In addition, the approach of creating a new data structure every
> time from scratch is costly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
