[
https://issues.apache.org/jira/browse/YARN-5638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Lu updated YARN-5638:
------------------------
Attachment: YARN-5638-trunk.v1.patch
First version of patch to demonstrate the idea. Right now I've split collector
discovery process into two steps: In the first step, collector manager reports
the collector to NM, and NM sends the collector data to the RM for
registration. In the second step, the RM (synchronously) assigns the collector
a timestamp (rm's timestamp and a version number) and store it in memory. The
RM then updates known collector data via heartbeats to all NMs as before. The
only difference is the RM attach timestamp information for each collector to
NMs so that once there's a rebuild process, NMs can report this information.
> Introduce a collector timestamp to uniquely identify collectors creation
> order in collector discovery
> -----------------------------------------------------------------------------------------------------
>
> Key: YARN-5638
> URL: https://issues.apache.org/jira/browse/YARN-5638
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Li Lu
> Assignee: Li Lu
> Attachments: YARN-5638-trunk.v1.patch
>
>
> As discussed in YARN-3359, we need to further identify timeline collectors'
> creation order to rebuild collector discovery data in the RM. This JIRA
> proposes to use <rm_timestamp, logical_version_number> to order collectors
> for each application in the RM. This timestamp can then be used when a
> standby RM becomes active and rebuild collector discovery data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
