[
https://issues.apache.org/jira/browse/YARN-3167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vrushali C updated YARN-3167:
-
Attachment: RM-AM-NM-Aggregator.png
Sequence_diagram_User_RM_AM_NM_Aggregator_Writer.png
Attaching a couple of diagrams that illustrate the design discussions between
me and Sangjin and Karthik and Robert.
Some notes:
- Per node aggregator process runs as an auxillary service inside the NM
- It has a per application datastructure to receive and write entity info for
that app
- The writer API has the following (at the least):
write (entity) : this will write all the contents of the entity to the
backend store
addEvent (application id, event) : this writes the lifecycle events
updateMetrics (application id, timeline metric) : this writes out the
metrics
- the base aggregator service api also has the above in its API
- in addition to these, the base aggregator service could do a sync/async write
call for each of the above.
For the PoC, we could choose to have every write as a synchronous write.
> [Aggregator implementation] Implement the core functionality of the base
> aggregator service
> ---
>
> Key: YARN-3167
> URL: https://issues.apache.org/jira/browse/YARN-3167
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
> Attachments: RM-AM-NM-Aggregator.png,
> Sequence_diagram_User_RM_AM_NM_Aggregator_Writer.png
>
>
> The basic skeleton of the timeline aggregator has been set up by YARN-3030.
> We need to implement the core functionality of the base aggregator service.
> The key things include
> - handling the requests from clients (sync or async)
> - buffering data
> - handling the aggregation logic
> - invoking the storage API
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
