[ 
https://issues.apache.org/jira/browse/YARN-3167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vrushali C updated YARN-3167:
-----------------------------
    Attachment: RM-AM-NM-Aggregator.png
                Sequence_diagram_User_RM_AM_NM_Aggregator_Writer.png


Attaching a couple of diagrams that illustrate the design discussions between 
me and Sangjin and Karthik and Robert.

Some notes:
- Per node aggregator process runs as an auxillary service inside the NM
- It has a per application datastructure to receive and write entity info for 
that app
- The writer API has the following (at the least): 
---- write (entity) : this will write all the contents of the entity to the 
backend store
---- addEvent (application id, event) : this writes the lifecycle events
---- updateMetrics (application id, timeline metric) : this writes out the 
metrics
- the base aggregator service api also has the above in its API
- in addition to these, the base aggregator service could do a sync/async write 
call for each of the above. 

For the PoC, we could choose to have every write as a synchronous write.




> [Aggregator implementation] Implement the core functionality of the base 
> aggregator service
> -------------------------------------------------------------------------------------------
>
>                 Key: YARN-3167
>                 URL: https://issues.apache.org/jira/browse/YARN-3167
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Sangjin Lee
>         Attachments: RM-AM-NM-Aggregator.png, 
> Sequence_diagram_User_RM_AM_NM_Aggregator_Writer.png
>
>
> The basic skeleton of the timeline aggregator has been set up by YARN-3030. 
> We need to implement the core functionality of the base aggregator service. 
> The key things include
> - handling the requests from clients (sync or async)
> - buffering data
> - handling the aggregation logic
> - invoking the storage API



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to