Junping Du commented on YARN-3031:

bq. The AggregateUpTo enum has the tracks to aggregate along, the 
TimelineEntityType enum has the types of entities that can exist. There may not 
be aggregations along all entity types. 
I see. Thanks [~vrushalic] for explanation here. However, "CLUSTER" seems to be 
missing here as it is an aggregation of "FLOW"s. Isn't it?

bq. The reasoning behind having two more apis for writing metrics and events in 
addition to the entity write is that, it would be good (efficient) to have the 
option to write a single metric or a single event. For example, say a job has 
many custom metrics and one particular metric is updated extremely frequently 
but not the others. We may want to write out only that particular metric 
without having to look through/write all other metrics and other information in 
that entity. Similarly for events. Perhaps we could do it differently that what 
is proposed in the patch, but the functionality of writing them individually 
would help in performance I believe.
Agree that we should have separated interfaces to write single data entry 
quickly and aggregate data entries. Also some aggregator (like RM) won't even 
call aggregation interface here (according to YARN-3167). IMO, it sounds like 
two interfaces are good enough so we can merge addEvent() and updateMetrics() 
into a single data entry writer which can accept more generic type? That will 
make interface more concisely and hiding more details that could be changed in 
future. Thoughts?

> [Storage abstraction] Create backing storage write interface for ATS writers
> ----------------------------------------------------------------------------
>                 Key: YARN-3031
>                 URL: https://issues.apache.org/jira/browse/YARN-3031
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Vrushali C
>         Attachments: Sequence_diagram_write_interaction.2.png, 
> Sequence_diagram_write_interaction.png, YARN-3031.01.patch, 
> YARN-3031.02.patch, YARN-3031.03.patch
> Per design in YARN-2928, come up with the interface for the ATS writer to 
> write to various backing storages. The interface should be created to capture 
> the right level of abstractions so that it will enable all backing storage 
> implementations to implement it efficiently.

This message was sent by Atlassian JIRA

Reply via email to