Li Lu commented on YARN-3904:

One question about our current writer design is, do we have a designated use 
case for the {{aggregate}} method? I remember at the time when we was designing 
the writer interface, there was no such concepts as "real-time" or "time-based" 
aggregations. For time-based aggregation writers, the current {{aggregate}} 
lacks of cluster/user information that forms the primary keys of the aggregated 
entities. So, do we want to keep this method for real-time aggregation, or we 
want to slightly modify it to accommodate both real-time and time-based 
aggregation? Is the current {{aggregate}} interface good enough for real-time 
aggregation? ([~vrushalic] am I missing anything here? )

> Refactor timelineservice.storage to add support to online and offline 
> aggregation writers
> -----------------------------------------------------------------------------------------
>                 Key: YARN-3904
>                 URL: https://issues.apache.org/jira/browse/YARN-3904
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Li Lu
>            Assignee: Li Lu
>         Attachments: YARN-3904-YARN-2928.001.patch, 
> YARN-3904-YARN-2928.002.patch, YARN-3904-YARN-2928.003.patch
> After we finished the design for time-based aggregation, we can adopt our 
> existing Phoenix storage into the storage of the aggregated data. In this 
> JIRA, I'm proposing to refactor writers to add support to aggregation 
> writers. Offline aggregation writers typically has less contextual 
> information. We can distinguish these writers by special naming. We can also 
> use CollectorContexts to model all contextual information and use it in our 
> writer interfaces. 

This message was sent by Atlassian JIRA

Reply via email to