[
https://issues.apache.org/jira/browse/YARN-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14641180#comment-14641180
]
Li Lu commented on YARN-3904:
-----------------------------
One question about our current writer design is, do we have a designated use
case for the {{aggregate}} method? I remember at the time when we was designing
the writer interface, there was no such concepts as "real-time" or "time-based"
aggregations. For time-based aggregation writers, the current {{aggregate}}
lacks of cluster/user information that forms the primary keys of the aggregated
entities. So, do we want to keep this method for real-time aggregation, or we
want to slightly modify it to accommodate both real-time and time-based
aggregation? Is the current {{aggregate}} interface good enough for real-time
aggregation? ([~vrushalic] am I missing anything here? )
> Refactor timelineservice.storage to add support to online and offline
> aggregation writers
> -----------------------------------------------------------------------------------------
>
> Key: YARN-3904
> URL: https://issues.apache.org/jira/browse/YARN-3904
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Li Lu
> Assignee: Li Lu
> Attachments: YARN-3904-YARN-2928.001.patch,
> YARN-3904-YARN-2928.002.patch, YARN-3904-YARN-2928.003.patch
>
>
> After we finished the design for time-based aggregation, we can adopt our
> existing Phoenix storage into the storage of the aggregated data. In this
> JIRA, I'm proposing to refactor writers to add support to aggregation
> writers. Offline aggregation writers typically has less contextual
> information. We can distinguish these writers by special naming. We can also
> use CollectorContexts to model all contextual information and use it in our
> writer interfaces.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
