[
https://issues.apache.org/jira/browse/YARN-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Lu updated YARN-3904:
------------------------
Attachment: YARN-3904-YARN-2928.006.patch
Upload the 006 version of the patch. In this patch I addressed most of
[~zjshen]'s review comments. I think I need some discussions on the following
two points:
bq. moving the table creation stuff into TimelineSchemaCreator.
I'm not 100% sure if that's what we would like to do. Maybe we would like to
decouple the offline aggregation module from our normal entity storage.
Therefore, maybe it's also appealing to allow users specify if they need to
create data schema in the offline aggregation process? Such as, setting one
flag in the offline aggregator to create data schema?
bq. As HBase backend is accessed both directly and via Phoenix, it's good for
us to cleanup the configuration to say we're using the HBase backend (comparing
to FS backend) instead of specifically HBase or Phoenix writer/reader.
After the changes in this JIRA, we will only have two types of TimelineWriters,
one for FS (test only) and one for HBase. The setting on the offline storage
should be independent from this setting, I assume?
> Refactor timelineservice.storage to add support to online and offline
> aggregation writers
> -----------------------------------------------------------------------------------------
>
> Key: YARN-3904
> URL: https://issues.apache.org/jira/browse/YARN-3904
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Li Lu
> Assignee: Li Lu
> Attachments: YARN-3904-YARN-2928.001.patch,
> YARN-3904-YARN-2928.002.patch, YARN-3904-YARN-2928.003.patch,
> YARN-3904-YARN-2928.004.patch, YARN-3904-YARN-2928.005.patch,
> YARN-3904-YARN-2928.006.patch
>
>
> After we finished the design for time-based aggregation, we can adopt our
> existing Phoenix storage into the storage of the aggregated data. In this
> JIRA, I'm proposing to refactor writers to add support to aggregation
> writers. Offline aggregation writers typically has less contextual
> information. We can distinguish these writers by special naming. We can also
> use CollectorContexts to model all contextual information and use it in our
> writer interfaces.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
