[
https://issues.apache.org/jira/browse/YARN-3411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14509951#comment-14509951
]
Li Lu commented on YARN-3411:
-----------------------------
Hi [~vrushalic], thanks for the patch! I'm OK with the major part of this patch
for now. Here, I'm listing some questions that we can have some discussion on.
# About null checks: so far we do not have a fixed standard on if and where we
need to do null checks. I noticed you assumed info, config, event, and other
similar fields are not null. Maybe we'd like to explicitly decide when all
those fields can be null or empty.
# Maybe we'd like to change TimelineWriterUtils to default access modifier? I
think it would be sufficient to make it visible in package?
# One thing I'd like to open a discussion is on deciding the way to store and
process metrics. Currently, in the hbase patch, startTime and endTime are not
used. In the Phoenix patch, I store time series as a flattened, non-queryable
strings. I think this part also requires some hint from the time-based
aggregations.
# Another thing I'd like to discuss here is if and how we'd like to set up a
separate "fast path" for metric only updates. On the storage layer, I'd
strongly +1 for a separate fast path such that we can only touch the
(frequently updated) metrics table. Any proposals everyone?
> [Storage implementation] explore the native HBase write schema for storage
> --------------------------------------------------------------------------
>
> Key: YARN-3411
> URL: https://issues.apache.org/jira/browse/YARN-3411
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Sangjin Lee
> Assignee: Vrushali C
> Priority: Critical
> Attachments: ATSv2BackendHBaseSchemaproposal.pdf,
> YARN-3411.poc.2.txt, YARN-3411.poc.txt
>
>
> There is work that's in progress to implement the storage based on a Phoenix
> schema (YARN-3134).
> In parallel, we would like to explore an implementation based on a native
> HBase schema for the write path. Such a schema does not exclude using
> Phoenix, especially for reads and offline queries.
> Once we have basic implementations of both options, we could evaluate them in
> terms of performance, scalability, usability, etc. and make a call.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
