[
https://issues.apache.org/jira/browse/YARN-3411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14546176#comment-14546176
]
Vrushali C commented on YARN-3411:
----------------------------------
Hi [~jrottinghuis]
Thanks for the review! My response below.
bq. On class range, the start and end are final right?
yes
bq. Wouln't it be better to do this in somewhat more of a streaming manner?
Agreed, modifying the code accordingly right now. Also we discussed today about
checking out what the hbase batching/flushing policy and accordingly
considering emitting puts as we see them.
bq. More general question: for TimelineWriter and its implementations, is there
an expectation set around concurrency?
Good point, I have filed YARN-3650 to discuss that
bq. it will be super-usefull to be able to have one single config key to prefix
all tables.
Agreed. I have filed YARN-3649 to work on that
bq. getRowKeyPrefix should cleanse input arguments to strip
EntityTableDetails.ROW_KEY_SEPARATOR_BYTES
yes, updating the code now.
bq. Overall, we should fill in the javadoc, describe inputs, explain what we
expect etc.
Yes, will update the documentation as much as I can along the lines of what we
discussed today.
bq. EntityColumnDetails probably needs a public static method that will take
bytes, iterates over the enum and returns an enum from those bytes.
Yes, I will add that in later if/when we decide to go forward with this method.
thanks
Vrushali
> [Storage implementation] explore the native HBase write schema for storage
> --------------------------------------------------------------------------
>
> Key: YARN-3411
> URL: https://issues.apache.org/jira/browse/YARN-3411
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Sangjin Lee
> Assignee: Vrushali C
> Priority: Critical
> Attachments: ATSv2BackendHBaseSchemaproposal.pdf,
> YARN-3411-YARN-2928.001.patch, YARN-3411-YARN-2928.002.patch,
> YARN-3411.poc.2.txt, YARN-3411.poc.3.txt, YARN-3411.poc.4.txt,
> YARN-3411.poc.5.txt, YARN-3411.poc.6.txt, YARN-3411.poc.7.txt,
> YARN-3411.poc.txt
>
>
> There is work that's in progress to implement the storage based on a Phoenix
> schema (YARN-3134).
> In parallel, we would like to explore an implementation based on a native
> HBase schema for the write path. Such a schema does not exclude using
> Phoenix, especially for reads and offline queries.
> Once we have basic implementations of both options, we could evaluate them in
> terms of performance, scalability, usability, etc. and make a call.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
