Vrushali C commented on YARN-3411:

Hi Sangjin,

Thank you for the comments! 
bq. I’m sure you're aware, but please don't forget to add the license to the 
new files later.
Ah, did I miss adding the license, let me ensure I add the licenses in the next 

bq. l.77: So do we need to depend on hbase-server? That's bit unexpected?
Let me try to work this out again, I had added it in as I was working through 
compilation and runtime errors, but will look into this again to confirm/modify.

bq. l.7: I think we're now moving away from the acronym "ats". Perhaps we 
should simply use "timeline.entity"?
Yes, will correct the literal value in the constant.

bq. (HBaseTimelineWriterImpl.java)
l.40: Just curious, does this mean a single writer instance has only on HBase 
client connection? We should be able to have multiple connections? What is your 
thought on this?
So, I have been thinking about this too. We would right now have one instance 
per NodeManager in this version, right? So that is already a lot of connections 
for cluster with thousands of nodes. I am wondering if we may not even want 
those many. Later on, when we move to having a timeline service writer per 
application, even then, say an app that does many writes would simply flood the 
writes to the region servers if it had a pool of connections at it’s disposal. 
Perhaps having one per app would help to rate limit it somewhat? I am still 
thinking out loud about this, haven’t decided which is better or worse. 

bq. Also, the initialization operations inside the constructor should probably 
belong in serviceInit() or serviceStart()?
Okay, will move it. 

> [Storage implementation] explore the native HBase write schema for storage
> --------------------------------------------------------------------------
>                 Key: YARN-3411
>                 URL: https://issues.apache.org/jira/browse/YARN-3411
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Vrushali C
>            Priority: Critical
>         Attachments: ATSv2BackendHBaseSchemaproposal.pdf, 
> YARN-3411.poc.2.txt, YARN-3411.poc.txt
> There is work that's in progress to implement the storage based on a Phoenix 
> schema (YARN-3134).
> In parallel, we would like to explore an implementation based on a native 
> HBase schema for the write path. Such a schema does not exclude using 
> Phoenix, especially for reads and offline queries.
> Once we have basic implementations of both options, we could evaluate them in 
> terms of performance, scalability, usability, etc. and make a call.

This message was sent by Atlassian JIRA

Reply via email to