Li Lu commented on YARN-3411:

Hi [~vrushalic], thanks for working on this! I just got back and looked at the 
latest (v6) patch. In general I think it's quite close. I just have a few quick 

# I notice you made {{EntityColumnFamilyDetails}} and 
{{EntityInfoColumnDetails}} to be public. Even though we're adding @Private 
tags to them, we may still want to use the strictest access modifier possible. 
If they're fine to be visible to package only, maybe we can use default, rather 
than public for them? (I tried with my local version and seems like they worked 
fine, but not sure if I missed anything. )
# I share the same concern as [~zjshen] on schema creation. Having a separate 
step to set up the HBase data schema appears to add some workload to Hadoop 
deployments. So, although it's totally fine to have this step for our 
performance benchmark, it will be great to keep this in mind when we 
consolidate the storage part. 
# Maybe it's helpful to move the entity creation steps of 
TestHBaseTimelineWriterImpl to TestTimelineWriterImpl? In this way we can 
improve test coverage for both writer implementations after this work. 

> [Storage implementation] explore the native HBase write schema for storage
> --------------------------------------------------------------------------
>                 Key: YARN-3411
>                 URL: https://issues.apache.org/jira/browse/YARN-3411
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Vrushali C
>            Priority: Critical
>         Attachments: ATSv2BackendHBaseSchemaproposal.pdf, 
> YARN-3411-YARN-2928.001.patch, YARN-3411-YARN-2928.002.patch, 
> YARN-3411-YARN-2928.003.patch, YARN-3411-YARN-2928.004.patch, 
> YARN-3411-YARN-2928.005.patch, YARN-3411-YARN-2928.006.patch, 
> YARN-3411.poc.2.txt, YARN-3411.poc.3.txt, YARN-3411.poc.4.txt, 
> YARN-3411.poc.5.txt, YARN-3411.poc.6.txt, YARN-3411.poc.7.txt, 
> YARN-3411.poc.txt
> There is work that's in progress to implement the storage based on a Phoenix 
> schema (YARN-3134).
> In parallel, we would like to explore an implementation based on a native 
> HBase schema for the write path. Such a schema does not exclude using 
> Phoenix, especially for reads and offline queries.
> Once we have basic implementations of both options, we could evaluate them in 
> terms of performance, scalability, usability, etc. and make a call.

This message was sent by Atlassian JIRA

Reply via email to