Li Lu updated YARN-2032:
    Attachment: YARN-2032-091114.patch

Hi everyone, based on [~mayank_bansal]'s patch, I've done an updated version 
for the HBase timeline storage. General strategy and data schema are kept the 
same as the original patch. Here're what I've done:

1. Finished fromTs and fromId function in getEntities. For fromId, I added one 
more check to change the first record of a scan if necessary. For fromTs, I 
added a separate column qualifier in the entity table to store insert time for 
each entity, and, during query, filter out later records if necessary. 

2. I restructured the code such that data schema and operations are decoupled 
from the actual HBase operations. Most timeline data storage logic is in the 
abstract store class now. While HBase storage class only needs to implement the 
abstract methods and interfaces required by the abstract storage class, 
including table creation, get, put, scan, and a few other helper functions. I 
hope this "pluggable" interface would simplify future extensions, and help to 
provide a unified abstract storage schema across different data storages. 
Comments on this design would certainly be more than welcome. 

3. I've added two more unit test cases, to test fromId and fromTs, for HBase 
storage. Currently, the UTs work fine with branch-2, or with a HttpServer.java 
accessible by HBaseClient. But UTs are failing in trunk branch since 
HttpServer.java has been replaced. I added "Ignore" tags to the UT for now, but 
feel free to check them under branch-2. 

> Implement a scalable, available TimelineStore using HBase
> ---------------------------------------------------------
>                 Key: YARN-2032
>                 URL: https://issues.apache.org/jira/browse/YARN-2032
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Mayank Bansal
>         Attachments: YARN-2032-091114.patch, YARN-2032-branch-2-1.patch, 
> YARN-2032-branch2-2.patch
> As discussed on YARN-1530, we should pursue implementing a scalable, 
> available Timeline store using HBase.
> One goal is to reuse most of the code from the levelDB Based store - 
> YARN-1635.

This message was sent by Atlassian JIRA

Reply via email to