[
https://issues.apache.org/jira/browse/YARN-2032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Lu updated YARN-2032:
------------------------
Attachment: YARN-2032-091114.patch
Hi everyone, based on [~mayank_bansal]'s patch, I've done an updated version
for the HBase timeline storage. General strategy and data schema are kept the
same as the original patch. Here're what I've done:
1. Finished fromTs and fromId function in getEntities. For fromId, I added one
more check to change the first record of a scan if necessary. For fromTs, I
added a separate column qualifier in the entity table to store insert time for
each entity, and, during query, filter out later records if necessary.
2. I restructured the code such that data schema and operations are decoupled
from the actual HBase operations. Most timeline data storage logic is in the
abstract store class now. While HBase storage class only needs to implement the
abstract methods and interfaces required by the abstract storage class,
including table creation, get, put, scan, and a few other helper functions. I
hope this "pluggable" interface would simplify future extensions, and help to
provide a unified abstract storage schema across different data storages.
Comments on this design would certainly be more than welcome.
3. I've added two more unit test cases, to test fromId and fromTs, for HBase
storage. Currently, the UTs work fine with branch-2, or with a HttpServer.java
accessible by HBaseClient. But UTs are failing in trunk branch since
HttpServer.java has been replaced. I added "Ignore" tags to the UT for now, but
feel free to check them under branch-2.
> Implement a scalable, available TimelineStore using HBase
> ---------------------------------------------------------
>
> Key: YARN-2032
> URL: https://issues.apache.org/jira/browse/YARN-2032
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Mayank Bansal
> Attachments: YARN-2032-091114.patch, YARN-2032-branch-2-1.patch,
> YARN-2032-branch2-2.patch
>
>
> As discussed on YARN-1530, we should pursue implementing a scalable,
> available Timeline store using HBase.
> One goal is to reuse most of the code from the levelDB Based store -
> YARN-1635.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
