Zhijie Shen commented on YARN-3411:

bq. Hmm. So, hbase stores values in bytes. Reading back a value written as a 
Long using Bytes.toInteger does not work. Hence, there needs to be a way to 
know what the data type is going to be. 

Maybe we don't need to take care of object to bytes ser/des ourselves. Please 
check {{GenericObjectMapper}}, which wraps over jackson object reader and 
writer. We used to use it to store bytes of the object into leveldb and read 
bytes fro leveldb and convert them to the object. I think HBase backend 
implementation can make use of it too, and we can make ser/des at the frontend 
interface and at the backend persistency consistent. I suggest Phoenix using 
the same approach too. /cc [~gtCarrera9].

> [Storage implementation] explore the native HBase write schema for storage
> --------------------------------------------------------------------------
>                 Key: YARN-3411
>                 URL: https://issues.apache.org/jira/browse/YARN-3411
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Vrushali C
>            Priority: Critical
>         Attachments: ATSv2BackendHBaseSchemaproposal.pdf, 
> YARN-3411.poc.2.txt, YARN-3411.poc.txt
> There is work that's in progress to implement the storage based on a Phoenix 
> schema (YARN-3134).
> In parallel, we would like to explore an implementation based on a native 
> HBase schema for the write path. Such a schema does not exclude using 
> Phoenix, especially for reads and offline queries.
> Once we have basic implementations of both options, we could evaluate them in 
> terms of performance, scalability, usability, etc. and make a call.

This message was sent by Atlassian JIRA

Reply via email to