[
https://issues.apache.org/jira/browse/YARN-3411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14517838#comment-14517838
]
Zhijie Shen commented on YARN-3411:
-----------------------------------
bq. Hmm. So, hbase stores values in bytes. Reading back a value written as a
Long using Bytes.toInteger does not work. Hence, there needs to be a way to
know what the data type is going to be.
Maybe we don't need to take care of object to bytes ser/des ourselves. Please
check {{GenericObjectMapper}}, which wraps over jackson object reader and
writer. We used to use it to store bytes of the object into leveldb and read
bytes fro leveldb and convert them to the object. I think HBase backend
implementation can make use of it too, and we can make ser/des at the frontend
interface and at the backend persistency consistent. I suggest Phoenix using
the same approach too. /cc [~gtCarrera9].
> [Storage implementation] explore the native HBase write schema for storage
> --------------------------------------------------------------------------
>
> Key: YARN-3411
> URL: https://issues.apache.org/jira/browse/YARN-3411
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Reporter: Sangjin Lee
> Assignee: Vrushali C
> Priority: Critical
> Attachments: ATSv2BackendHBaseSchemaproposal.pdf,
> YARN-3411.poc.2.txt, YARN-3411.poc.txt
>
>
> There is work that's in progress to implement the storage based on a Phoenix
> schema (YARN-3134).
> In parallel, we would like to explore an implementation based on a native
> HBase schema for the write path. Such a schema does not exclude using
> Phoenix, especially for reads and offline queries.
> Once we have basic implementations of both options, we could evaluate them in
> terms of performance, scalability, usability, etc. and make a call.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
