Varun Saxena commented on YARN-4053:

bq. it might be good to restrict the numeric types the metric will support. 
long and double sounds good to me. Can add verification as you said.

bq. HBase already provides a facility to encode and decode between numbers and 
Yes I know. As I had to append one byte in front of the byte array, I moved the 
logic in Bytes.toBytes to a separate method. This was done to avoid creation of 
2 byte arrays(one inside Bytes.toBytes and one in ATS code) and henceforth 
copying over result from Bytes.toBytes to the byte array created inside ATS 
code. Although this is just 8 bytes. So maybe can do above.

bq. Also, instead of encoding the info whether this is an integral type vs. 
floating type into the value, it would be better to have this information in 
the column qualifier. 
I see some issue in having this info in column qualifier. Because certain HBase 
filters like SingleColumnValueFilter require exact column qualifier name. So we 
will have to again guess about the type(similar to current patch) when we use 
Probably we can discuss this offline and conclude there. Will send a mail.

> Change the way metric values are stored in HBase Storage
> --------------------------------------------------------
>                 Key: YARN-4053
>                 URL: https://issues.apache.org/jira/browse/YARN-4053
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>         Attachments: YARN-4053-YARN-2928.01.patch
> Currently HBase implementation uses GenericObjectMapper to convert and store 
> values in backend HBase storage. This converts everything into a string 
> representation(ASCII/UTF-8 encoded byte array).
> While this is fine in most cases, it does not quite serve our use case for 
> metrics. 
> So we need to decide how are we going to encode and decode metric values and 
> store them in HBase.

This message was sent by Atlassian JIRA

Reply via email to