[
https://issues.apache.org/jira/browse/YARN-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14703889#comment-14703889
]
Sangjin Lee commented on YARN-4053:
-----------------------------------
I discussed this with [~jrottinghuis] a little more, and have some high level
comments.
First off, HBase already provides a facility to encode and decode between
numbers and bytes: {{Bytes.toBytes(long)}}, {{Bytes.toLong(byte[])}}, etc. We
should use them instead of rolling our own.
Also, instead of encoding the info whether this is an integral type vs.
floating type into the value, it would be better to have this information in
the column qualifier. For example, if metric "foo" is of an integral type, then
the column qualifier could contain "...=foo=l..." (where "l" denotes a long
value). That way, we can easily read out the type of values and deserialize
appropriately. It would also remove any uncertainty around having different
types of data for the same metric. There may be other variations of the idea.
Finally, I think it might be good to restrict the numeric types the metric will
support. My proposal would be to limit it to {{Long}} and {{Double}}. They
should be able to encompass pretty much all types of metrics. This would
simplify support quite a bit. We can have verification in
{{TimelineMetric.addValue()}} and so on to check the incoming type.
> Change the way metric values are stored in HBase Storage
> --------------------------------------------------------
>
> Key: YARN-4053
> URL: https://issues.apache.org/jira/browse/YARN-4053
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: YARN-2928
> Reporter: Varun Saxena
> Assignee: Varun Saxena
> Attachments: YARN-4053-YARN-2928.01.patch
>
>
> Currently HBase implementation uses GenericObjectMapper to convert and store
> values in backend HBase storage. This converts everything into a string
> representation(ASCII/UTF-8 encoded byte array).
> While this is fine in most cases, it does not quite serve our use case for
> metrics.
> So we need to decide how are we going to encode and decode metric values and
> store them in HBase.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
