Varun Saxena commented on YARN-4053:

Naga, yes double can also be used. 
But the problem with storing double is that we will be sending back a value 
with decimals and client may not be able to interpret it properly if its not 
expecting a decimal value(for integral values). IMO, we cant make assumptions 
about client because it may be assuming that a particular metric has been 
scored as an integral value in ATS. Although same problem will be with longs as 
well. But long is being thought of, as we are assuming we wont be supporting 
floating points as of now. A more concrete solution would be as under :

Assume solution 2 which I proposed above. Here we can have a flag (1 byte 
although 1 bit should be enough but for HBase storage 1 byte would be needed). 
This can indicate whether  a value is to be interpreted as long or double. We 
can take care of encoding/decoding while reading/writing from Hbase.

     1                    8
   |   |                                     |
   | 0 |    (Integral value stored as long)  |
   |   |                                     |

     1                    8
   |   |                                     |
   | 1 |  (Floating value stored as double)  |
   |   |                                     |

If flag is 0, it indicates integral value and if 1 it indicates floating point 


And we can place a restriction on clients saying that if they want a metric to 
be of floating point format, in all interactions with ATS the value should be 
in decimals i.e. 40 should be sent as 40.0 for instance. That will be a fair 
enough restriction to place.

However it all depends on whether we want to support floating points at the 

> Change the way metric values are stored in HBase Storage
> --------------------------------------------------------
>                 Key: YARN-4053
>                 URL: https://issues.apache.org/jira/browse/YARN-4053
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
> Currently HBase implementation uses GenericObjectMapper to convert and store 
> values in backend HBase storage. This converts everything into a string 
> representation(ASCII/UTF-8 encoded byte array).
> While this is fine in most cases, it does not quite serve our use case for 
> metrics. 
> So we need to decide how are we going to encode and decode metric values and 
> store them in HBase.

This message was sent by Atlassian JIRA

Reply via email to