Li Lu commented on YARN-4053:

Hi [~varun_saxena], thanks for the patch! With regard to the POC, I thought we 
agreed on the general plan of the POC on a web UI connected to the timeline 
reader during our weekly standup discussion? If this is the case, I would 
personally give the RESTful API patch slightly higher priority since that is 
critical to the whole workflow of the reader/webUI interface? 

About this patch, I totally agree that we should directly store the byte 
representation of the numbers instead of using generic object mapper. Having 
looked at the patch, I have some general comments here. Maybe what we want here 
is a way to model the types of the timeline metrics, so that the type 
information can be carried over from TimelineMetric objects to the storage 
layer? We may have something like TimelineData.FLOAT, TimelineData.LONG, 
TimelineData.GENERIC_OBJECT, etc., so that we can easily transfer those 
messages? My main concern is on the ColumnPrefix descriptions, where we now use 
a boolean flag to indicate if the column is numeric or not. This will also help 
us to better organize the serialization and deserialization helper methods and 
all related tests. Let me know if this idea works here, thanks! 

> Change the way metric values are stored in HBase Storage
> --------------------------------------------------------
>                 Key: YARN-4053
>                 URL: https://issues.apache.org/jira/browse/YARN-4053
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>         Attachments: YARN-4053-YARN-2928.01.patch
> Currently HBase implementation uses GenericObjectMapper to convert and store 
> values in backend HBase storage. This converts everything into a string 
> representation(ASCII/UTF-8 encoded byte array).
> While this is fine in most cases, it does not quite serve our use case for 
> metrics. 
> So we need to decide how are we going to encode and decode metric values and 
> store them in HBase.

This message was sent by Atlassian JIRA

Reply via email to