Zhijie Shen commented on YARN-3551:

bq. Also, think along with singleData, we also need a timestamp member 
variable, else we would not know which timestamp this singleData value belongs 

If we want to know the timestamp of the value, what's difference between single 
data and time series? Or can I rephrase your proposal in another way: there 
exists two modes of a metric:

1. We remember all history values of one metric.
2. We only remember the last value of one metric.

So at the client for the first one, we use the data model api to buffer 
multiple data points of time series, while for the second one, we keep update 
the single point with the latest value. Once it is handed over to the server, 
for the first one, the server will persisted all the points in HBase by using 
version, while for the second one, the server will only track the latest 
version of the point.

bq. How about making the timeline metric generic?

I'm +0 about using generic type:

\+ It's good to restrict one metric just contains one type of value.
\- The caller will only know the object is TimelineMetric<?> but won't know 
what the "?" is before checking the object itself. It's no easier than 
inspecting the data directly.
\- It may not be complete consistent with the way jackson marshal and unmarshal 
the JSON object. For example, if we send a TimelineMetric<Long>(1L, 2L, 3L, 
4L), we will actually get TimelineMetric<Integer>(1, 2, 3, 4) according 
jasckon's rule of ser/des.

bq. Hmm, yes, we do need a datatype per metric to be stored, this will have to 
be considered in the writer impls.

Please take a look at my comment on 
 Once we receive the data at the server side, we know what kind of number the 
metric is. To ser/des with the backend, we can make use of GenericObjectMapper. 

> Consolidate data model change according to the backend implementation
> ---------------------------------------------------------------------
>                 Key: YARN-3551
>                 URL: https://issues.apache.org/jira/browse/YARN-3551
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Zhijie Shen
>            Assignee: Zhijie Shen
>         Attachments: YARN-3551.1.patch, YARN-3551.2.patch, YARN-3551.3.patch
> Based on the comments on 
> [YARN-3134|https://issues.apache.org/jira/browse/YARN-3134?focusedCommentId=14512080&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14512080]
>  and 
> [YARN-3411|https://issues.apache.org/jira/browse/YARN-3411?focusedCommentId=14512098&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14512098],
>  we need to change the data model to restrict the data type of 
> info/config/metric section.
> 1. Info: the value could be all kinds object that is able to be 
> serialized/deserialized by jackson.
> 2. Config: the value will always be assumed as String.
> 3. Metric: single data or time series value have to be number for aggregation.
> Other than that, info/start time/finish time of metric seem not to be 
> necessary for storage. They should be removed.

This message was sent by Atlassian JIRA

Reply via email to