Li Lu commented on YARN-3862:

Hi [~varun_saxena], thanks for working on this! After looking at the WIP patch, 
I have some general questions and think some discussions will definitely be 

My first confusion is about the relationship between this JIRA and YARN-3863. 
Both JIRAs appear to be related to timeline filter designs. However, if we'd 
like to support filters in timeline v2, I think a natural workflow may be:
# Filter object model and support them in FS storage implementation for tests. 
# Supporting timeline filters in our HBase reader
# Connecting the filter implementation to our existing web services interface
Each steps may not necessarily be a separate JIRA. If I understand correctly 
we're including some changes in step 1 and 2 in this JIRA, and you're planning 
to add WS support later. So I think this will nicely address our current 
request on timeline filters. What is the plan for YARN-3863 then? Or I'm 
missing some major piece? 

Secondly it's about the role of our planned filters. I notice you're putting 
filters in a package within timeline reader. My feeling is that the concept of 
timeline filter may become a part of our object model, so that client users can 
easily communicate? 

A follow up question from this is, are we treating our timeline filters as 
pure-data objects (models) or something that include both filter data and the 
way we actually filter (models+algorithm)? I slightly prefer the former because 
this may decouple the filter information (in timeline server land) and the 
storage implementations. I agree our filter design should at least slightly 
favor HBase filters, but converting a timeline filter to an HBase filter should 
be restricted in the land of the actual storage. 

Finally a quick question: is it easy, or possible, for us to implement a 
"paging filter"? We have a lot of web UI use cases that may need such a filter. 

> Decide which contents to retrieve and send back in response in TimelineReader
> -----------------------------------------------------------------------------
>                 Key: YARN-3862
>                 URL: https://issues.apache.org/jira/browse/YARN-3862
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>         Attachments: YARN-3862-YARN-2928.wip.01.patch
> Currently, we will retrieve all the contents of the field if that field is 
> specified in the query API. In case of configs and metrics, this can become a 
> lot of data even though the user doesn't need it. So we need to provide a way 
> to query only a set of configs or metrics.
> As a comma spearated list of configs/metrics to be returned will be quite 
> cumbersome to specify, we have to support either of the following options :
> # Prefix match
> # Regex
> # Group the configs/metrics and query that group.
> We also need a facility to specify a metric time window to return metrics in 
> a that window. This may be useful in plotting graphs 

This message was sent by Atlassian JIRA

Reply via email to