Varun Saxena commented on YARN-3862:

Attached a WIP patch.

This patch attempts to do handling for all the tables(creation of filter list 
based on fields) and do prefix matching for configs and filters. Previous WIP 
patch was only attempting to handle for entity table because of the 
implications of this patch on config and metric filters' matching. Have handled 
this scenario in this patch. 
When YARN-3863 is done, some changes will be warranted though(some conditions 
to pass config and metric filters will have to be removed). 
Have added a few tests to test the change as well.

I have still not hooked up this code to REST API layer.
For that, we first need to decide as to whether the TimelineFilter code will be 
part of our object model or not.
For prefix matching of configs and metrics to return, at the REST layer this 
can simply come as a query param (a comma separated list)

But when we code for complex filters (especially metric filters) in YARN-3863 
we will have to support SQL type queries with ANDs', ORs', >,<,=operators, etc.
If we make TimelineFilter as part of our client object model and interpret 
filters as a JSON string associated with a query param, we might have to 
rethink a bit on few of the classes and  including additional checks(as this 
will be used by client).
This can increase size of the URL though.

If we do not include filter as part of our object model, we will have to decide 
how to specify complex config and metric filters containing ANDs' and ORs' and 
different relational operators(because some of the symbols will be reserved) 
and reach a consensus on that.

> Decide which contents to retrieve and send back in response in TimelineReader
> -----------------------------------------------------------------------------
>                 Key: YARN-3862
>                 URL: https://issues.apache.org/jira/browse/YARN-3862
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>         Attachments: YARN-3862-YARN-2928.wip.01.patch, 
> YARN-3862-YARN-2928.wip.02.patch, YARN-3862-YARN-2928.wip.03.patch
> Currently, we will retrieve all the contents of the field if that field is 
> specified in the query API. In case of configs and metrics, this can become a 
> lot of data even though the user doesn't need it. So we need to provide a way 
> to query only a set of configs or metrics.
> As a comma spearated list of configs/metrics to be returned will be quite 
> cumbersome to specify, we have to support either of the following options :
> # Prefix match
> # Regex
> # Group the configs/metrics and query that group.
> We also need a facility to specify a metric time window to return metrics in 
> a that window. This may be useful in plotting graphs 

This message was sent by Atlassian JIRA

Reply via email to