[
https://issues.apache.org/jira/browse/YARN-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696131#comment-14696131
]
Li Lu commented on YARN-3862:
-----------------------------
Hi [~varun_saxena], thanks for working on this! After looking at the WIP patch,
I have some general questions and think some discussions will definitely be
helpful.
My first confusion is about the relationship between this JIRA and YARN-3863.
Both JIRAs appear to be related to timeline filter designs. However, if we'd
like to support filters in timeline v2, I think a natural workflow may be:
# Filter object model and support them in FS storage implementation for tests.
# Supporting timeline filters in our HBase reader
# Connecting the filter implementation to our existing web services interface
Each steps may not necessarily be a separate JIRA. If I understand correctly
we're including some changes in step 1 and 2 in this JIRA, and you're planning
to add WS support later. So I think this will nicely address our current
request on timeline filters. What is the plan for YARN-3863 then? Or I'm
missing some major piece?
Secondly it's about the role of our planned filters. I notice you're putting
filters in a package within timeline reader. My feeling is that the concept of
timeline filter may become a part of our object model, so that client users can
easily communicate?
A follow up question from this is, are we treating our timeline filters as
pure-data objects (models) or something that include both filter data and the
way we actually filter (models+algorithm)? I slightly prefer the former because
this may decouple the filter information (in timeline server land) and the
storage implementations. I agree our filter design should at least slightly
favor HBase filters, but converting a timeline filter to an HBase filter should
be restricted in the land of the actual storage.
Finally a quick question: is it easy, or possible, for us to implement a
"paging filter"? We have a lot of web UI use cases that may need such a filter.
> Decide which contents to retrieve and send back in response in TimelineReader
> -----------------------------------------------------------------------------
>
> Key: YARN-3862
> URL: https://issues.apache.org/jira/browse/YARN-3862
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: YARN-2928
> Reporter: Varun Saxena
> Assignee: Varun Saxena
> Attachments: YARN-3862-YARN-2928.wip.01.patch
>
>
> Currently, we will retrieve all the contents of the field if that field is
> specified in the query API. In case of configs and metrics, this can become a
> lot of data even though the user doesn't need it. So we need to provide a way
> to query only a set of configs or metrics.
> As a comma spearated list of configs/metrics to be returned will be quite
> cumbersome to specify, we have to support either of the following options :
> # Prefix match
> # Regex
> # Group the configs/metrics and query that group.
> We also need a facility to specify a metric time window to return metrics in
> a that window. This may be useful in plotting graphs
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
