[
https://issues.apache.org/jira/browse/YARN-3862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14693605#comment-14693605
]
Varun Saxena commented on YARN-3862:
------------------------------------
Attached a WIP patch so that we can further discuss.
The patch does the following :
# Adds some filters along the lines of HBase Filters.
# Uses family and qualifier filters to reduce the amount of data fetched from
backend. We do not need to get data related to all the column families(and
qualifiers) if fields to retrieve does not contain them.
# This WIP patch supports prefix filters to fetch specific metrics and configs.
What we further need to decide is as under :
# The filter interface added is primarily a placeholder and all the filter
classes act as data classes. Currently the patch converts these filters into
HBase Filters in a util class(based on filter type). But we can use
polymorphism as well to let each filter class generate an analogous HBase
Filter. However, I am not sure if we would want storage related(HBase) code
logic these filter classes. Let us decide here and do what the consensus is.
# We also need to decide as to what all we need to support with regards to
configs and metrics we want to retrieve. Do we support substring and regex as
well ?
# REST API related code will be added once YARN-3814 goes in.
# Please note that this needs to work in conjunction with changes wrt to config
and metric filters so that certain data is not missed. So, this JIRA is
somewhat dependent on work in YARN-3863 as well.
> Decide which contents to retrieve and send back in response in TimelineReader
> -----------------------------------------------------------------------------
>
> Key: YARN-3862
> URL: https://issues.apache.org/jira/browse/YARN-3862
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: YARN-2928
> Reporter: Varun Saxena
> Assignee: Varun Saxena
> Attachments: YARN-3862-YARN-2928.wip.01.patch
>
>
> Currently, we will retrieve all the contents of the field if that field is
> specified in the query API. In case of configs and metrics, this can become a
> lot of data even though the user doesn't need it. So we need to provide a way
> to query only a set of configs or metrics.
> As a comma spearated list of configs/metrics to be returned will be quite
> cumbersome to specify, we have to support either of the following options :
> # Prefix match
> # Regex
> # Group the configs/metrics and query that group.
> We also need a facility to specify a metric time window to return metrics in
> a that window. This may be useful in plotting graphs
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
