[
https://issues.apache.org/jira/browse/YARN-4447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15260663#comment-15260663
]
Varun Saxena commented on YARN-4447:
------------------------------------
[~sjlee0], kindly review.
Apologise for the delay in updating this patch. I was away for a week and was
busy with internal work earlier so could not work on it.
I will add a few more test cases to test application fetching flow. But the
main code can be reviewed anyways.
Will give a small description of what has been done.
# Metric filters are of the form {{(((metric1 gt 23) AND (metric2 lt 40)) OR
(metric5 eq 40))}}. Comparison operators supported are as follows :
#* gt(Greater than) / ge(Greater than or equals)
#* lt(Less than) / le(Less than or equals)
#* eq(Equals) / ne(Not equals. If key(metric/config/info) does not exist, this
would mean a match) / ene(Exists and not equals. Key must exist).
# Config and info filters are of the same form as metric filters except that
only eq,ne and ene comparison operators are supported for them.
# Event filters will take the form {{(((event1,event2) AND \!(event3,event4))
OR event5, event6)}}. This means for an entity to match, event1 and event2
should exist and event3 and event4 should exist. Or, event5 and event6 should
exist. ! indicates not equals(non-existence). A not(\!) should be followed by
an opening bracket i.e. (
# Relation filters will take the same form as event filters. Here instead of
each event we will have type-entities expression i.e. of the form
{{type1:entity1:entity2}}. This part of the expression cannot contain spaces.
Relation filters hence would be of the form {{type1:entity11:entity12 ,
type2:entity21 AND !(type3:entity31)}}.
Also, ene kind of case wont be supported here. If entity type does not exist,
match wont occur.
# Metrics to retrieve and configs to retrieve will have similar format to
above. However ANDs' and ORs' do not make much sense here. Hence an expression
of the form conf1,conf2 means return configs conf1 and conf2. And expression of
the form \!(conf1,conf2) means returns all configs other than conf1 and conf2.
# Pls note that metric filters have not yet been supported for flow runs. They
need to be matched locally as summation for metrics happens in coprocessor. Can
be done in a separate JIRA.
Now coming to implementation, I had 2 options. To implement parsing logic in
static methods or encapsulate this logic in a class. Went with latter as it
makes it easier to break code into multiple methods without a need to pass
several parameters to helper methods. And makes the code cleaner IMO. This
would mean that an extra object will have to be created everytime though. Would
like to know thoughts of others on the approach.
# There is a new interface named {{TimelineParser}} added which needs to be
implemented for parsing different expressions.
# There are 2 abstract classes added namely TimelineParserForCompareExpr(for
expressions of the form explained above for metric/config/info filters) and
TimelineParserForEqualityExpr(for expressions of the form explained above for
event/relation filters). These classes will have abstract methods, which will
be implemented by concrete implementations, for deciding what kind of filter
needs to be constructed for filter list, how to parse the values and how to set
value, compareop, etc. to the filters.
# These abstract classes will then have concrete implementation for different
filters. These include TimelineParserForNumericFilters(for metric filters),
TimelineParserForKVFilters(for config/info filters),
TimelineParserForExistFilters(used for filters which check for existence such
as event filters) and TimelineParserForRelationFilters(for relation filters).
# Some code between TimelineParserForCompareExpr and
TimelineParserForEqualityExpr is similar. It can be moved to another base
abstract class. But this might make the code confusing. So have left it as it
is. Would like to know thoughts of others on this.
> Provide a mechanism to represent complex filters and parse them at the REST
> layer
> ----------------------------------------------------------------------------------
>
> Key: YARN-4447
> URL: https://issues.apache.org/jira/browse/YARN-4447
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: YARN-2928
> Reporter: Varun Saxena
> Assignee: Varun Saxena
> Labels: yarn-2928-1st-milestone
> Attachments: YARN-4447-YARN-2928.01.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
