[
https://issues.apache.org/jira/browse/YARN-3863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172005#comment-15172005
]
Varun Saxena commented on YARN-3863:
------------------------------------
Latest patch fixes the comments given by Sangjin.
# I have split TestHBaseTimelineStorage into TestHBaseTimelineStorageApps and
TestHBaseTimelineStorageEntities. Also moved the part related to loading apps
and entities to a separate class.
# Methods to match filters in TimelineStorageUtils have been refactored for
better readability and to club common code together. Have passed on enum to
decide which filter we are trying to match. Suggestions for better name of this
enum ?
# For separation of logic between Generic and Application entity reader, I have
refactored methods which were exclusively in GenericEntityReader but were being
used by ApplicationEntityReader. Now relevant logic will exist both the
classes(with EntityColumnPrefix used in GenericEntityReader and
ApplicationColumnPrefix used in ApplicationEntityReader). Have tried to move
some of the common code used in these methods to utils classes.
# I have also moved previously written methods in GenericEntityReader to read
relations, events,etc. to TimelineStorageUtils. These methods were being used
by ApplicationEntityReader as well in addition to GenericEntityReader.
# Refactored createSingleColValueFiltersByRange() and
createHBaseSingleColValueFilter() so that createSingleColValueFiltersByRange
can call createHBaseSingleColValueFilter().
# Fixed javadoc related comments and made members final in the classes pointed
out.
# Removed preconditions check for filters not being null. Now if filters are
null, I create TimelineEntityFilters object with default values in
augmentParams.
# Used == instead of equals to match enums.
# Changed name of TimelineEqualityFilter and TimelineMultiValEqualityFilter to
TimelineKeyValueFilter and TimelineKeyValuesFilter respectively.
# There was a comment on why getCompoundColQualBytes is being used. I had
missed using it in previous patch. It is to be used for events. If say event to
be fetched is UPDATE_APP with associated info as info1 as key, ts as timestamp
and val1 as value, then the column is of the form {{e!UPDATE_APP=ts=info1}}.
This kind of column within code is referred to as a compound column. The part
after the prefix {{e!}} i.e part with {{=}} as separator is what we want to
construct with getCompoundColQualBytes.
So, if we have to match event filters, the qualifier filter will have to be
matched with prefix {{e!UPDATE_APP=}}. The getCompoundColQualBytes bit for
event filters comes in here.
# For the comments for TimelineReaderWebServicesUtils.java, kindy refer to the
explanation in the comment in which I have detailed what I have done. The
changes here are required.
I could not quite get below comment. I did not make any change on line 448.
Sangjin, can you elaborate. Maybe you meant some other line.
{quote}
(HBaseTimelineWriterImpl.java)
l.448: it should simply be a else if
{quote}
cc [~sjlee0], [~djp]
As the patch is quite big, refer to comment
https://issues.apache.org/jira/browse/YARN-3863?focusedCommentId=15169833&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15169833
to get details of what has been implemented.
> Support complex filters in TimelineReader
> -----------------------------------------
>
> Key: YARN-3863
> URL: https://issues.apache.org/jira/browse/YARN-3863
> Project: Hadoop YARN
> Issue Type: Sub-task
> Affects Versions: YARN-2928
> Reporter: Varun Saxena
> Assignee: Varun Saxena
> Labels: yarn-2928-1st-milestone
> Attachments: YARN-3863-YARN-2928.v2.01.patch,
> YARN-3863-YARN-2928.v2.02.patch, YARN-3863-YARN-2928.v2.03.patch,
> YARN-3863-feature-YARN-2928.wip.003.patch,
> YARN-3863-feature-YARN-2928.wip.01.patch,
> YARN-3863-feature-YARN-2928.wip.02.patch,
> YARN-3863-feature-YARN-2928.wip.04.patch,
> YARN-3863-feature-YARN-2928.wip.05.patch
>
>
> Currently filters in timeline reader will return an entity only if all the
> filter conditions hold true i.e. only AND operation is supported. We can
> support OR operation for the filters as well. Additionally as primary backend
> implementation is HBase, we can design our filters in a manner, where they
> closely resemble HBase Filters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
