[
https://issues.apache.org/jira/browse/YARN-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16263850#comment-16263850
]
Haibo Chen commented on YARN-7213:
----------------------------------
The patch includes changes to the final HBase FilterList that is built from
ATSv2 metrics/configs to retrieve and info/metric/config filters, specifically
it adds FamilyFilters for column families that are covered in
info/metric/config filters but not already in metrics/configs to retrieve.
Copying again the internal discussion from our HBase folks.
> Say you have row1:
> r1 c1 q1 v1
> r1 c1 q2 v2
> r1 c2 – --
> With filters: SCVF(=c1/q2/v2) AND FamilyFilter(=c2)
> The filters will first get applied to r1/c1/q1/v1: the SCVF doesn't match so
> it says to skip this cell and go to the next row. The FamilyFilter >says it
> doesn't match either and it wants to go to the next family. In this
> combination, the FamilyFilter "wins" and the next cell evaluated is >r1/c2.
> Because the SCVF never saw r1/c1/q2, the filterIfMissing kicks in and then
> the whole row is dropped.
The above filter is very similar to what we construct in ATSV2, IIUC. We build
the HBase filters in the following way:
(FilterList based on fields to retrieve AND FilterList based on
conf/info/metric filters)
Say if we want to just retrieve metrics with a config filter, the query will
currently be translated to a filter like SCVF(=c1/q2/v2) AND FamilyFilter(=m),
which is incorrect. Hence, the change around the HBase Filters.
[~varun_saxena] as the author of most of the filter code, can you confirm if
this is indeed an issue? Should we fix this in a separate
jira? CC [~vrushalic] [~rohithsharma].
> [Umbrella] Test and validate HBase-2.0.x with Atsv2
> ---------------------------------------------------
>
> Key: YARN-7213
> URL: https://issues.apache.org/jira/browse/YARN-7213
> Project: Hadoop YARN
> Issue Type: Task
> Reporter: Rohith Sharma K S
> Assignee: Rohith Sharma K S
> Attachments: YARN-7213.prelim.patch, YARN-7213.wip.patch
>
>
> Hbase-2.0.x officially support hadoop-alpha compilations. And also they are
> getting ready for Hadoop-beta release so that HBase can release their
> versions compatible with Hadoop-beta. So, this JIRA is to keep track of
> HBase-2.0 integration issues.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
