[
https://issues.apache.org/jira/browse/YARN-7581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16271360#comment-16271360
]
Haibo Chen commented on YARN-7581:
----------------------------------
We have the relevant data (entity 5 that is supposed to be returned in the
result) in HBase as
| Row Key | Column Family | Column Qualifier | Cell Value |
|
----------------|------------------------|-------------------------|-----------------|
| r1 | c | config_param1 |
value1 |
| r1 | c | config_param2 |
value2 |
| r1 | c | cfg_param1 |
value3 |
| r1 | i | info1 |
cluster1 |
When given the above filter, HBase regionserver happens to evaluate
the FamilyFilter first while iterating over the cells
Quoting analysis from [~appy]
> Looks like HBASE-13122 is the culprit.
> Since we do SVCF on cf "c" and FamilyFilter on "i", earlier scanner was still
> iterating on each cell of cf "c" and at some point found the ones we were
> filtering for.
> But after the change, we skipped the whole cf "c" after seeing the first
> cell, as a result SVCF didn't see other cells and failed to match the row.
> So despite HBASE-13122 being in 1.2.6 also, the difference came because of
> this filter:
{code}
FilterList OR (1/1):[
FilterList AND (5/5):[
FamilyFilter (EQUAL, i),
QualifierFilter (NOT_EQUAL, e!),
QualifierFilter (NOT_EQUAL, i!),
QualifierFilter (NOT_EQUAL, s!),
QualifierFilter (NOT_EQUAL, r!)]]]
{code}
> In 1.2.6, filter.filterKeyValue (filter=AND) will return SKIP_ROW, break out
> of loop and return "rc" which was initialized to SKIP. So we still keep going
> with current StoreScanner.
> But in 2.0.0, filter.internalFiterCell (filter=AND) will return SKIP_ROW, and
> rc will be set to SKIP_ROW before returning. As a result we skip the
> StoreScanner and as a result, all cells in this column family.
> ATSv2 does not construct HBase filters correctly in HBase 2.0
> -------------------------------------------------------------
>
> Key: YARN-7581
> URL: https://issues.apache.org/jira/browse/YARN-7581
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: ATSv2
> Affects Versions: 3.0.0-beta1
> Reporter: Haibo Chen
> Assignee: Haibo Chen
>
> TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters() and
> TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters()
> start to fail after we upgrade HBase to 2.0-alpha4 (To reproduce locally,
> apply YARN-7581.prelim.patch that is attached in YARN-7346 and run the atsv2
> unit tests)
> *Error Message*
> [ERROR] Failures:
> [ERROR]
> TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesConfigFilters:1266
> expected:<2> but was:<0>
> [ERROR]
> TestTimelineReaderWebServicesHBaseStorage.testGetEntitiesMetricFilters:1523
> expected:<1> but was:<0>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
