ZHA_Moonlight created HBASE-18879: ------------------------------------- Summary: Hbase FilterList cause KeyOnlyFilter not work Key: HBASE-18879 URL: https://issues.apache.org/jira/browse/HBASE-18879 Project: HBase Issue Type: Bug Components: Filters Affects Versions: 1.2.4 Environment: OS: Red Hat 4.4.7-11 Hadoop: 2.6.4 Hbase: 1.2.4 Reporter: ZHA_Moonlight
when use FilterList and KeyOnlyFilter together, if we put KeyOnlyFilter before FilterList, the KeyOnlyFilter may not work, means it will also grab the cell values: {code:java} List<Filter> filters = new ArrayList<Filter>(); Filter filter1 = new SingleColumnValueFilter(Bytes.toBytes("cf"), Bytes.toBytes("column1"), CompareOp.EQUAL, Bytes.toBytes("value1")); Filter filter2 = new SingleColumnValueFilter(Bytes.toBytes("cf"), Bytes.toBytes("column1"), CompareOp.EQUAL, Bytes.toBytes("value2")); filters.add(filter1); filters.add(filter2); FilterList filterListAll = new FilterList(Operator.MUST_PASS_ALL, new KeyOnlyFilter(), new FilterList(Operator.MUST_PASS_ONE, filters)); {code} use the above code as filter to scan a table, it will return the cells with value instead of only return the key, if we put KeyOnlyFilter after FilterList as following, it works well. {code:java} FilterList filterListAll = new FilterList(Operator.MUST_PASS_ALL, new FilterList(Operator.MUST_PASS_ONE, filters), new KeyOnlyFilter()); {code} the cause should due to the following code at hbase-client FilterList.java {code:java} @Override @edu.umd.cs.findbugs.annotations.SuppressWarnings(value="SF_SWITCH_FALLTHROUGH", justification="Intentional") public ReturnCode filterKeyValue(Cell v) throws IOException { this.referenceKV = v; // Accumulates successive transformation of every filter that includes the Cell: Cell transformed = v; ReturnCode rc = operator == Operator.MUST_PASS_ONE? ReturnCode.SKIP: ReturnCode.INCLUDE; int listize = filters.size(); for (int i = 0; i < listize; i++) { Filter filter = filters.get(i); if (operator == Operator.MUST_PASS_ALL) { if (filter.filterAllRemaining()) { return ReturnCode.NEXT_ROW; } {color:red} ReturnCode code = filter.filterKeyValue(v); {color} switch (code) { // Override INCLUDE and continue to evaluate. case INCLUDE_AND_NEXT_COL: rc = ReturnCode.INCLUDE_AND_NEXT_COL; // FindBugs SF_SWITCH_FALLTHROUGH case INCLUDE: {color:red} transformed = filter.transformCell(transformed);{color} continue; case SEEK_NEXT_USING_HINT: seekHintFilter = filter; return code; default: return code; } } {code} notice the {color:red} label line, first line is a recursive invocation, it will assign a Cell results to the FilterList.transformedKV(we call it A), the results is from the FilterList with 2 SingleColumnValueFilter, so A with contains the cell value, while the second line with return A to the var transformed. back to the following loop, we can see the FilterList return results is var "transformed " which will override in each loop, so the value is determined by the last filter, so the order of KeyOnlyFilter will impact the results. {code:java} Cell transformed = v; ReturnCode rc = operator == Operator.MUST_PASS_ONE? ReturnCode.SKIP: ReturnCode.INCLUDE; int listize = filters.size(); for (int i = 0; i < listize; i++) { Filter filter = filters.get(i); if (operator == Operator.MUST_PASS_ALL) { if (filter.filterAllRemaining()) { return ReturnCode.NEXT_ROW; } ReturnCode code = filter.filterKeyValue(v); switch (code) { // Override INCLUDE and continue to evaluate. case INCLUDE_AND_NEXT_COL: rc = ReturnCode.INCLUDE_AND_NEXT_COL; // FindBugs SF_SWITCH_FALLTHROUGH case INCLUDE: transformed = filter.transformCell(transformed); continue; case SEEK_NEXT_USING_HINT: seekHintFilter = filter; return code; default: return code; } {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)