[
https://issues.apache.org/jira/browse/DRILL-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529423#comment-16529423
]
ASF GitHub Bot commented on DRILL-4020:
---------------------------------------
asfgit closed pull request #309: DRILL-4020: The not-equal operator returns
incorrect results when used on the HBase row key
URL: https://github.com/apache/drill/pull/309
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseFilterBuilder.java
b/contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseFilterBuilder.java
index 8d2e8ffd7c..6e1efe512c 100644
---
a/contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseFilterBuilder.java
+++
b/contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseFilterBuilder.java
@@ -61,6 +61,7 @@ public HBaseScanSpec parseTree() {
* remove it since its effect is also achieved through startRow and
stopRow.
*/
if (parsedSpec.filter instanceof RowFilter &&
+ ((RowFilter)parsedSpec.filter).getOperator() != CompareOp.NOT_EQUAL
&&
((RowFilter)parsedSpec.filter).getComparator() instanceof
BinaryComparator) {
parsedSpec.filter = null;
}
diff --git
a/contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseFilterPushDown.java
b/contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseFilterPushDown.java
index 0e14cb183e..c17b00ee62 100644
---
a/contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseFilterPushDown.java
+++
b/contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseFilterPushDown.java
@@ -44,6 +44,24 @@ public void testFilterPushDownRowKeyEqual() throws Exception
{
PlanTestBase.testPlanMatchingPatterns(sqlHBase, expectedPlan,
excludedPlan);
}
+ @Test
+ public void testFilterPushDownRowKeyNotEqual() throws Exception {
+ setColumnWidths(new int[] {8, 38, 38});
+ final String sql = "SELECT\n"
+ + " *\n"
+ + "FROM\n"
+ + " hbase.`[TABLE_NAME]` tableName\n"
+ + "WHERE\n"
+ + " row_key <> 'b4'";
+
+ runHBaseSQLVerifyCount(sql, 7);
+
+ final String[] expectedPlan = {".*startRow=, stopRow=, filter=RowFilter
\\(NOT_EQUAL, b4\\).*"};
+ final String[] excludedPlan ={};
+ final String sqlHBase = canonizeHBaseSQL(sql);
+ PlanTestBase.testPlanMatchingPatterns(sqlHBase, expectedPlan,
excludedPlan);
+ }
+
@Test
public void testFilterPushDownRowKeyEqualWithItem() throws Exception {
setColumnWidths(new int[] {20, 30});
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> The not-equal operator returns incorrect results when used on the HBase row
> key
> -------------------------------------------------------------------------------
>
> Key: DRILL-4020
> URL: https://issues.apache.org/jira/browse/DRILL-4020
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - HBase
> Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.5.0
> Environment: Drill Sandbox
> Reporter: Akihiko Kusanagi
> Assignee: Akihiko Kusanagi
> Priority: Critical
> Labels: ready-to-commit
> Fix For: 1.14.0
>
>
> Create a test HBase table:
> {noformat}
> hbase> create 'table', 'f'
> hbase> put 'table', 'row1', 'f:c', 'value1'
> hbase> put 'table', 'row2', 'f:c', 'value2'
> hbase> put 'table', 'row3', 'f:c', 'value3'
> {noformat}
> The table looks like this:
> {noformat}
> 0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM
> hbase.`table`;
> +---------+
> | EXPR$0 |
> +---------+
> | row1 |
> | row2 |
> | row3 |
> +---------+
> 1 row selected (4.596 seconds)
> {noformat}
> However, this query returns incorrect results when a not-equal operator is
> used on the row key:
> {noformat}
> 0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM
> hbase.`table` WHERE row_key <> 'row1';
> +---------+
> | EXPR$0 |
> +---------+
> | row1 |
> | row2 |
> | row3 |
> +---------+
> 1 row selected (0.573 seconds)
> {noformat}
> In the query plan, there is no RowFilter:
> {noformat}
> 00-00 Screen
> 00-01 Project(EXPR$0=[CONVERT_FROMUTF8($0)])
> 00-02 Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec
> [tableName=table, startRow=, stopRow=, filter=null], columns=[`row_key`]]])
> {noformat}
> When the query has multiple not-equal operators, it works fine:
> {noformat}
> 0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM
> hbase.`table` WHERE row_key <> 'row1' AND row_key <> 'row2';
> +---------+
> | EXPR$0 |
> +---------+
> | row3 |
> +---------+
> 1 row selected (0.255 seconds)
> {noformat}
> In the query plan, a FilterList has two RowFilters with NOT_EQUAL operators:
> {noformat}
> 00-00 Screen
> 00-01 Project(EXPR$0=[CONVERT_FROMUTF8($0)])
> 00-02 Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec
> [tableName=table, startRow=, stopRow=, filter=FilterList AND (2/2):
> [RowFilter (NOT_EQUAL, row1), RowFilter (NOT_EQUAL, row2)]],
> columns=[`row_key`]]])
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
