[
https://issues.apache.org/jira/browse/DRILL-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16498872#comment-16498872
]
ASF GitHub Bot commented on DRILL-4020:
---------------------------------------
parthchandra closed pull request #309: DRILL-4020: The not-equal operator
returns incorrect results when used on the HBase row key
URL: https://github.com/apache/drill/pull/309
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseFilterBuilder.java
b/contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseFilterBuilder.java
index 0e25fa6605..0e33b2ac7a 100644
---
a/contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseFilterBuilder.java
+++
b/contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseFilterBuilder.java
@@ -62,6 +62,7 @@ public HBaseScanSpec parseTree() {
* remove it since its effect is also achieved through startRow and
stopRow.
*/
if (parsedSpec.filter instanceof RowFilter &&
+ ((RowFilter)parsedSpec.filter).getOperator() != CompareOp.NOT_EQUAL
&&
((RowFilter)parsedSpec.filter).getComparator() instanceof
BinaryComparator) {
parsedSpec.filter = null;
}
diff --git
a/contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseFilterPushDown.java
b/contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseFilterPushDown.java
index 05fb0b7c14..1b5d908bf5 100644
---
a/contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseFilterPushDown.java
+++
b/contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseFilterPushDown.java
@@ -40,6 +40,24 @@ public void testFilterPushDownRowKeyEqual() throws Exception
{
PlanTestBase.testPlanMatchingPatterns(sqlHBase, expectedPlan,
excludedPlan);
}
+ @Test
+ public void testFilterPushDownRowKeyNotEqual() throws Exception {
+ setColumnWidths(new int[] {8, 38, 38});
+ final String sql = "SELECT\n"
+ + " *\n"
+ + "FROM\n"
+ + " hbase.`[TABLE_NAME]` tableName\n"
+ + "WHERE\n"
+ + " row_key <> 'b4'";
+
+ runHBaseSQLVerifyCount(sql, 6);
+
+ final String[] expectedPlan = {".*startRow=, stopRow=, filter=RowFilter
\\(NOT_EQUAL, b4\\).*"};
+ final String[] excludedPlan ={};
+ final String sqlHBase = canonizeHBaseSQL(sql);
+ PlanTestBase.testPlanMatchingPatterns(sqlHBase, expectedPlan,
excludedPlan);
+ }
+
@Test
public void testFilterPushDownRowKeyEqualWithItem() throws Exception {
setColumnWidths(new int[] {20, 30});
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> The not-equal operator returns incorrect results when used on the HBase row
> key
> -------------------------------------------------------------------------------
>
> Key: DRILL-4020
> URL: https://issues.apache.org/jira/browse/DRILL-4020
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - HBase
> Affects Versions: 1.2.0, 1.3.0, 1.4.0, 1.5.0
> Environment: Drill Sandbox
> Reporter: Akihiko Kusanagi
> Priority: Critical
>
> Create a test HBase table:
> {noformat}
> hbase> create 'table', 'f'
> hbase> put 'table', 'row1', 'f:c', 'value1'
> hbase> put 'table', 'row2', 'f:c', 'value2'
> hbase> put 'table', 'row3', 'f:c', 'value3'
> {noformat}
> The table looks like this:
> {noformat}
> 0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM
> hbase.`table`;
> +---------+
> | EXPR$0 |
> +---------+
> | row1 |
> | row2 |
> | row3 |
> +---------+
> 1 row selected (4.596 seconds)
> {noformat}
> However, this query returns incorrect results when a not-equal operator is
> used on the row key:
> {noformat}
> 0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM
> hbase.`table` WHERE row_key <> 'row1';
> +---------+
> | EXPR$0 |
> +---------+
> | row1 |
> | row2 |
> | row3 |
> +---------+
> 1 row selected (0.573 seconds)
> {noformat}
> In the query plan, there is no RowFilter:
> {noformat}
> 00-00 Screen
> 00-01 Project(EXPR$0=[CONVERT_FROMUTF8($0)])
> 00-02 Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec
> [tableName=table, startRow=, stopRow=, filter=null], columns=[`row_key`]]])
> {noformat}
> When the query has multiple not-equal operators, it works fine:
> {noformat}
> 0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM
> hbase.`table` WHERE row_key <> 'row1' AND row_key <> 'row2';
> +---------+
> | EXPR$0 |
> +---------+
> | row3 |
> +---------+
> 1 row selected (0.255 seconds)
> {noformat}
> In the query plan, a FilterList has two RowFilters with NOT_EQUAL operators:
> {noformat}
> 00-00 Screen
> 00-01 Project(EXPR$0=[CONVERT_FROMUTF8($0)])
> 00-02 Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec
> [tableName=table, startRow=, stopRow=, filter=FilterList AND (2/2):
> [RowFilter (NOT_EQUAL, row1), RowFilter (NOT_EQUAL, row2)]],
> columns=[`row_key`]]])
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
