[
https://issues.apache.org/jira/browse/DRILL-6489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16510889#comment-16510889
]
ASF GitHub Bot commented on DRILL-6489:
---------------------------------------
arina-ielchiieva closed pull request #1318: DRILL-6489: Fix filter push down
for Hbase & Mapr-DB binary tables when convert function is used in a view
URL: https://github.com/apache/drill/pull/1318
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/CompareFunctionsProcessor.java
b/contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/CompareFunctionsProcessor.java
index 0672b53c09..59df4e2cc7 100644
---
a/contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/CompareFunctionsProcessor.java
+++
b/contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/CompareFunctionsProcessor.java
@@ -22,6 +22,8 @@
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
import org.apache.drill.common.expression.CastExpression;
import org.apache.drill.common.expression.ConvertExpression;
@@ -51,6 +53,10 @@
import com.google.common.collect.ImmutableSet;
public class CompareFunctionsProcessor extends AbstractExprVisitor<Boolean,
LogicalExpression, RuntimeException> {
+
+ // to check that function name starts with convert_from disregarding the
case and has encoding after
+ private static final Pattern convertFromPattern =
Pattern.compile(String.format("^%s(.+)", ConvertExpression.CONVERT_FROM),
Pattern.CASE_INSENSITIVE);
+
private byte[] value;
private boolean success;
private boolean isEqualityFn;
@@ -511,6 +517,17 @@ public Boolean visitSchemaPath(SchemaPath path,
LogicalExpression valueArg) thro
return false;
}
+ @Override
+ public Boolean visitFunctionCall(FunctionCall call, LogicalExpression
valueArg) {
+ Matcher matcher = convertFromPattern.matcher(call.getName());
+ if (matcher.find()) {
+ // convert function call to ConvertExpression
+ ConvertExpression convert = new
ConvertExpression(ConvertExpression.CONVERT_FROM, matcher.group(1),
call.args.get(0), call.getPosition());
+ return visitConvertExpression(convert, valueArg);
+ }
+ return false;
+ }
+
protected static ByteBuf newByteBuf(int size, boolean bigEndian) {
return Unpooled.wrappedBuffer(new byte[size])
.order(bigEndian ? ByteOrder.BIG_ENDIAN : ByteOrder.LITTLE_ENDIAN)
diff --git
a/contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseFilterPushDown.java
b/contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseFilterPushDown.java
index 0e14cb183e..e6eff116f9 100644
---
a/contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseFilterPushDown.java
+++
b/contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseFilterPushDown.java
@@ -780,5 +780,21 @@ public void testDummyColumnsAreAvoided() throws Exception {
runHBaseSQLVerifyCount(sql, 2);
}
+ @Test
+ public void testConvertFromPushDownWithView() throws Exception {
+ test("create view dfs.tmp.pd_view as\n" +
+ "select convert_from(byte_substr(row_key, 1, 8), 'date_epoch_be') as
d\n" +
+ "from hbase.`TestTableCompositeDate`");
+
+ String query = "select d from dfs.tmp.pd_view where d > date '2015-06-13'
and d < DATE '2015-06-18'";
+ String[] expectedPlan = {
+ "startRow=\\\\x00\\\\x00\\\\x01M\\\\xEF\\]\\\\xA0\\\\x00, " +
+ "stopRow=\\\\x00\\\\x00\\\\x01N\\\\x03\\\\xF7\\\\x10\\\\x00, " +
+ "filter=null"};
+ String[] excludedPlan ={"Filter\\("};
+ PlanTestBase.testPlanMatchingPatterns(query, expectedPlan, excludedPlan);
+
+ runHBaseSQLVerifyCount(query, 12);
+ }
}
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Fix filter push down for Hbase & Mapr-DB binary tables when convert function
> is used in a view
> ----------------------------------------------------------------------------------------------
>
> Key: DRILL-6489
> URL: https://issues.apache.org/jira/browse/DRILL-6489
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - HBase, Storage - MapRDB
> Affects Versions: 1.13.0
> Reporter: Arina Ielchiieva
> Assignee: Arina Ielchiieva
> Priority: Major
> Labels: ready-to-commit
> Fix For: 1.14.0
>
>
> Query
> {noformat}
> select convert_from(byte_substr(row_key, 1, 8), 'date_epoch_be') as d
> from hbase.`t`
> where convert_from(byte_substr(row_key, 1, 8), date_epoch_be') = date
> '2015-06-13';
> {noformat}
> returns plan with the push down:
> {noformat}
> 00-00 Screen
> 00-01 Project(d=[CONVERT_FROMDATE_EPOCH_BE(BYTE_SUBSTR($0, 1, 8))])
> 00-02 Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec
> [tableName=TestTableCompositeDate, startRow=\x00\x00\x01M\xEA7D\x00,
> stopRow=\x00\x00\x01M\xEF]\xA0\x00, filter=null], columns=[`row_key`]]])
> {noformat}
> While the same query in a view does not:
> {noformat}
> create view dfs.tmp.v as select convert_from(byte_substr(row_key, 1, 8),
> 'date_epoch_be') as d from hbase.`t`;
> select d from dfs.tmp.v where d = date '2015-06-13';
> {noformat}
> {noformat}
> 00-00 Screen
> 00-01 Project(d=[CONVERT_FROMDATE_EPOCH_BE(BYTE_SUBSTR($0, 1, 8))])
> 00-02 SelectionVectorRemover
> 00-03 Filter(condition=[=(CONVERT_FROMDATE_EPOCH_BE(BYTE_SUBSTR($0,
> 1, 8)), 2015-06-13)])
> 00-04 Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec
> [tableName=TestTableCompositeDate, startRow=null, stopRow=null, filter=null],
> columns=[`row_key`]]])
> {noformat}
> The problem that {{CompareFunctionsProcessor}} waits for
> {{ConvertExpression}} but receives {{FunctionCall}} with convert function. If
> convert function first appears in filter it is re-presented as
> {{ConvertExpression}} (case without view). If convert function first appears
> in select is re-presented as {{FunctionCall}} for convert function (case with
> view). The reason of such difference is the appliance of the
> {{PreProcessLogicalRel}} visitor. The solution in this case would be to check
> {{FunctionCall}} in CompareFunctionsProcessor}} and if this function call for
> convert from function, process it as {{ConvertExpression}}.
> https://github.com/apache/drill/blob/master/contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/CompareFunctionsProcessor.java#L171
> https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/handlers/DefaultSqlHandler.java#L667
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
