[
https://issues.apache.org/jira/browse/DRILL-6259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16402086#comment-16402086
]
ASF GitHub Bot commented on DRILL-6259:
---------------------------------------
GitHub user arina-ielchiieva opened a pull request:
https://github.com/apache/drill/pull/1173
DRILL-6259: Support parquet filter push down for complex types
Details in [DRILL-6259](https://issues.apache.org/jira/browse/DRILL-6259).
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/arina-ielchiieva/drill DRILL-6259
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/drill/pull/1173.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1173
----
commit 7a694cedc76d76ce062b393ddd30002e8a6ba11a
Author: Arina Ielchiieva <arina.yelchiyeva@...>
Date: 2018-03-13T17:54:25Z
DRILL-6259: Support parquet filter push down for complex types
----
> Support parquet filter push down for complex types
> --------------------------------------------------
>
> Key: DRILL-6259
> URL: https://issues.apache.org/jira/browse/DRILL-6259
> Project: Apache Drill
> Issue Type: Improvement
> Affects Versions: 1.13.0
> Reporter: Arina Ielchiieva
> Assignee: Arina Ielchiieva
> Priority: Major
> Fix For: 1.14.0
>
>
> Currently parquet filter push down is not working for complex types
> (including arrays).
> This Jira aims to implement filter push down for complex types which
> underneath type is among supported simple types for filter push down. For
> instance, currently Drill does not support filter push down for varchars,
> decimals etc. Though once Drill will start support, this support will be
> applied for complex type automatically.
> Complex fields will be pushed down the same way regular fields are, except
> for one case with arrays.
> Query with predicate {{where users.hobbies_ids[2] is null}} won't be able to
> push down because we are not able to determine exact number of nulls in
> arrays fields.
> {{Consider [1, 2, 3]}} vs {{[1, 2]}} if these arrays are in different files.
> Statistics for the second case won't show any nulls but when querying from
> two files, in terms of data the third value in array is null.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
