[
https://issues.apache.org/jira/browse/DRILL-5797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16194306#comment-16194306
]
ASF GitHub Bot commented on DRILL-5797:
---------------------------------------
GitHub user dprofeta opened a pull request:
https://github.com/apache/drill/pull/976
DRILL-5797: Choose parquet reader from read columns
ParquetRecordReader is not able to read complex columns. However it is
able to read simple columns in a file containing complex
columns. Instead of looking at the file to choose the reader, we
now choose which reader to use based on what columns is asked and if
they are simple or not.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dprofeta/drill DRILL-5797
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/drill/pull/976.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #976
----
commit 9669dd2c0c61e56c76bc9939c4f1c01aab908baf
Author: Damien Profeta <[email protected]>
Date: 2017-10-06T08:40:22Z
DRILL-5797: Choose parquet reader from read columns
ParquetRecordReader is not able to read complex columns. However it is
able to read simple columns in a file containing complex
columns. Instead of looking at the file to choose the reader, we
now choose which reader to use based on what columns is asked and if
they are simple or not.
----
> Use more often the new parquet reader
> -------------------------------------
>
> Key: DRILL-5797
> URL: https://issues.apache.org/jira/browse/DRILL-5797
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - Parquet
> Reporter: Damien Profeta
> Assignee: Damien Profeta
> Fix For: 1.12.0
>
>
> The choice of using the regular parquet reader of the optimized one is based
> of what type of columns is in the file. But the columns that are read by the
> query doesn't matter. We can increase a little bit the cases where the
> optimized reader is used by checking is the projected column are simple or
> not.
> This is an optimization waiting for the fast parquet reader to handle complex
> structure.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
