[
https://issues.apache.org/jira/browse/DRILL-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306157#comment-14306157
]
Jason Altekruse edited comment on DRILL-2022 at 2/4/15 11:03 PM:
-----------------------------------------------------------------
The code change looks good. It would be nice if we could have a unit test for
this, but unfortunately the query provides no information about which reader
was selected. Only setting the use_new_reader option will guarantee a
particular one will be selected, and otherwise this decision is made after
planning has completed, so we cannot check the plan to see which reader will be
used. With this in mind I will verify the functionality on my local machine
through debugging and open an enhancement request to make this execution time
decision testable.
was (Author: jaltekruse):
The code change looks good. It would be nice if we could have a unit test for
this, but unfortunately the query provides no information about which reader
was selected. Only setting the use_new_reader option will guarantee a
particular one will be selected, and otherwise this decision is made after
planning has completed, so we cannot check the plan to see which reader will be
used. With this in mind I will verify the functionality on my local machine
through debugging and open an enhancement request to make this execution time
decision is testable.
> Parquet engine falls back to "new" Parquet reader unnecessarily
> ---------------------------------------------------------------
>
> Key: DRILL-2022
> URL: https://issues.apache.org/jira/browse/DRILL-2022
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - Parquet
> Affects Versions: 0.8.0
> Reporter: Adam Gilmore
> Assignee: Jason Altekruse
> Priority: Minor
> Attachments: DRILL-2022.1.patch.txt
>
>
> The Parquet engine falls back to the "new" Parquet reader whenever a Parquet
> file that is "complex" (i.e. not purely primitive types) is found.
> The engine should still use the faster reader when all the projected columns
> are primitive types and only fall back to the other reader when columns
> containing complex types are selected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
