[
https://issues.apache.org/jira/browse/IMPALA-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-2482.
-----------------------------------
Resolution: Won't Do
> Adjust expected Parquet Footer size
> -----------------------------------
>
> Key: IMPALA-2482
> URL: https://issues.apache.org/jira/browse/IMPALA-2482
> Project: IMPALA
> Issue Type: Task
> Components: Backend
> Affects Versions: Impala 2.3.0
> Reporter: Sailesh Mukil
> Assignee: Sailesh Mukil
> Priority: Minor
> Labels: impala, performance
>
> The HdfsParquetScanner::FOOTER_SIZE is the field which holds the expected
> footer size. Since we do not know the exact footer size before hand, we do a
> disk read for at least this much and increase it if we do not reach the magic
> bytes "PAR1".
> This had been set to 100KB until now which would be an over estimation on
> the average case.
> This has to be brought down after some analysis on actual footer sizes of
> large data sets.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
