[
https://issues.apache.org/jira/browse/PARQUET-427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liwei Lin updated PARQUET-427:
------------------------------
Attachment: Parquet-427 v0.1.pdf
[~julienledem][~rdblue] would you take a look at it, please? Any comments are
welcome. Also, who else should we @ ? :-)
> Push predicates into the whole read path
> ----------------------------------------
>
> Key: PARQUET-427
> URL: https://issues.apache.org/jira/browse/PARQUET-427
> Project: Parquet
> Issue Type: Improvement
> Components: parquet-mr
> Affects Versions: 1.8.0, 1.8.1
> Reporter: Liwei Lin
> Assignee: Liwei Lin
> Fix For: 1.9.0
>
> Attachments: Parquet-427 v0.1.pdf
>
>
> Regarding primitive types, there are 3 import primitive types set:
> - the file schema primitive type set
> - the requested schema primitive type set
> - the predicates primitive type set
> Currently the file schema primitive type set and the requested schema
> primitive type set has been pushed into the whole read path, but not the the
> predicates primitive type set.
> This brings some problems like:
> - PARQUET-389, SPARK-11103, SPARK-11434
> - PARQUET-425, PARQUET-426
> - PARQUET-295
> This issue propose to push the predicates primitive type set into the whole
> read path as well. When this is resolved, hopefully the issues listed above
> would also be resolved as well.
> Attached is a change proposal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
