Liwei Lin created PARQUET-427:
---------------------------------
Summary: Push predicates into the whole read path
Key: PARQUET-427
URL: https://issues.apache.org/jira/browse/PARQUET-427
Project: Parquet
Issue Type: Improvement
Components: parquet-mr
Affects Versions: 1.8.0, 1.8.1
Reporter: Liwei Lin
Assignee: Liwei Lin
Fix For: 1.9.0
Regarding primitive types, there are 3 import primitive types set:
- the file schema primitive type set
- the requested schema primitive type set
- the predicates primitive type set
Currently the file schema primitive type set and the requested schema primitive
type set has been pushed into the whole read path, but not the the predicates
primitive type set.
This brings some problems like:
- PARQUET-389, SPARK-11103, SPARK-11434
- PARQUET-425, PARQUET-426
- PARQUET-295
This issue propose to push the predicates primitive type set into the whole
read path as well. When this is resolved, hopefully the issues listed above
would also be resolved as well.
Attached is a change proposal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
