tianshuo created PARQUET-61:
-------------------------------
Summary: Avoid fixing protocol events when there is not required
field missing
Key: PARQUET-61
URL: https://issues.apache.org/jira/browse/PARQUET-61
Project: Parquet
Issue Type: Improvement
Reporter: tianshuo
Assignee: tianshuo
Currently, due to the projection pushdown, in the ThriftRecordConverter, it
will check if there is any protocol events missing for the required fields and
fix them if there is. This happens for all the records.
The fixing of missing protocol can be done in a more precise condition to
improve performance:
Only when the requested schema missing some required fields that are present in
the full schema
So even if there a projection, as long as the projection is not getting rid of
the required field, the protocol events amender will not be called.
https://github.com/apache/incubator-parquet-mr/pull/28
--
This message was sent by Atlassian JIRA
(v6.2#6252)
