Colin Ma created HIVE-16969:
-------------------------------
Summary: Improvement performance of MapOperator for Parquet
Key: HIVE-16969
URL: https://issues.apache.org/jira/browse/HIVE-16969
Project: Hive
Issue Type: Improvement
Affects Versions: 3.0.0
Reporter: Colin Ma
Assignee: Colin Ma
Fix For: 3.0.0
For a table with many partition files,
MapOperator.cloneConfsForNestedColPruning() will update the
hive.io.file.readNestedColumn.paths many times. The larger value of
hive.io.file.readNestedColumn.paths will cause the poor performance for
ParquetHiveSerDe.processRawPrunedPaths().
So, the unnecessary paths should be appended to
hive.io.file.readNestedColumn.paths.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
