Owen O'Malley created ORC-74:
--------------------------------
Summary: The ORC writer should suppress the indexes when they are
redundant
Key: ORC-74
URL: https://issues.apache.org/jira/browse/ORC-74
Project: Orc
Issue Type: Improvement
Reporter: Owen O'Malley
Currently we always include column indexes and statistics at three levels:
* file
* stripe
* row group (10k rows)
At each level, there isn't any value if the higher level index is exactly the
same. Thus, if there is a single stripe in a file, we shouldn't have stripe
statistics. If there is a single row group in the stripe, we shouldn't include
the row group indexes.
This should improve use cases like Apache NiFi where each file often has less
than 10k rows.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
