Ryan Blue created PARQUET-548:
---------------------------------
Summary: Add Java metadata for PageEncodingStats
Key: PARQUET-548
URL: https://issues.apache.org/jira/browse/PARQUET-548
Project: Parquet
Issue Type: New Feature
Components: parquet-mr
Reporter: Ryan Blue
Assignee: Ryan Blue
PARQUET-384 needs to determine whether an entire column chunk is
dictionary-encoded, but it is difficult to detect that case based on the set of
encodings for a column. For 1.0, this can be done by checking for a PLAIN page
because both dictionary pages and dictionary-encoded pages use PLAIN_DICTIONARY
and RLE/BIT_PACKING is only used for repetition and definition levels. But for
2.0, dictionary pages might be using PLAIN and there is no way to tell if a
column has fallen back.
[PageEncodingStats|https://github.com/apache/parquet-format/blob/master/src/main/thrift/parquet.thrift#L449]
were added to the format to solve this problem, so we just need to implement
them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
