[
https://issues.apache.org/jira/browse/PARQUET-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15215467#comment-15215467
]
Wes McKinney commented on PARQUET-575:
--------------------------------------
[~fmilo] parquet-cpp handles this prefix part of the RLE-hybrid-bit-packed
format outside of {{RleDecoder}}
https://github.com/apache/parquet-cpp/blob/master/src/parquet/column/levels.h#L112
> Different RLE Encoding Specification
> -------------------------------------
>
> Key: PARQUET-575
> URL: https://issues.apache.org/jira/browse/PARQUET-575
> Project: Parquet
> Issue Type: Improvement
> Reporter: Fabrizio Milo
> Priority: Trivial
>
> In the parquet-format specification
> https://github.com/Parquet/parquet-format/blob/master/Encodings.md#run-length-encoding--bit-packing-hybrid-rle--3
> is written that the RLE encoding starts with
> ```
> rle-bit-packed-hybrid: <length> <encoded-data>
> length := length of the <encoded-data> in bytes stored as 4 bytes little
> endian
> ```
> while in the cpp implementation there is this description
> https://github.com/apache/parquet-cpp/blob/master/src/parquet/util/rle-encoding.h#L42
> and the implementation seems to follow that specification
> which does not include the initial <length> <encoded-data>
> https://github.com/apache/parquet-cpp/blob/master/src/parquet/util/rle-encoding.h#L272
> So which one is the correct? seems that the parquet-format is the wrong one.
> DataPage.definitionLevels uses RLE and none of the example format files seem
> to have that initial <length> <encoded-data>
> Also the use of both names `literal` and `bit-encoding` is confusing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
