[
https://issues.apache.org/jira/browse/PARQUET-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15218715#comment-15218715
]
Fabrizio Milo commented on PARQUET-575:
---------------------------------------
Thanks. It does make sense and I was able to reverse engineer it. But this
arrangement is not explicit in any way in the parquet-format documentation
https://github.com/apache/parquet-format
I think that a clarification would help.
Also seems there is a lot of stagnant pull requests for the format:
https://github.com/apache/parquet-format/pulls
should I open another ticket in JIRA ?
> Different RLE Encoding Specification
> -------------------------------------
>
> Key: PARQUET-575
> URL: https://issues.apache.org/jira/browse/PARQUET-575
> Project: Parquet
> Issue Type: Improvement
> Reporter: Fabrizio Milo
> Priority: Trivial
>
> In the parquet-format specification
> https://github.com/Parquet/parquet-format/blob/master/Encodings.md#run-length-encoding--bit-packing-hybrid-rle--3
> is written that the RLE encoding starts with
> ```
> rle-bit-packed-hybrid: <length> <encoded-data>
> length := length of the <encoded-data> in bytes stored as 4 bytes little
> endian
> ```
> while in the cpp implementation there is this description
> https://github.com/apache/parquet-cpp/blob/master/src/parquet/util/rle-encoding.h#L42
> and the implementation seems to follow that specification
> which does not include the initial <length> <encoded-data>
> https://github.com/apache/parquet-cpp/blob/master/src/parquet/util/rle-encoding.h#L272
> So which one is the correct? seems that the parquet-format is the wrong one.
> DataPage.definitionLevels uses RLE and none of the example format files seem
> to have that initial <length> <encoded-data>
> Also the use of both names `literal` and `bit-encoding` is confusing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
