pitrou commented on code in PR #14176: URL: https://github.com/apache/arrow/pull/14176#discussion_r1064826617
########## docs/source/format/Columnar.rst: ########## @@ -765,6 +765,72 @@ application. We discuss dictionary encoding as it relates to serialization further below. +.. _run-end-encoded-layout: + +Run-End Encoded Layout +------------------------- + +Run-end encoding (REE) is a variation of run-length encoding (RLE). These +encodings are well-suited for representing data containing sequences of the +same value, called runs. In run-end encoding, each run is represented as a +value and an integer giving the index in the array where the run ends. + +Any array can be run-end encoded. A run-end encoded array has no buffers +by itself, but has two child arrays. The first one holds a signed integer +called a "run end" for each run. The run ends array can hold either 16, 32, or +64-bit integers. The actual values of each run are held in the second child array. +For the purposes of determining field names and schemas, these child arrays +are prescribed the standard names of **run_ends** and **values** respectively. + +The values in the first child array represent the length of each run. They do +not hold the length of the respective run directly, but the accumulated length +of all runs from the first to the current one, i.e. the logical index where the +current run ends. This allows relatively efficient random access from a logical Review Comment: Yes, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
