hi Joerg, You can see the logic where we initialize the hybrid RLE-bitpacked decoder in the C++ library here:
https://github.com/apache/parquet-cpp/blob/master/src/parquet/column_reader.cc#L38 * The bit width of values is determined by the maximum repetition or definition level based on the schema. So for non-nested fields (max definition level 1 or 0) the bit width is 1 * The number of bytes in the encoded levels is in the first 4 bytes, so if you want to skip the levels, you can decode the length (see line 46) and then skip that many bytes in the stream - Wes On Mon, Aug 28, 2017 at 10:43 AM, Jörg Anders <[email protected]> wrote: > Hi all! > For some reason I haev to know much more about the parquet format. Meanwhile > I know the column chunk can be found at > FileMetaData.row_groups[x].columns[x].file_offsetIf I compreheded all right > at file_offset starts the repetition level array followed by the definition > leve array followed by thecolumn values. Is there a way > to skip both arrays and to go directly to tha values? How long are both > arrays? I would read them. But how? All I know is: They areBitPacked.. So I > expect there is a MAX VALUE field or perhaps a BITS_PER_VALUE field and if it > is 3 then > 7 2 1 0 3 > is coded > 111010001000011 > But where is MAX-VAUE or BITS_PER_VALUE ? > An URL or pointer to a line in some C (++) or Java code would suffice. > Thank you in advance > Jörg
