Hi ZJ, parquet-mr clearly writes repetition levels and definition levels according to the specification. See the following code references. For V1 pages: https://github.com/apache/parquet-mr/blob/master/parquet-column/src/main/java/org/apache/parquet/column/impl/ColumnWriterV1.java#L60 For V2 pages: https://github.com/apache/parquet-mr/blob/master/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileWriter.java#L655 https://github.com/apache/parquet-mr/blob/master/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ColumnChunkPageWriteStore.java#L221-L225
Regards, Gabor On Fri, May 22, 2020 at 6:35 AM Zhuo Jia Dai <[email protected]> wrote: > I raise this issue https://github.com/JuliaIO/Parquet.jl/issues/60 > > where the official parquet documentation states that repetition levels are > written before definition levels. However, in the Julia Parquet package the > parquet implementation reads definition before the repetition levels and > the author insists on him being right but did not provide further evidence. > > I wanted to double-check this with the parquet dev community? Is it true > that definitions levels need to be written before repetition levels? If > true then the parquet documentation is wrong then I am happy to PR a fix. > > Regards > -- > ZJ > > [email protected] >
