Hi Micah, The way you described is how parquet-mr works at the write path. Meanwhile, based on the parquet-mr code, it seems that the scenario explained can be read properly. (If we think we have/will have writers that support such scenarios we shall write unit tests for them.)
Cheers, Gabor On Tue, Jun 1, 2021 at 8:52 PM Micah Kornfield <[email protected]> wrote: > I couldn't find anything in the specification on this, but is there any > constraint on the ordering of encoded pages in a column for a row group. > > I think in practice most implementations try to dictionary-encode first and > then fallback to another encoding if the dictionary doesn't yield benefits > or grows too big. But in theory is an ordering of pages described below > possible? > > 1. Dictionary Encoded Page > 2. Plain Encoded Page > 3. Dictionary Encoded Page > > Thanks, > Micah >
