On Tue, 7 Jul 2026 at 17:18, Antoine Pitrou <[email protected]> wrote: > > Le 06/07/2026 à 23:29, Gunnar Morling a écrit : > >>> 2. Without the logical type (and with a little bit extra complexity) a > >>> smart enough *reader* can walk the def/rep levels before decoding, infer > > > >> At the cost of higher implementation complexity and maintenance cost. > >> Does any mainstream open source implementation of Parquet do this? > > > > Triggered by the conversation on the call last week, I implemented > > pretty much this in Hardwood [1]. > > Great, thank you. `FixedSizeListDetector.java` is highly non-trivial and > definitely has a maintenance cost. Though part of it seems about not > having a RLE parser abstraction available.
Yes, I think we all agree that a dedicated type will make maintainers' lives much easier and is the right solution eventually. But until that has landed, I think there's some juice worth the squeeze here. > > Regards > > Antoine. > >
