I am not in favor of changing allowed types simply to make them consistent. I think the bar for such a change should be substantial end-user benefit.
I don't think new allowed physical types offers any benefit for users but it would make the spec more complicated, and could require non trivial changes in downstream implementations. Andrew On Wed, Jul 9, 2025 at 9:10 AM Raphael Taylor-Davies <r.taylordav...@googlemail.com.invalid> wrote: > > Perhaps we should > > consider making FLBA(4) valid physical for IntType(32) and FLBA(8) valid > > physical for IntType(64). For most readers this shouldn't be much of a > > change. > At least for parquet-rs, where the primitive types have specialized > decoding paths, this would be a relatively non-trivial change. One would > need to lift all the logical type handling machinery that currently > resides within these specialized primitive decoders into some higher > level component. Achievable, but not trivial. > > On 09/07/2025 13:54, Alkis Evlogimenos wrote: > > The only drawback of FLBA(16) is breaking consistency. Perhaps we should > > consider making FLBA(4) valid physical for IntType(32) and FLBA(8) valid > > physical for IntType(64). For most readers this shouldn't be much of a > > change. > > > > On Wed, Jul 9, 2025 at 2:16 PM Antoine Pitrou <anto...@python.org> > wrote: > > > >> On Wed, 9 Jul 2025 05:57:57 -0400 > >> Andrew Lamb <andrewlam...@gmail.com> > >> wrote: > >>> From a Rust perspective, I don't see any significant fundamental > >> difference > >>> between Fixed Length Binary(16) and an Int128. Therefore I also favor > >> using > >>> the existing type rather than adding a new one. > >>> > >>> The fact we already have optimized readers/writers for Fixed Length > >> Binary > >>> means that it would likely be less work to reuse the existing types > >>> rather than support a new i128 type. > >>> > >>> One potential issue with using Fixed Length Binary would be that it is > >> not > >>> consistent with the existing physical types (Int32/Int64/Float/Double, > >> etc) > >>> but I don't see that as a deal breaker. > >> Especially as we already have FLOAT16 defined as a logical type over > >> FLBA(2). > >> > >> Regards > >> > >> Antoine. > >> > >> > >> >
