Perhaps we should
consider making FLBA(4) valid physical for IntType(32) and FLBA(8) valid
physical for IntType(64). For most readers this shouldn't be much of a
change.
At least for parquet-rs, where the primitive types have specialized
decoding paths, this would be a relatively non-trivial change. One would
need to lift all the logical type handling machinery that currently
resides within these specialized primitive decoders into some higher
level component. Achievable, but not trivial.
On 09/07/2025 13:54, Alkis Evlogimenos wrote:
The only drawback of FLBA(16) is breaking consistency. Perhaps we should
consider making FLBA(4) valid physical for IntType(32) and FLBA(8) valid
physical for IntType(64). For most readers this shouldn't be much of a
change.
On Wed, Jul 9, 2025 at 2:16 PM Antoine Pitrou <anto...@python.org> wrote:
On Wed, 9 Jul 2025 05:57:57 -0400
Andrew Lamb <andrewlam...@gmail.com>
wrote:
From a Rust perspective, I don't see any significant fundamental
difference
between Fixed Length Binary(16) and an Int128. Therefore I also favor
using
the existing type rather than adding a new one.
The fact we already have optimized readers/writers for Fixed Length
Binary
means that it would likely be less work to reuse the existing types
rather than support a new i128 type.
One potential issue with using Fixed Length Binary would be that it is
not
consistent with the existing physical types (Int32/Int64/Float/Double,
etc)
but I don't see that as a deal breaker.
Especially as we already have FLOAT16 defined as a logical type over
FLBA(2).
Regards
Antoine.
