hcrosse opened a new pull request, #1556:
URL: https://github.com/apache/arrow-go/pull/1556

   ### Rationale for this change
   
   Reading or writing a `FIXED_LEN_BYTE_ARRAY` column with `DELTA_BYTE_ARRAY` 
panicked with `unimplemented encoding type`.
   
   ### What changes are included in this PR?
   
   `DeltaByteArrayFixedLenByteArrayEncoder` and 
`DeltaByteArrayFixedLenByteArrayDecoder` reuse the `BYTE_ARRAY` delta codec, 
since the layout does not depend on the physical type. Arrow C++ also shares 
one implementation:
   
   
https://github.com/apache/arrow/blob/976779b2aac587f51b87c3bd2e6ae4c36f75ab6c/cpp/src/parquet/encoder.cc#L1694-L1704
   
https://github.com/apache/arrow/blob/976779b2aac587f51b87c3bd2e6ae4c36f75ab6c/cpp/src/parquet/decoder.cc#L2042-L2049
   
   The `BYTE_ARRAY` decoder checks FLBA lengths before consuming values, with 
the C++ error:
   
   
https://github.com/apache/arrow/blob/976779b2aac587f51b87c3bd2e6ae4c36f75ab6c/cpp/src/parquet/decoder.cc#L2294-L2299
   
   The encoder rejects wrong lengths too, whereas plain encoding truncates long 
values. `WriteBatch` and `WriteBatchSpacedWithError` return that error.
   
   ### Are these changes tested?
   
   `TestDeltaByteArrayFixedLenMatchesByteArray` compares bytes with 
`BYTE_ARRAY` delta. `TestDeltaByteArrayFixedLenDecoderRejectsWrongLength` and 
`TestFixedLenByteArrayDeltaByteArrayRejectsWrongLength` cover wrong lengths. 
All new cases fail on main and pass with this change.
   
   pyarrow 26.0.1 and this change read each other's nullable 
`FixedSizeBinary(16)` files.
   
   With Go 1.26.10, vet and race tests for the changed packages, full `go test 
-race ./parquet/...`, `golangci-lint` 2.13.2 and the Release Audit Tool passed.
   
   Decoding benchmarks match `BYTE_ARRAY` delta. Encoding is about 2% slower, 
from the separate pass that checks lengths before encoding.
   
   ### Are there any user-facing changes?
   
   Yes. Fixed size binary, decimal and float16 columns can use 
`DELTA_BYTE_ARRAY`.
   
   Closes https://github.com/apache/arrow-go/issues/1537.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to