hcrosse opened a new pull request, #1556: URL: https://github.com/apache/arrow-go/pull/1556
### Rationale for this change Reading or writing a `FIXED_LEN_BYTE_ARRAY` column with `DELTA_BYTE_ARRAY` panicked with `unimplemented encoding type`. ### What changes are included in this PR? `DeltaByteArrayFixedLenByteArrayEncoder` and `DeltaByteArrayFixedLenByteArrayDecoder` reuse the `BYTE_ARRAY` delta codec, since the layout does not depend on the physical type. Arrow C++ also shares one implementation: https://github.com/apache/arrow/blob/976779b2aac587f51b87c3bd2e6ae4c36f75ab6c/cpp/src/parquet/encoder.cc#L1694-L1704 https://github.com/apache/arrow/blob/976779b2aac587f51b87c3bd2e6ae4c36f75ab6c/cpp/src/parquet/decoder.cc#L2042-L2049 The `BYTE_ARRAY` decoder checks FLBA lengths before consuming values, with the C++ error: https://github.com/apache/arrow/blob/976779b2aac587f51b87c3bd2e6ae4c36f75ab6c/cpp/src/parquet/decoder.cc#L2294-L2299 The encoder rejects wrong lengths too, whereas plain encoding truncates long values. `WriteBatch` and `WriteBatchSpacedWithError` return that error. ### Are these changes tested? `TestDeltaByteArrayFixedLenMatchesByteArray` compares bytes with `BYTE_ARRAY` delta. `TestDeltaByteArrayFixedLenDecoderRejectsWrongLength` and `TestFixedLenByteArrayDeltaByteArrayRejectsWrongLength` cover wrong lengths. All new cases fail on main and pass with this change. pyarrow 26.0.1 and this change read each other's nullable `FixedSizeBinary(16)` files. With Go 1.26.10, vet and race tests for the changed packages, full `go test -race ./parquet/...`, `golangci-lint` 2.13.2 and the Release Audit Tool passed. Decoding benchmarks match `BYTE_ARRAY` delta. Encoding is about 2% slower, from the separate pass that checks lengths before encoding. ### Are there any user-facing changes? Yes. Fixed size binary, decimal and float16 columns can use `DELTA_BYTE_ARRAY`. Closes https://github.com/apache/arrow-go/issues/1537. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
