zeroshade opened a new pull request, #1382: URL: https://github.com/apache/arrow-go/pull/1382
### Rationale for this change `pqarrow` wrote incorrect data for `FixedSizeList` columns without returning an error: - `fixedSizeRangeSelector` computed child ranges as `idx * listSize`, ignoring the parent array's offset. A sliced `FixedSizeList` (e.g. `NewSlice(arr, 1, 2)` selecting `[[20, 21]]`) was written as the leading rows of the child array (`[[10, 11]]`). Because the writer slices columns per row group, any unsliced `FixedSizeList` column spanning more than one row group was affected too: `[[10, 11], [20, 21]]` with row-group size 1 read back as `[[10, 11], [10, 11]]`. - Null `FixedSizeList` slots still occupy child storage, so a null list between non-null lists produces several disjoint visited leaf ranges. The writer rejected this with `lists with non-zero length null components are not supported`. Thanks to ChipLab (Donmakino LLC) for reporting the offset issue. ### What changes are included in this PR? - `fixedSizeRangeSelector` carries the parent offset and computes `(offset + idx) * listSize`. - `arrowColumnWriter.Write` concatenates multiple visited leaf ranges into the values passed to `WriteArrowToColumn` (the single-range case remains a zero-copy slice). ### Are these changes tested? Yes. `TestWriteFixedSizeListHonorsOffset` round-trips flat, element-null, list-null, nested, and nested-with-nulls `FixedSizeList` columns, both sliced and unsliced, with one row per row group and a single row group. A file containing null lists written by Go was also read back correctly with PyArrow. ### Are there any user-facing changes? Yes: sliced or multi-row-group `FixedSizeList` columns are now written with the correct values, and `FixedSizeList` columns with null lists between non-null lists can now be written. **This PR contains a "Critical Fix".** Previously, writing such columns silently produced incorrect data. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
