Mola-maker opened a new pull request, #12268:
URL: https://github.com/apache/arrow-rs/pull/12268
- Closes #12249
# Rationale for this change
An IPC stream containing a `RunEndEncoded` array with `Utf8View` (or
`BinaryView`) values sliced to zero rows does not read back. In
`encode_record_batch_data`, the buffers and the variadic buffer counts came
from two different arrays: `write_array_data` writes the children of
`unslice_run_array(array_data)`, but `append_variadic_buffer_counts` walked the
original `array_data`. For a zero-row slice, `into_zero_offset_run_array`
returns an empty array whose Utf8View values own no data buffers (0 written),
while the original values still own one (count 1). The reader then fails with
`IpcError("Invalid variadic count 1 for Utf8View column, with 2 buffer(s) left
in the message")`. The same happens when the view array is nested inside the
run values, e.g. `RunEndEncoded<Int32, ListView<Utf8View>>`.
# What changes are included in this PR?
- In `encode_record_batch_data`, RunEndEncoded columns are unsliced before
the variadic counts are computed, so the counts come from the same unsliced
array the writer wrote. `append_variadic_buffer_counts` itself is unchanged.
- Three regression tests in `arrow-ipc/src/reader.rs`: the issue reproducer
exactly (Utf8View values with a non-inlined string, batch sliced to 0 rows),
plus the nested `ListView<Utf8View>` and `List<Utf8View>` variants from the
issue.
# Are these changes tested?
Yes. All three new tests fail on current main with the exact error from the
issue and pass with the fix. The full `arrow-ipc` suite (169 tests) and the
downstream `arrow-array` and `arrow-data` suites (1068 tests) pass, and `cargo
fmt` and `cargo clippy` are clean.
# Are there any significant user-facing changes?
Behavior fix only: zero-row slices of RunEndEncoded arrays with view values
now round-trip correctly. No public API change.
# AI usage disclosure
AI assistance was used to implement the fix and tests, following the
approach described in #12249; the changes were verified with the crate's test
suite, clippy and fmt.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]