linhongyu510 opened a new pull request, #10971: URL: https://github.com/apache/arrow-rs/pull/10971
# Which issue does this PR close? - Closes #10876. # Rationale for this change `UnionDecoderBranches::counts` tracks the next dense-union offset for each branch. `flush` drained the type IDs, offsets, and child decoders but left these counters at the previous batch totals, so the next batch referenced positions beyond its newly emptied children. # What changes are included in this PR? - Reset all per-branch union counters after `UnionArray::try_new` succeeds. - Add a regression test that decodes and flushes two consecutive batches through the same dense union decoder and verifies each batch starts at offset zero. The reset intentionally happens only after successful array construction, so a failed flush does not additionally mutate the counters. # Are these changes tested? Yes. TDD evidence on current `main`: - Before the production change, the second flush failed with `Offsets must be non-negative and within the length of the Array`. - After the change, the focused regression passes. - `cargo test -p arrow-avro --all-features`: 487 passed. - Doc tests: 27 passed, 1 repository-marked ignored. - `cargo fmt --all -- --check`: passed. - `cargo clippy -p arrow-avro --all-targets --all-features -- -D warnings`: passed. - `git diff --check`: passed. The first all-features run had 399 passes and 88 failures because the official `testing` submodule was not initialized; every failure referenced missing `testing/data/avro` fixtures. After `git submodule update --init --depth 1 testing`, the complete suite passed as reported above. # Are there any user-facing changes? Yes. A decoder containing a multi-branch Avro union can now emit multiple batches without producing stale dense-union offsets. There are no API changes. # AI assistance I used AI assistance to investigate the decoder state, implement the focused regression and fix, and prepare this description. I reviewed the complete diff and verified all reported commands and outputs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
