MichaHoffmann opened a new pull request, #11715: URL: https://github.com/apache/arrow-rs/pull/11715
( after #11714 ) # Rationale for this change Interleaving List/LargeList with Struct children expands every selected child range into (array, row) pairs, then gathers each field separately. For primitive fields this loses the contiguous ranges already available from the parent list and allocates scratch proportional to child count. # What changes are included in this PR? Replay borrowed parent ranges through Struct children supported by the primitive dispatcher, sharing typed range copying and bitmap assembly with the existing primitive-list path. This removes the child-index vector without allocating a separate span vector; output values still copy. Preserve parent, Struct and field validity independently, including hidden payloads, slice offsets, field metadata and parameterized types. Keep the original fallback for unsupported Struct fields and selections with at most one child per selected parent on average. Offset overflow is still checked before copying. No public API or new unsafe code. On an i7-13800H with Rust 1.98 and x86-64-v3, fresh baseline/patched/ patched/baseline runs of the added Criterion benchmarks measured these changes in mean elapsed time: | Children per list | Non-null Struct | Nullable Struct | | --- | ---: | ---: | | 1 | +3.5% | +4.1% | | 15 | -59.6% | -52.9% | | 90 | -62.4% | -66.5% | | Mixed | -67.7% | -73.0% | Each run used 30 samples, 1 s warmup and a 3 s measurement target per case. Primitive-list control means ranged from -9.2% to +0.2%. Material run-to-run drift remains; these kernel measurements do not establish a universal or end-to-end speedup. The singleton tradeoff remains visible. # Are these changes tested? Nine added tests cover independent validity layers and hidden values, combined slices, repeated selections, empty and zero-field Structs, primitive types and metadata, unsupported fields, early offset overflow, and shared primitive FixedSizeList copying. Compare against the existing expanded-index path and independently check offsets and raw float bits. The scoped main-source test run passed 58 tests. Local validation used Rust 1.98; upstream's pinned 1.99.1 toolchain remains for CI coverage. # Are there any significant user-facing changes? No API or logical-result changes. # AI usage AI assistance was used for implementation, tests, benchmarking and independent source review. Human review and ownership remain with the submitter. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
