Rich-T-kid opened a new issue, #12341:
URL: https://github.com/apache/arrow-rs/issues/12341

   The RunArray specialization from #12218 optimizes contiguous `copy_rows`, 
but `BatchCoalescer::push_batch_with_indices` can still materialize an 
intermediate taken array before copying it into the accumulator.
   
   A prototype added a RunArray-specific `copy_rows_by_indices` implementation 
that maps logical indices to physical runs, retains the referenced value 
ranges, and merges consecutive output rows that reference the same physical 
run. This can avoid the intermediate `take` allocation without adding 
specialized filter or interleave dispatch.
   
   Local `coalesce_kernels` results for `single_ree` against the exact #12218 
merge base:
   
   | Null density | Selectivity | Base | Prototype | Speedup |
   | ---: | ---: | ---: | ---: | ---: |
   | 0 | 0.001 | 21.157 ms | 2.887 ms | 7.33x |
   | 0 | 0.01 | 4.905 ms | 1.877 ms | 2.61x |
   | 0 | 0.1 | 2.955 ms | 1.811 ms | 1.63x |
   | 0 | 0.8 | 3.006 ms | 1.797 ms | 1.67x |
   | 0.1 | 0.001 | 24.861 ms | 3.072 ms | 8.09x |
   | 0.1 | 0.01 | 8.660 ms | 2.074 ms | 4.18x |
   | 0.1 | 0.1 | 3.597 ms | 2.022 ms | 1.78x |
   | 0.1 | 0.8 | 3.557 ms | 2.023 ms | 1.76x |
   
   The prototype was removed from #12218 to keep that PR focused. A follow-up 
should preserve the existing generic fallback and cover repeated and 
out-of-order indices, nullable values, sliced RunArrays, target-batch 
splitting, Int16/Int32/Int64 run ends, and run-end overflow.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to