Rich-T-kid opened a new issue, #11940:
URL: https://github.com/apache/arrow-rs/issues/11940

   ### Is your feature request related to a problem or challenge?
   
   Follow-up to #11743 and #11174.
   
   #11743 adds a specialized in-progress coalescer for byte dictionaries, but 
the indexed-copy path introduced by #11174 will still use the default 
`InProgressArray::copy_rows_by_indices` implementation. That implementation 
calls `take` to materialize an intermediate dictionary before copying it into 
the coalescer.
   
   ### Describe the solution you'd like
   
   Add `copy_rows_by_indices` support to `InProgressDictionaryArray` for 
`Dictionary<K, Utf8 | LargeUtf8 | Binary | LargeBinary>`.
   
   For a pointer-identical values array, gather the selected primitive keys and 
validity directly into the in-progress buffers while reusing the values `Arc`. 
This should avoid the intermediate `take` allocation while preserving repeated 
and out-of-order indices, null keys, bounds validation, and target-batch 
chunking. Inputs with a different values mapping must continue to use a correct 
fallback.
   
   Use the dictionary `take` cases in `coalesce_kernels` to compare the 
specialized path with the existing materialized path for both shared and 
changing values arrays.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to