ninglinLiu commented on PR #11925: URL: https://github.com/apache/arrow-rs/pull/11925#issuecomment-6435025715
Makes sense, added it here in 9f1e1c97c. `merge_kernels` now has an `int8_dictionaries` group with two array-vs-array cases over the usual masks: two dictionaries of 50 shared values (the concatenated dictionary still fits Int8 keys, so this is the existing `MutableArrayData` path) and two of 100 shared values (concatenation overflows, so this goes through the new dictionary-merging fallback). Locally the fallback is roughly 3x slower on the mixed masks (e.g. 50% true: ~19µs vs ~59µs for 8192 rows), which gives a baseline for remapping the keys directly later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
