alamb commented on code in PR #10248:
URL: https://github.com/apache/arrow-rs/pull/10248#discussion_r3501048818
##########
arrow-cast/src/cast/dictionary.rs:
##########
@@ -59,6 +59,24 @@ fn dictionary_to_dictionary_cast<K: ArrowDictionaryKeyType>(
) -> Result<ArrayRef, ArrowError> {
use DataType::*;
+ // Fast path for a nested dictionary source (`Dictionary<K, Dictionary<K2,
V>>`).
+ // Both layers index into the same inner values, so the two index layers
can
+ // be composed into one rather than materializing the values: `take`
gathers
+ // the inner keys through the outer keys and reuses the inner values buffer
+ // untouched, so no value data is rewritten. The flattened single-level
+ // dictionary is then cast to the requested index/value types.
+ if matches!(array.values().data_type(), Dictionary(_, _)) {
Review Comment:
that is pretty fancy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
