sunchao commented on code in PR #1990:
URL: https://github.com/apache/arrow-rs/pull/1990#discussion_r914389732
##########
arrow/src/compute/kernels/arity.rs:
##########
@@ -78,10 +82,120 @@ where
PrimitiveArray::<O>::from(data)
}
+macro_rules! unary_dict_op {
+ ($array: expr, $op: expr, $value_ty: ty) => {{
+ // Safety justification: Since the inputs are valid Arrow arrays, all
values are
+ // valid indexes into the dictionary (which is verified during
construction)
+
+ let array_iter = unsafe {
+ $array
+ .values()
+ .as_any()
+ .downcast_ref::<$value_ty>()
+ .unwrap()
+ .take_iter_unchecked($array.keys_iter())
+ };
+
+ let values = array_iter.map(|v| v.map(|value| $op(value))).collect();
+
+ Ok(values)
+ }};
+}
+
+/// A helper function that applies an unary function to a dictionary array
with primitive value type.
+fn unary_dict<K, F, T>(array: &DictionaryArray<K>, op: F) ->
Result<PrimitiveArray<T>>
Review Comment:
does this need to be public so it can be used by other mods like
`arithmetic.rs`?
##########
arrow/src/compute/kernels/arity.rs:
##########
@@ -78,10 +82,120 @@ where
PrimitiveArray::<O>::from(data)
}
+macro_rules! unary_dict_op {
+ ($array: expr, $op: expr, $value_ty: ty) => {{
+ // Safety justification: Since the inputs are valid Arrow arrays, all
values are
+ // valid indexes into the dictionary (which is verified during
construction)
+
+ let array_iter = unsafe {
+ $array
+ .values()
+ .as_any()
+ .downcast_ref::<$value_ty>()
+ .unwrap()
+ .take_iter_unchecked($array.keys_iter())
Review Comment:
Hmm, is it possible to directly apply the `op` on dictionary values? if
values are large strings, the current approach will need to first decode the
dictionary and convert it to a "plain" array, and then apply the `op` to each
value in there, which is expensive.
##########
arrow/src/compute/kernels/arity.rs:
##########
@@ -78,10 +82,120 @@ where
PrimitiveArray::<O>::from(data)
}
+macro_rules! unary_dict_op {
+ ($array: expr, $op: expr, $value_ty: ty) => {{
+ // Safety justification: Since the inputs are valid Arrow arrays, all
values are
+ // valid indexes into the dictionary (which is verified during
construction)
+
+ let array_iter = unsafe {
+ $array
+ .values()
+ .as_any()
+ .downcast_ref::<$value_ty>()
+ .unwrap()
+ .take_iter_unchecked($array.keys_iter())
+ };
+
+ let values = array_iter.map(|v| v.map(|value| $op(value))).collect();
+
+ Ok(values)
+ }};
+}
+
+/// A helper function that applies an unary function to a dictionary array
with primitive value type.
+fn unary_dict<K, F, T>(array: &DictionaryArray<K>, op: F) ->
Result<PrimitiveArray<T>>
+where
+ K: ArrowNumericType,
+ T: ArrowPrimitiveType,
+ F: Fn(T::Native) -> T::Native,
+{
+ unary_dict_op!(array, op, PrimitiveArray<T>)
Review Comment:
do we need this macro? I think we can just inline it:
```rust
let array_iter = unsafe {
array
.values()
.as_any()
.downcast_ref::<PrimitiveArray<T>>()
.unwrap()
.take_iter_unchecked(array.keys_iter())
};
let values = array_iter.map(|v| v.map(|value| op(value))).collect();
Ok(values)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
