mbutrovich commented on code in PR #2099:
URL: https://github.com/apache/datafusion-comet/pull/2099#discussion_r2268042922
##########
native/spark-expr/src/static_invoke/char_varchar_utils/read_side_padding.rs:
##########
@@ -71,44 +101,101 @@ fn spark_read_side_padding2(
}
}
+enum RPadArgument {
+ ConstLength(i32),
+ ColArray(ArrayRef),
+}
+
fn spark_read_side_padding_internal<T: OffsetSizeTrait>(
array: &ArrayRef,
- length: i32,
truncate: bool,
+ rpad_argument: RPadArgument,
) -> Result<ColumnarValue, DataFusionError> {
let string_array = as_generic_string_array::<T>(array)?;
- let length = 0.max(length) as usize;
- let space_string = " ".repeat(length);
+ match rpad_argument {
+ RPadArgument::ColArray(array_int) => {
+ let int_pad_array = array_int.as_primitive::<Int32Type>();
+ let mut str_pad_value_map = HashMap::new();
Review Comment:
I might be missing something obvious: why do we need a `HashMap`? It doubles
the memory usage of this operator. If we iterate through the arrays to build
the `HashMap`, why can't we iterate through the arrays with the builder below
(i.e. `zip` through them or something). This approach would also have the
benefit of preserving input order (though I'm not sure that's required).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
