HaoYang670 commented on code in PR #1832:
URL: https://github.com/apache/arrow-rs/pull/1832#discussion_r894301166
##########
arrow/src/compute/kernels/substring.rs:
##########
@@ -182,24 +182,66 @@ pub fn substring_by_char<OffsetSize: OffsetSizeTrait>(
start: i64,
length: Option<u64>,
) -> Result<GenericStringArray<OffsetSize>> {
- Ok(array
- .iter()
- .map(|val| {
- val.map(|val| {
- let char_count = val.chars().count();
- let start = if start >= 0 {
- start.to_usize().unwrap().min(char_count)
- } else {
- char_count - (-start).to_usize().unwrap().min(char_count)
- };
- let length = length.map_or(char_count - start, |length| {
- length.to_usize().unwrap().min(char_count - start)
- });
+ let mut vals = BufferBuilder::<u8>::new(array.value_data().len());
+ let mut offsets = BufferBuilder::<OffsetSize>::new(array.len() + 1);
+ offsets.append(OffsetSize::zero());
+ let length = length.map(|len| len.to_usize().unwrap());
+
+ array.iter().for_each(|val| {
Review Comment:
> https://doc.rust-lang.org/std/str/fn.from_utf8_unchecked.html is free and
safe in this context FWIW
Our `GenericStringIter` is implemented based on this. That's why I guess
`array.iter()` is optimal enough.
BTW, skipping nulls could not be the hotspot in this function, because it is
just a bitwise operation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
