0lai0 commented on code in PR #5215:
URL: https://github.com/apache/datafusion-comet/pull/5215#discussion_r3699625311
##########
native/spark-expr/src/predicate_funcs/rlike.rs:
##########
@@ -71,7 +79,14 @@ impl RLike {
})
}
- fn is_match(&self, inputs: &StringArray) -> BooleanArray {
+ /// Match the pre-compiled pattern against a string array of any Arrow
string layout.
+ ///
+ /// Keeps the plan-time compiled [`Regex`] rather than calling Arrow's
+ /// `regexp_is_match(_scalar)`, which recompiles the pattern on every
batch.
+ fn is_match<'a, S>(&'a self, inputs: &'a S) -> BooleanArray
+ where
+ &'a S: StringArrayType<'a>,
+ {
Review Comment:
Optional and non-blocking
This follows on from the comment by @andygrove below about `is_nullable()`
Since `is_nullable()` is `logical_null_count() != 0`, the else branch only
runs on an array with no nulls, and `StringArrayType` gives us `iter()`.
`BooleanArray` implements `FromIterator<Option<bool>>`, so both branches
collapse into:
```
fn is_match<'a, S>(&self, inputs: &'a S) -> BooleanArray
where
&'a S: StringArrayType<'a>,
{
inputs.iter().map(|v| v.map(|s| self.pattern.is_match(s))).collect()
}
```
The uncovered branch stops existing rather than needing a test, and null
handling no longer depends on `is_nullable()`.
`process_parse_url `in `url_funcs/parse_url.rs` already uses the same
`StringArrayType` bound and the same iter/collect shape.
Worth noting `ArrayIter`'s docs call interleaved null-mask handling
suboptimal, but relative to `Regex::is_match` I would expect that to be noise
ref: https://docs.rs/arrow/latest/arrow/array/struct.ArrayIter.html
Also, `&'a self` ties the borrow of `self` to the input lifetime and nothing
is borrowed out of it, so plain `&self` would do
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
