yongster opened a new issue, #11083:
URL: https://github.com/apache/arrow-rs/issues/11083
### Is your feature request related to a problem or challenge?
`regexp_is_match` already caches compiled `Regex` values in a
`HashMap<String, Regex>`, but it builds an owned `String` for **every row**
before consulting that cache:
- no flags: `pattern.to_string()`
- with flags: `format!("(?{flag}){pattern}")`
Cache misses also `get` and then `entry`, hashing the key twice. For a
low-cardinality pattern column (one or a few patterns applied to many rows)
Regex compilation is already avoided, but the per-row allocation and copy
remain. On short strings that fixed cost dominates.
The no-flags path is also boxed as `Box<dyn Iterator<Item =
Option<String>>>`, which is unnecessary.
`regexp_is_match_scalar` already compiles once and does not have this
overhead. The array/array kernel should be able to get much closer to that for
repeated patterns.
### Describe the solution you'd like
A focused change to `regexp_is_match` only:
- No-flags path: `HashMap<&str, Regex>` so a cache hit allocates nothing.
- Flags path: borrowed key `(&str, Option<&str>)`; build `(?flags)pattern`
only on `Vacant`.
- Use `HashMap::entry` for a single lookup.
- Specialize the no-flags path instead of a boxed iterator.
- Do **not** change `regexp_match` or the scalar kernel in the same PR
(`regexp_match` is dominated by result construction; a borrowed-key prototype
did not help there).
- Tests for repeated patterns, repeated flags, null/empty patterns, and Utf8
/ LargeUtf8 / Utf8View.
Independent remeasurement on arm64 macOS against `381eea177`, 65,536
one-byte values, one repeated pattern. Prototype output matched the current
kernel.
| Case | Current | Borrowed-key prototype | Speedup |
|---|---:|---:|---:|
| repeated pattern, no flags | 1.906 ms | 1.033 ms | 1.85× |
| repeated pattern + flags `"i"` | 2.980 ms | 1.556 ms | 1.91× |
| `regexp_is_match_scalar` (same pattern) | 250 µs | — | current array path
is 7.6× slower than scalar |
### Describe alternatives you've considered
- Interning owned `String` keys. That still allocates on the first insert of
each distinct pattern, and hits still have to construct the lookup `String`.
- A small linear cache instead of `HashMap`. Fine for tiny cardinality, but
`HashMap<&str, Regex>` is already the natural fit and matches the current
design.
- Folding `regexp_match` into the same PR. Local checks showed no benefit;
keep that kernel separate.
### Additional context
#5235 already removed per-row `Regex` clones. #5246 added a scalar path for
`regexp_match`. Neither addresses the per-row `String` allocated before the
cache lookup in `regexp_is_match`.
The in-tree `regexp_kernels` bench only covers `regexp_match`, not
`regexp_is_match`. A PR should add a repeated-pattern `regexp_is_match` bench.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
