fangchenli opened a new pull request, #50479:
URL: https://github.com/apache/arrow/pull/50479
### Rationale for this change
The scalar string predicate/measurement kernels return `NotImplemented` for
`string_view`/`binary_view` input, so evaluating a string predicate (e.g. a
`LIKE` filter) on a view array first requires casting the whole column to
`utf8`/`binary` — a full copy. This closes that gap so the predicates run
directly on view arrays.
Part of the view-kernel effort (umbrella #44336, tracking #39634); follows
grouper keys (#50224) and cast (#50166). Take/filter (#43010) is handled
separately by #50164.
### What changes are included in this PR?
Adds `STRING_VIEW`/`BINARY_VIEW` kernels for the input-view, fixed-output
subset:
- `match_substring`(`_regex`), `match_like`, `starts_with`, `ends_with`
- `find_substring`(`_regex`), `count_substring`(`_regex`)
- `binary_length`, `utf8_length`
- `string_is_ascii`, `ascii_is_*`, `utf8_is_*`
Implementation notes:
- `MatchSubstringImpl` gains an `if constexpr` branch that iterates view
elements
via `ArrayIterator` (the packed-offset fast path is unchanged for base
binary/string). `match_like` and the regex / ignore-case variants route
through
the same path.
- The applicator-based kernels (`find`/`count`/`length`/classifiers) already
handle the view layout, so they only needed registration entries. A small
`StringOffsetType` helper maps the view types to `Int32` output (a view
element
length is int32-sized).
Kernels that **emit** strings/lists (`utf8_upper`/`lower`/`trim*`,
`replace_substring`, `split_pattern`, `utf8_slice_codeunits`, …) are out of
scope
for a follow-up, since constructing view output is the harder half.
### Are these changes tested?
Yes — new `StringViewPredicates.*` cases in `scalar_string_test.cc` cover
empty,
inlined (≤12 byte), out-of-line (>12 byte), sliced, and null values for both
`binary_view` and `utf8_view`, asserting results match the plain
string/binary
semantics.
A `MatchSubstringView` / `MatchSubstringViewCast` benchmark pair quantifies
running the predicate directly on a view array vs. the pre-PR cast-to-utf8
workaround. Release build, ~1M rows of avg-16-byte ASCII, mean of 3 reps:
| Benchmark | Time | Throughput |
| --- | --- | --- |
| `MatchSubstring` — utf8, direct | 28.1 ms | 564 MiB/s |
| `MatchSubstringView` — utf8_view, direct (this PR) | 31.3 ms | 506 MiB/s |
| `MatchSubstringViewCast` — view → cast to utf8 → match (pre-PR workaround)
| 42.0 ms | 378 MiB/s |
Evaluating the predicate directly on the view array is ~1.34x faster than
casting to utf8 first, by skipping the full-column copy. Direct view
evaluation
is ~11% slower than native utf8 (inherent to the view layout's per-element
decode) but avoids the cast entirely.
### Are there any user-facing changes?
Yes — these compute functions now accept `string_view`/`binary_view` input.
The
`compute.rst` type-support tables are updated accordingly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
