u70b3 opened a new issue, #23709:
URL: https://github.com/apache/datafusion/issues/23709
### Describe the bug
The regex kernels' array-array path (regex_match_dyn) does not support
Dictionary(_, Utf8)-encoded needle arrays, so SIMILAR TO (and the ~ regex
operators) fail with an internal error when the value side is
dictionary-encoded and the pattern is a non-scalar array:
```sql
CREATE TABLE t AS SELECT * FROM (VALUES ('user auth failed')) v(s);
CREATE TABLE p AS SELECT * FROM (VALUES ('(auth|login)')) v(pat);
SELECT arrow_cast(t.s, 'Dictionary(Int32, Utf8)') SIMILAR TO p.pat FROM t
CROSS JOIN p;
```
```
Internal error: Data type Dictionary(Int32, Utf8) not supported for
regex_match_dyn on string array.
```
### Expected behavior
The query should evaluate successfully (return true for this data), like the
equivalent LIKE query already does:
```sql
SELECT arrow_cast(t.s, 'Dictionary(Int32, Utf8)') LIKE p.pat FROM t CROSS
JOIN p;
-- returns true
```
### Additional context
- The scalar fast path (regex_match_dyn_scalar) already handles Dictionary
arrays (see datafusion/physical-expr/src/expressions/binary/kernels.rs), and
the LIKE kernels handle dictionary needles in the array-array path, so this is
a gap specific to the regex kernels' array-array dispatch.
- Suggested fix: add a DataType::Dictionary arm to regex_match_dyn that
matches over the dictionary value type (mirroring the scalar path / like
kernels).
- Surfaced during review of #23704 (which fixed the missing type coercion
for SIMILAR TO and turned the former panic paths into this proper error). Not a
regression from that PR — the limitation is pre-existing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
