u70b3 opened a new pull request, #23704:
URL: https://github.com/apache/datafusion/pull/23704
## Which issue does this PR close?
- Closes #22886
## Rationale for this change
`SIMILAR TO` panics with `failed to downcast array` whenever its operands
end up as arrays of different physical types:
```sql
SELECT 'a' SIMILAR TO NULL; -- panics during planning (constant folding)
CREATE TABLE t AS SELECT * FROM (VALUES ('user auth failed')) v(s);
CREATE TABLE p AS SELECT * FROM (VALUES ('(auth|login)')) v(pat);
SELECT arrow_cast(t.s, 'Utf8View') SIMILAR TO p.pat FROM t CROSS JOIN p; --
panics at runtime
```
Root cause: `SIMILAR TO` is planned as a regex binary operator (`RegexMatch`
et al.), whose kernel downcasts both arrays to the left operand's array type.
But unlike `LIKE` and the regex operators (`~`, `~*`, ...), the `TypeCoercion`
analyzer never coerced `Expr::SimilarTo` operands — it was listed in the
"nothing to coerce" arm of `TypeCoercionRewriter` — so any type mismatch
reached the kernel un-normalized and hit a blind `.expect()`. Literal patterns
take a scalar fast path, which is why the common `col SIMILAR TO 'pattern'`
form never showed this.
## What changes are included in this PR?
Two commits:
1. **Coerce `SIMILAR TO` operands in the analyzer**
(`datafusion/optimizer/src/analyzer/type_coercion.rs`)
- Extract the existing `LIKE`-operand coercion into
`coerce_like_operands` (behavior for `LIKE`/`ILIKE` is unchanged, including the
`Dictionary(_, Utf8)` special case and error texts).
- Apply `regex_coercion` to `Expr::SimilarTo` — the same coercion the
physical regex operators it is planned into use
(`expr-common/src/type_coercion/binary.rs:218`). A `NULL` pattern is coerced to
a typed NULL and evaluates to NULL (matching `expr ~ NULL`), and mixed string
types are unified before planning.
- Regression coverage in `type_coercion.slt` (both reproducers, NULL
variants, and the planning-error path) plus analyzer unit tests.
2. **Defense in depth in the regex kernels**
(`datafusion/physical-expr/src/expressions/binary/kernels.rs`)
- Replace `.expect("failed to downcast array")` in
`regexp_is_match_flag!` / `regexp_is_match_flag_scalar!` with `exec_err!`, so
any expression path that bypasses the analyzer (e.g. a hand-built plan going
straight to physical planning) returns an error instead of panicking. Includes
a unit test constructing a mismatched `RegexMatch` directly.
## Are these changes tested?
Yes:
- sqllogictest: 7 new cases in `type_coercion.slt` (issue reproducers,
`NULL`/`NOT SIMILAR TO NULL`, `LargeUtf8` × `Utf8`, incompatible-type planning
error).
- Unit tests: analyzer coercion plans (NULL pattern, cross string types,
error case) and a kernel-level test proving mismatched arrays now error instead
of panic.
- Verified `./dev/rust_lint.sh`, `cargo test -p datafusion-optimizer --lib`,
`cargo test -p datafusion-physical-expr --lib`, and the relevant sqllogictest
files (`type_coercion`, `strings`, `scalar`, `regexp`, `string`) — all green.
## Are there any user-facing changes?
Only the bug fix: queries that previously panicked now either evaluate
correctly (`Utf8View` × `Utf8` etc.) or return NULL / a planning error.
`LIKE`/`ILIKE` behavior is unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
