DKER2 opened a new pull request, #26196: URL: https://github.com/apache/datafusion/pull/26196
## Which issue does this PR close? - Part of #13232. ## Rationale for this change `EliminateOuterJoin` can rewrite an outer join to a cheaper join type when a filter rejects the NULL-padded rows the outer join produces. For a filter that calls a scalar function, it can only prove that when the function is marked strict (`ScalarUDFImpl::is_strict`, added in #23148). So far only the math functions are marked (#23527), so a query like this keeps its `LEFT JOIN` even though the filter can never pass for unmatched rows: ```sql SELECT t1.a FROM t1 LEFT JOIN t2 ON t1.a = t2.x WHERE upper(t2.z) = 'P'; ``` With this PR the join above becomes an `INNER JOIN`, which also lets the filter be pushed down to the `t2` scan. ## What changes are included in this PR? Marks the built-in string functions that return NULL whenever any argument is NULL as strict: - `string` module: `ascii`, `bit_length`, `btrim`, `chr`, `contains`, `ends_with`, `levenshtein`, `lower`, `ltrim`, `octet_length`, `repeat`, `replace`, `rtrim`, `split_part`, `starts_with`, `to_hex`, `upper` - `unicode` module: `character_length`, `find_in_set`, `initcap`, `left`, `lpad`, `reverse`, `right`, `rpad`, `strpos`, `substr`, `substr_index`, `translate` Not marked: - `concat` and `concat_ws`, because they skip NULL arguments instead of returning NULL - `uuid`, because it takes no arguments There are no changes to how any function executes; each one only gains an `is_strict` override returning `true`. ## What is the testing strategy for this PR? - Unit tests `strict_string_functions_propagate_nulls` and `strict_unicode_functions_propagate_nulls`. They follow the existing `strict_math_functions_propagate_nulls` test in `datafusion/functions/src/math/mod.rs` from #23527: for each function they assert `is_strict()` and then call it with every combination of NULL scalar arguments, checking the result is NULL exactly when an argument is NULL. Functions with optional arguments (`btrim`, `ltrim`, `rtrim`, `lpad`, `rpad`, `substr`) are covered at each arity, and `octet_length` is covered for both string and binary input. - `sqllogictest` cases in `eliminate_outer_join.slt` (new "Strict string function matrix" section), mirroring the math section: LEFT, RIGHT and FULL joins, unary and binary functions, and the nullable column in either argument position, each with the plan and the query result. A `concat` case shows a non-strict function keeps the `LEFT JOIN` and that its NULL-padded rows do pass the filter. Like the math test, the unit tests use scalar arguments only. Array, `LargeUtf8`, `Utf8View` and dictionary-encoded inputs were checked locally for all 29 functions with a broader version of the test, which is not included here to keep the tests consistent with the existing one. ## Are there any user-facing changes? Queries with a null-rejecting filter over one of these functions on the nullable side of an outer join may now get a more efficient plan. There are no public API changes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
