vinycalves opened a new pull request, #27274:
URL: https://github.com/apache/datafusion/pull/27274
## Which issue does this PR close?
- Closes #26316.
## Rationale for this change
A `NULL` literal of a nested type is not preserved when a logical plan is
unparsed back to SQL. A `NULL` list becomes `[]`, so the generated query
returns an empty array instead of `NULL`. A `NULL` struct, map, or union
returns `NotImplemented("Unsupported scalar: ...")` and the plan cannot be
unparsed at all.
These literals show up in optimized plans after type coercion casts `NULL`
to a
nested type and constant folding turns the cast into a literal, for example
in
`CASE WHEN ... THEN NULL ELSE list_col END` or
`... UNION ALL SELECT NULL`.
This is a focused resubmission of the approach in #26722, which was closed.
The change is limited to null handling and includes unit and plan-level
coverage.
## What changes are included in this PR?
`Unparser::scalar_to_sql` now handles a null list, struct, map, or union
before
the existing arms. Null values of these types are rendered as `NULL`, the
same
way other typed nulls already are.
Non-null struct, map, and union values still return `NotImplemented`, because
rendering them requires dialect-specific syntax and is out of scope for this
issue. Empty lists still render as `[]`, and lists that contain null elements
keep their existing rendering.
## What is the testing strategy for this PR?
- `test_null_nested_scalar_value_to_sql` checks that nulls of all eight
affected types (`List`, `LargeList`, `FixedSizeList`, `ListView`,
`LargeListView`, `Struct`, `Map`, `Union`) unparse as `NULL`.
- `test_non_null_nested_scalar_value_to_sql` checks that empty lists, lists
with null elements, and non-null structs keep their previous behavior.
- `unparse_null_list_literal` checks the plan-to-SQL path and asserts that a
projection with a null list literal becomes `NULL AS l`.
`cargo test -p datafusion-sql` passes (101 unit and 608 integration tests),
and
`cargo clippy -p datafusion-sql --all-targets --all-features -- -D warnings`
passes. `cargo fmt --all -- --check` is clean.
## Are there any user-facing changes?
Yes. Unparsed SQL now contains `NULL` where it previously contained `[]` or
returned an error for null nested-type literals. There are no API changes.
## AI assistance
I used AI assistance to help prepare the change and the tests. I reviewed the
final diff and ran the checks above, and I will respond to review comments
in my
own words.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
