[
https://issues.apache.org/jira/browse/SPARK-56945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Nikoliukin updated SPARK-56945:
---------------------------------------
Description:
`SQLFunctionExpression` is the analyzer's placeholder for a SQL UDF call before
`ResolveSQLFunctions` inlines its body as `SQLScalarFunction`. The placeholder
hard-codes `override def nullable: Boolean = true`, yet it reports `resolved =
true` as soon as its inputs resolve (its `dataType` the user-declared
`returnType`).
This leads to CTE that captures wrong nullability, e.g.
{code:java}
CREATE FUNCTION non_null_one() RETURNS INT RETURN 1;
CREATE FUNCTION wrap_int(x INT) RETURNS INT RETURN x;
WITH cte AS (SELECT wrap_int(non_null_one()) AS x)
SELECT x FROM cte;
-- schema: x: int (nullable = true){code}
> Fix stale nullability in CTE references containing SQL UDFs
> -----------------------------------------------------------
>
> Key: SPARK-56945
> URL: https://issues.apache.org/jira/browse/SPARK-56945
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.2.0
> Reporter: Mikhail Nikoliukin
> Priority: Major
>
> `SQLFunctionExpression` is the analyzer's placeholder for a SQL UDF call
> before `ResolveSQLFunctions` inlines its body as `SQLScalarFunction`. The
> placeholder hard-codes `override def nullable: Boolean = true`, yet it
> reports `resolved = true` as soon as its inputs resolve (its `dataType` the
> user-declared `returnType`).
> This leads to CTE that captures wrong nullability, e.g.
> {code:java}
> CREATE FUNCTION non_null_one() RETURNS INT RETURN 1;
> CREATE FUNCTION wrap_int(x INT) RETURNS INT RETURN x;
> WITH cte AS (SELECT wrap_int(non_null_one()) AS x)
> SELECT x FROM cte;
> -- schema: x: int (nullable = true){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
