[
https://issues.apache.org/jira/browse/SPARK-60029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Jie updated SPARK-60029:
-----------------------------
Description:
A column DEFAULT that uses NULLIF or BETWEEN is rejected with
INVALID_DEFAULT_VALUE.NOT_CONSTANT even when every argument is a literal, for
example `CREATE TABLE t (id INT, c INT DEFAULT nullif(1, 3)) USING parquet` or
`DEFAULT (1 + 1) BETWEEN 1 AND 3`, while `SELECT nullif(nullif(1, 2), 3)`
returns 1.
NullIf and Between build a With expression unless
spark.sql.alwaysInlineCommonExpr is set. ResolveDefaultColumnsUtil.analyze runs
FinishAnalysis, whose ReplaceExpressions leaves that With in the default, and
ConstantFolding, but not RewriteWithExpression; a With is not foldable, so the
analyzed default is not foldable.
Raised in https://github.com/apache/spark/pull/59209#discussion_r4199845666
> A column DEFAULT using NULLIF or BETWEEN is rejected as not constant
> --------------------------------------------------------------------
>
> Key: SPARK-60029
> URL: https://issues.apache.org/jira/browse/SPARK-60029
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 5.0.0
> Reporter: Yang Jie
> Priority: Major
>
> A column DEFAULT that uses NULLIF or BETWEEN is rejected with
> INVALID_DEFAULT_VALUE.NOT_CONSTANT even when every argument is a literal, for
> example `CREATE TABLE t (id INT, c INT DEFAULT nullif(1, 3)) USING parquet`
> or `DEFAULT (1 + 1) BETWEEN 1 AND 3`, while `SELECT nullif(nullif(1, 2), 3)`
> returns 1.
> NullIf and Between build a With expression unless
> spark.sql.alwaysInlineCommonExpr is set. ResolveDefaultColumnsUtil.analyze
> runs FinishAnalysis, whose ReplaceExpressions leaves that With in the
> default, and ConstantFolding, but not RewriteWithExpression; a With is not
> foldable, so the analyzed default is not foldable.
> Raised in https://github.com/apache/spark/pull/59209#discussion_r4199845666
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
