Daniel Tenedorio created SPARK-60433:
----------------------------------------
Summary: Support non-constant expressions in VALUES lists
Key: SPARK-60433
URL: https://issues.apache.org/jira/browse/SPARK-60433
Project: Spark
Issue Type: New Feature
Components: SQL
Affects Versions: 5.0.0
Reporter: Daniel Tenedorio
VALUES lists (inline tables) only accept expressions that can be
constant-folded, apart from expressions like current_timestamp(), because the
analyzer evaluates their rows on the driver. So queries such as the following
fail with INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE or
UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.SCALAR_SUBQUERY_IN_VALUES:
{code:sql}
SELECT * FROM VALUES (rand());
INSERT INTO t VALUES (uuid(), 1);
INSERT INTO t VALUES ((SELECT max(id) FROM t) + 1);
{code}
Users have to rewrite them as INSERT ... SELECT or as a UNION ALL of SELECTs,
while most other SQL engines accept them.
This ticket allows VALUES lists to contain nondeterministic functions,
Scala/Java UDFs, higher-order functions, raise_error() and uncorrelated scalar
subqueries, by evaluating such a VALUES list when the query runs. VALUES lists
that are valid today keep their current behavior. SQL and Python UDFs, EXISTS
and IN subqueries, and references to outer queries stay unsupported for now;
SPARK-28291 tracks UDFs in VALUES lists.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
