[
https://issues.apache.org/jira/browse/SPARK-60433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Tenedorio updated SPARK-60433:
-------------------------------------
Description:
VALUES lists (inline tables) only accept expressions that can be
constant-folded, apart from expressions like current_timestamp(), because the
analyzer evaluates their rows on the driver. So queries such as the following
fail with INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE or
UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.SCALAR_SUBQUERY_IN_VALUES:
{code:sql}
SELECT * FROM VALUES (rand());
INSERT INTO t VALUES (uuid(), 1);
INSERT INTO t VALUES ((SELECT max(id) FROM t) + 1);
{code}
Users have to rewrite them as INSERT ... SELECT or as a UNION ALL of SELECTs,
while most other SQL engines accept them.
This ticket allows VALUES lists to contain new expressions like
nondeterministic functions, Scala/Java UDFs, higher-order functions,
raise_error() and uncorrelated scalar subqueries, by evaluating such a VALUES
list when the query runs. VALUES lists that are valid today keep their current
behavior. SQL and Python UDFs, EXISTS and IN subqueries, and references to
outer queries stay unsupported for now; SPARK-28291 tracks UDFs in VALUES lists.
was:
VALUES lists (inline tables) only accept expressions that can be
constant-folded, apart from expressions like current_timestamp(), because the
analyzer evaluates their rows on the driver. So queries such as the following
fail with INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE or
UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.SCALAR_SUBQUERY_IN_VALUES:
{code:sql}
SELECT * FROM VALUES (rand());
INSERT INTO t VALUES (uuid(), 1);
INSERT INTO t VALUES ((SELECT max(id) FROM t) + 1);
{code}
Users have to rewrite them as INSERT ... SELECT or as a UNION ALL of SELECTs,
while most other SQL engines accept them.
This ticket allows VALUES lists to contain nondeterministic functions,
Scala/Java UDFs, higher-order functions, raise_error() and uncorrelated scalar
subqueries, by evaluating such a VALUES list when the query runs. VALUES lists
that are valid today keep their current behavior. SQL and Python UDFs, EXISTS
and IN subqueries, and references to outer queries stay unsupported for now;
SPARK-28291 tracks UDFs in VALUES lists.
> Support non-constant expressions in VALUES lists
> ------------------------------------------------
>
> Key: SPARK-60433
> URL: https://issues.apache.org/jira/browse/SPARK-60433
> Project: Spark
> Issue Type: Epic
> Components: SQL
> Affects Versions: 5.0.0
> Reporter: Daniel Tenedorio
> Assignee: Daniel Tenedorio
> Priority: Major
> Labels: pull-request-available
>
> VALUES lists (inline tables) only accept expressions that can be
> constant-folded, apart from expressions like current_timestamp(), because the
> analyzer evaluates their rows on the driver. So queries such as the following
> fail with INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE or
> UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.SCALAR_SUBQUERY_IN_VALUES:
> {code:sql}
> SELECT * FROM VALUES (rand());
> INSERT INTO t VALUES (uuid(), 1);
> INSERT INTO t VALUES ((SELECT max(id) FROM t) + 1);
> {code}
> Users have to rewrite them as INSERT ... SELECT or as a UNION ALL of SELECTs,
> while most other SQL engines accept them.
> This ticket allows VALUES lists to contain new expressions like
> nondeterministic functions, Scala/Java UDFs, higher-order functions,
> raise_error() and uncorrelated scalar subqueries, by evaluating such a VALUES
> list when the query runs. VALUES lists that are valid today keep their
> current behavior. SQL and Python UDFs, EXISTS and IN subqueries, and
> references to outer queries stay unsupported for now; SPARK-28291 tracks UDFs
> in VALUES lists.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
