[
https://issues.apache.org/jira/browse/SPARK-60433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Tenedorio updated SPARK-60433:
-------------------------------------
Description:
`VALUES` lists (inline tables) only accept expressions that can be
constant-folded, apart from expressions like `current_timestamp()`, because the
analyzer evaluates their rows on the driver. So queries such as the following
fail with `INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE` or
`UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.SCALAR_SUBQUERY_IN_VALUES`:
{code:sql}
SELECT * FROM VALUES (rand());
INSERT INTO t VALUES (uuid(), 1);
INSERT INTO t VALUES ((SELECT max(id) FROM t) + 1);
{code}
Users have to rewrite them as `INSERT ... SELECT` or as a `UNION ALL` of
`SELECT`s, while most other SQL engines accept them.
This ticket allows `VALUES` lists to contain new expressions like
nondeterministic functions, Scala/Java UDFs, higher-order functions,
`raise_error()` and uncorrelated scalar subqueries, by evaluating such a
`VALUES` list when the query runs. `VALUES` lists that are valid today keep
their current behavior.
was:
VALUES lists (inline tables) only accept expressions that can be
constant-folded, apart from expressions like current_timestamp(), because the
analyzer evaluates their rows on the driver. So queries such as the following
fail with INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE or
UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.SCALAR_SUBQUERY_IN_VALUES:
{code:sql}
SELECT * FROM VALUES (rand());
INSERT INTO t VALUES (uuid(), 1);
INSERT INTO t VALUES ((SELECT max(id) FROM t) + 1);
{code}
Users have to rewrite them as INSERT ... SELECT or as a UNION ALL of SELECTs,
while most other SQL engines accept them.
This ticket allows VALUES lists to contain new expressions like
nondeterministic functions, Scala/Java UDFs, higher-order functions,
raise_error() and uncorrelated scalar subqueries, by evaluating such a VALUES
list when the query runs. VALUES lists that are valid today keep their current
behavior.
> Support non-constant expressions in VALUES lists
> ------------------------------------------------
>
> Key: SPARK-60433
> URL: https://issues.apache.org/jira/browse/SPARK-60433
> Project: Spark
> Issue Type: Epic
> Components: SQL
> Affects Versions: 5.0.0
> Reporter: Daniel Tenedorio
> Assignee: Daniel Tenedorio
> Priority: Major
> Labels: pull-request-available
>
> `VALUES` lists (inline tables) only accept expressions that can be
> constant-folded, apart from expressions like `current_timestamp()`, because
> the analyzer evaluates their rows on the driver. So queries such as the
> following fail with
> `INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE` or
> `UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.SCALAR_SUBQUERY_IN_VALUES`:
> {code:sql}
> SELECT * FROM VALUES (rand());
> INSERT INTO t VALUES (uuid(), 1);
> INSERT INTO t VALUES ((SELECT max(id) FROM t) + 1);
> {code}
> Users have to rewrite them as `INSERT ... SELECT` or as a `UNION ALL` of
> `SELECT`s, while most other SQL engines accept them.
> This ticket allows `VALUES` lists to contain new expressions like
> nondeterministic functions, Scala/Java UDFs, higher-order functions,
> `raise_error()` and uncorrelated scalar subqueries, by evaluating such a
> `VALUES` list when the query runs. `VALUES` lists that are valid today keep
> their current behavior.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
