[
https://issues.apache.org/jira/browse/SPARK-60433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Tenedorio updated SPARK-60433:
-------------------------------------
Description:
{{VALUES}} lists (inline tables) only accept expressions that can be
constant-folded, apart from expressions like {{{}current_timestamp(){}}},
because the analyzer evaluates their rows on the driver. So queries such as the
following fail with
{{INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE}} or
{{{}UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.SCALAR_SUBQUERY_IN_VALUES{}}}:
{code:sql}
SELECT * FROM VALUES (rand());
INSERT INTO t VALUES (uuid(), 1);
INSERT INTO t VALUES ((SELECT max(id) FROM t) + 1);
{code}
Users have to rewrite them as {{INSERT ... SELECT}} or as a {{UNION ALL}} of
{{{}SELECT{}}}s, while most other SQL engines accept them.
This ticket allows {{VALUES}} lists to contain new expressions like
nondeterministic functions, Scala/Java UDFs, higher-order functions,
{{raise_error()}} and uncorrelated scalar subqueries, by evaluating such a
{{VALUES}} list when the query runs. {{VALUES}} lists that are valid today keep
their current behavior.
was:
`VALUES` lists (inline tables) only accept expressions that can be
constant-folded, apart from expressions like `current_timestamp()`, because the
analyzer evaluates their rows on the driver. So queries such as the following
fail with `INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE` or
`UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.SCALAR_SUBQUERY_IN_VALUES`:
{code:sql}
SELECT * FROM VALUES (rand());
INSERT INTO t VALUES (uuid(), 1);
INSERT INTO t VALUES ((SELECT max(id) FROM t) + 1);
{code}
Users have to rewrite them as `INSERT ... SELECT` or as a `UNION ALL` of
`SELECT`s, while most other SQL engines accept them.
This ticket allows `VALUES` lists to contain new expressions like
nondeterministic functions, Scala/Java UDFs, higher-order functions,
`raise_error()` and uncorrelated scalar subqueries, by evaluating such a
`VALUES` list when the query runs. `VALUES` lists that are valid today keep
their current behavior.
> Support non-constant expressions in VALUES lists
> ------------------------------------------------
>
> Key: SPARK-60433
> URL: https://issues.apache.org/jira/browse/SPARK-60433
> Project: Spark
> Issue Type: Epic
> Components: SQL
> Affects Versions: 5.0.0
> Reporter: Daniel Tenedorio
> Assignee: Daniel Tenedorio
> Priority: Major
> Labels: pull-request-available
>
> {{VALUES}} lists (inline tables) only accept expressions that can be
> constant-folded, apart from expressions like {{{}current_timestamp(){}}},
> because the analyzer evaluates their rows on the driver. So queries such as
> the following fail with
> {{INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE}} or
> {{{}UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.SCALAR_SUBQUERY_IN_VALUES{}}}:
> {code:sql}
> SELECT * FROM VALUES (rand());
> INSERT INTO t VALUES (uuid(), 1);
> INSERT INTO t VALUES ((SELECT max(id) FROM t) + 1);
> {code}
> Users have to rewrite them as {{INSERT ... SELECT}} or as a {{UNION ALL}} of
> {{{}SELECT{}}}s, while most other SQL engines accept them.
> This ticket allows {{VALUES}} lists to contain new expressions like
> nondeterministic functions, Scala/Java UDFs, higher-order functions,
> {{raise_error()}} and uncorrelated scalar subqueries, by evaluating such a
> {{VALUES}} list when the query runs. {{VALUES}} lists that are valid today
> keep their current behavior.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
