This is an automated email from the ASF dual-hosted git repository.

dtenedor pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.x by this push:
     new ef0ef73c68e3 [SPARK-60485][SQL] Initial support for non-constant 
expressions in VALUES list (excluding correlation references and SQL UDFs)
ef0ef73c68e3 is described below

commit ef0ef73c68e3480073e677abba85044f96e6e363
Author: Daniel Tenedorio <[email protected]>
AuthorDate: Thu Oct 22 14:20:26 2026 -0700

    [SPARK-60485][SQL] Initial support for non-constant expressions in VALUES 
list (excluding correlation references and SQL UDFs)
    
    ### What changes were proposed in this pull request?
    
    This PR allows VALUES lists (inline tables) to contain expressions that are 
not constant-foldable, for example:
    - nondeterministic functions such as `rand()` and `uuid()`;
    - Scala/Java UDFs;
    - higher-order functions with lambdas;
    - `raise_error()`;
    - uncorrelated scalar subqueries.
    
    Such a VALUES list is planned as an `Expand` over a `OneRowRelation`, with 
one projection per row, so its rows are evaluated by the existing `ExpandExec` 
when the query runs.
    
    VALUES lists that are valid today keep exactly their current behavior: they 
go through the same validation, produce the same `LocalRelation` or 
`ResolvedInlineTable`, and are evaluated by the unchanged `EvalInlineTables` 
rule. The new code path only applies to VALUES lists that fail analysis today.
    
    **Analysis**
    - `EvaluateUnresolvedInlineTable.validateInputEvaluable` accepts every 
VALUES list it accepts today. When some expressions of a list can only be 
evaluated when the query runs, all its expressions other than constants must be 
evaluable by themselves when the query runs, including the ones with 
CURRENT_LIKE expressions. Expressions that are not evaluable are rejected with 
the existing `INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE` 
error:
      - aggregate, window, and generator functions;
      - references to columns of an outer query, e.g. `LATERAL (VALUES (t.a))`;
      - subqueries other than uncorrelated scalar subqueries, such as `EXISTS` 
and `IN`;
      - unevaluable expressions that would have to be rewritten into separate 
operators, such as SQL UDFs and Python UDFs.
    - If every expression of a VALUES list is foldable or contains a 
CURRENT_LIKE expression, which is what the current validation requires, the 
list takes the current code path unchanged. An expression with a subquery 
doesn't qualify through CURRENT_LIKE expressions, inside the subquery or next 
to it, because the subquery can only be evaluated when the query runs. Neither 
does a nondeterministic expression, such as `concat(current_user(), uuid())`, 
because `CheckAnalysis` rejects nondete [...]
      - Foldable cells are evaluated into literals during analysis, so 
evaluation errors in constant cells are still reported during analysis.
      - CURRENT_LIKE cells are replaced with literals at the end of the 
analysis phase, like today.
    - `ResolveSubquery` now resolves subqueries in `UnresolvedInlineTable`. 
`CheckAnalysis` only reports 
`UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.SCALAR_SUBQUERY_IN_VALUES` with the 
conf off, so VALUES lists with scalar subqueries that stay unresolved report 
the actual error, such as an unresolved column or a missing table.
    - The single-pass resolver throws `ExplicitlyUnsupportedResolverFeature` 
for scalar subqueries in VALUES lists, and for VALUES lists that become an 
`Expand`. These queries fall back to the fixed-point analyzer.
    
    **Execution**
    - No new operators. `ExpandExec` applies the projections to the single row 
of its child, so all the rows of the VALUES list are evaluated in one task, 
with whole-stage codegen.
    - Each projection is evaluated once, for partition 0. As a result, `VALUES 
(rand(5)), (rand(5))` returns the same values as `SELECT rand(5) UNION ALL 
SELECT rand(5)`.
    - Scalar subqueries are planned and executed by `PlanSubqueries` and 
`PlanAdaptiveSubqueries` before `ExpandExec` runs.
    - Like in a `SELECT` list, `ColumnPruning` removes the expressions of 
columns that are not read.
    - `ExpandExec` fix: the generated code evaluated a column once per input 
row when the column had equal expressions in all projections, even if they were 
nondeterministic. For example, a nondeterministic UDF called the same way in 
every row of a VALUES list returned the same value in every row, unlike the 
interpreted path. Now only deterministic columns are shared.
    
    **Configuration**
    - New internal conf `spark.sql.inlineTable.nonFoldableExpressions.enabled`, 
default `true`. Setting it to `false` restores the previous behavior.
    
    **References to other DBMSes**
    - These engines accept general expressions in VALUES lists, including 
scalar subqueries: PostgreSQL, SQL Server, Oracle 23ai, MySQL 8.0.19+ and 
SQLite. BigQuery documents scalar subqueries in `INSERT ... VALUES`.
    - Snowflake and Hive keep restrictions similar to Spark's current behavior.
    
    ### Why are the changes needed?
    
    Today, every expression in a VALUES list must be constant-foldable, with 
the exception of expressions like `current_timestamp()`, because the analyzer 
evaluates the rows on the driver. So queries such as the following fail:
    - `INSERT INTO t VALUES (uuid(), ...)`
    - `SELECT * FROM VALUES (rand())`
    - `INSERT INTO t VALUES ((SELECT max(id) FROM t) + 1, ...)`
    
    Users have to rewrite them as `INSERT ... SELECT` or as a `UNION ALL` of 
`SELECT`s. Most other SQL engines accept these queries.
    
    SPARK-28291 tracks UDFs in VALUES lists. This PR supports Scala/Java UDFs 
there, but not SQL or Python UDFs.
    
    ### Does this PR introduce _any_ user-facing change?
    
    Yes. Queries that failed before now succeed:
    
    ```sql
    SELECT * FROM VALUES ('one', rand(5)), ('two', 3.0D) AS data(a, b);
    -- Before: [INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE]
    --         Cannot evaluate the expression "rand(5)" in inline table 
definition.
    -- After:
    -- one  0.02390696427502892
    -- two  3.0
    
    INSERT INTO t VALUES ((SELECT max(id) FROM t) + 1);
    -- Before: 
[UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.SCALAR_SUBQUERY_IN_VALUES]
    --         Scalar subqueries in the VALUES clause.
    -- After: inserts the next id.
    ```
    
    Expressions that still can't be evaluated in a VALUES list fail with 
`INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE`. An inline 
table with non-foldable expressions shows up as an `Expand` over a 
`OneRowRelation` in plans. VALUES lists that are valid today are not affected.
    
    ### How was this patch tested?
    
    New tests:
    - `InlineTableNonFoldableExpressionsSuite` (23 tests), covering:
      - evaluation on executors, in a single task;
      - constant lists, which remain a `LocalTableScan`;
      - lists that are valid today: the same analyzed plans and results with 
the conf on and off, CURRENT_LIKE expressions inside a Scala UDF still 
evaluated on the driver, and evaluation errors next to CURRENT_LIKE expressions 
still reported by the optimizer;
      - nondeterministic semantics: a distinct `uuid()` per row, `rand(5)` 
matching `SELECT rand(5)`, stable values when a plan is re-executed, and equal 
nondeterministic UDF calls evaluated separately for every row, with whole-stage 
codegen on and off;
      - `current_timestamp()` evaluated once per query;
      - nondeterministic expressions next to CURRENT_LIKE expressions in the 
same expression, including a nondeterministic Scala UDF, and the previous error 
with the conf off;
      - type coercion and nullability;
      - uncorrelated scalar subqueries, including `INSERT ... VALUES` that 
reads its target table, and subqueries with CURRENT_LIKE expressions inside 
them or next to them;
      - unresolved columns and missing tables next to and inside scalar 
subqueries, in `SELECT` and `INSERT`;
      - rejected cases: `LATERAL` outer references, also inside a scalar 
subquery, `EXISTS` and `IN` (also with CURRENT_LIKE expressions), aggregate, 
window and generator functions (also over CURRENT_LIKE expressions), SQL UDFs, 
and Python UDFs;
      - runtime errors (`raise_error`), and unread columns that are not 
evaluated;
      - `INSERT` with `DEFAULT` and `uuid()`;
      - DataFrame self-joins;
      - whole-stage codegen and AQE on and off;
      - SQL metrics;
      - a 1000-row list;
      - the conf set to `false`.
    - `ResolveInlineTablesSuite`, covering:
      - validation;
      - the `Expand` that replaces lists with non-foldable expressions, with 
constant cells folded;
      - lists without non-foldable expressions resolving to the same plans with 
the conf on and off;
      - aliased (`DEFAULT`) cells;
      - CURRENT_LIKE expressions;
      - subqueries with CURRENT_LIKE expressions, which take the `Expand` path 
or are rejected;
      - nondeterministic expressions next to CURRENT_LIKE expressions, which 
take the `Expand` path;
      - analysis-time errors in constant cells;
      - `CheckAnalysis` of nondeterministic expressions.
    - New queries in `inline-table.sql`. The `rand(5)` cases in 
`inline-table.sql` and `udf/udf-inline-table.sql` now succeed.
    - `SQLQuerySuite`: the SPARK-49659 test runs with the conf set to `false` 
and checks that the query now works with the conf on.
    
    Regenerated `scripting/randomly_generated_scripts.sql`:
    - 22 scripts used to fail with `SCALAR_SUBQUERY_IN_VALUES` and exit through 
their `EXIT HANDLER`. They contain `INSERT ... VALUES` statements with scalar 
subqueries, and they now run to completion.
    - These scripts share tables, so the data they now insert changes the 
control flow of 6 later scripts:
      - 3 now reach `UPDATE` statements, which are unsupported for these 
Parquet tables.
      - 3 no longer reach such statements.
    - I verified every changed outcome by running the file with all `EXIT 
HANDLER`s removed, once with the conf on and once with it off.
    
    Existing suites:
    - `Expand` execution: `WholeStageCodegenSuite`, `PlannerSuite`, 
`AdaptivePartialAggregationSuite`, `DatasetUnpivotSuite`, 
`DataFrameAggregateSuite`, and the `GROUPING SETS` and UNPIVOT golden files.
    - Catalyst analyzer suites, including `AnalysisSuite`, 
`AnalysisErrorSuite`, `ResolveSubquerySuite` and `PlanParserSuite`.
    - `InlineTableParsingImprovementsSuite`, `ResolveDefaultColumnsSuite`, the 
`*InsertSuite`s and the `*SubquerySuite`s.
    - The single-pass resolver suites: `ResolverGuardSuite`, 
`HybridAnalyzerSuite`, the `*ResolverSuite`s, `ResolutionValidatorSuite`, 
`ExplicitlyUnsupportedResolverFeatureSuite`, `DataFrameAnalyzerTestGapsSuite`.
    - The `SQLQueryTestSuite` golden files for inline tables, CTEs, 
`create_view`, `insert`, session variables and scripting.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: Cursor (Claude Opus 5.5)
    
    Closes #59681 from dtenedor/values-list-with-non-constant-exprs.
    
    Authored-by: Daniel Tenedorio <[email protected]>
    Signed-off-by: Daniel Tenedorio <[email protected]>
    (cherry picked from commit 8d3e0c52c3902a1f82d0536425ecae18ba9e5ade)
    Signed-off-by: Daniel Tenedorio <[email protected]>
---
 docs/sql-ref-syntax-qry-select-inline-table.md     |  14 +
 .../spark/sql/catalyst/analysis/Analyzer.scala     |   4 +
 .../sql/catalyst/analysis/CheckAnalysis.scala      |   4 +-
 .../sql/catalyst/analysis/resolver/Resolver.scala  |  10 +
 .../resolver/SubqueryExpressionResolver.scala      |   4 +
 .../spark/sql/catalyst/analysis/unresolved.scala   |   5 +-
 .../sql/catalyst/optimizer/finishAnalysis.scala    |  25 +-
 .../util/EvaluateUnresolvedInlineTable.scala       |  99 +++++-
 .../org/apache/spark/sql/internal/SQLConf.scala    |  13 +
 .../analysis/ResolveInlineTablesSuite.scala        | 181 ++++++++--
 .../apache/spark/sql/execution/ExpandExec.scala    |   6 +-
 .../analyzer-results/inline-table.sql.out          | 110 ++++++-
 .../scripting/randomly_generated_scripts.sql.out   |  56 ++--
 .../analyzer-results/udf/udf-inline-table.sql.out  |  16 +-
 .../resources/sql-tests/inputs/inline-table.sql    |  23 +-
 .../sql-tests/inputs/udf/udf-inline-table.sql      |   2 +-
 .../sql-tests/results/inline-table.sql.out         | 119 ++++++-
 .../scripting/randomly_generated_scripts.sql.out   | 112 +++----
 .../sql-tests/results/udf/udf-inline-table.sql.out |  19 +-
 .../InlineTableNonFoldableExpressionsSuite.scala   | 365 +++++++++++++++++++++
 .../scala/org/apache/spark/sql/SQLQuerySuite.scala |  25 +-
 21 files changed, 1004 insertions(+), 208 deletions(-)

diff --git a/docs/sql-ref-syntax-qry-select-inline-table.md 
b/docs/sql-ref-syntax-qry-select-inline-table.md
index 7f0372ea787e..7c1731046b7f 100644
--- a/docs/sql-ref-syntax-qry-select-inline-table.md
+++ b/docs/sql-ref-syntax-qry-select-inline-table.md
@@ -34,6 +34,11 @@ VALUES ( expression [ , ... ] ) [ table_alias ]
 * **expression**
 
     Specifies a combination of one or more values, operators and SQL functions 
that results in a value.
+    Since Spark 4.4, the expression does not need to be constant: 
nondeterministic functions such as
+    `rand()`, Scala and Java user-defined functions, and uncorrelated scalar 
subqueries are evaluated
+    when the query runs. Aggregate, window and generator functions, SQL and 
Python user-defined
+    functions, other kinds of subqueries, and references to columns of an 
outer query are not
+    supported.
 
 * **table_alias**
 
@@ -70,6 +75,15 @@ SELECT * FROM VALUES ("one", array(0, 1)), ("two", array(2, 
3)) AS data(a, b);
 |one|[0, 1]|
 |two|[2, 3]|
 +---+------+
+
+-- expressions that are not constant, such as nondeterministic functions and 
scalar subqueries
+SELECT a, length(b) FROM VALUES (1, uuid()), ((SELECT max(id) FROM range(5)), 
uuid()) AS data(a, b);
++---+---------+
+|  a|length(b)|
++---+---------+
+|  1|       36|
+|  4|       36|
++---+---------+
 ```
 
 ### Related Statements
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
index 29384c9020d7..8c5237adba9b 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
@@ -2801,6 +2801,10 @@ class Analyzer(
         resolveSubQueries(j, j)
       case tvf: UnresolvedTableValuedFunction =>
         resolveSubQueries(tvf, tvf)
+      // An inline table has no output, so its subqueries cannot reference its 
columns.
+      case t: UnresolvedInlineTable
+          if 
conf.getConf(SQLConf.INLINE_TABLE_NON_FOLDABLE_EXPRESSIONS_ENABLED) =>
+        resolveSubQueries(t, t)
       case s: SupportsSubquery if s.childrenResolved =>
         resolveSubQueries(s, s)
     }
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
index 57ed47bcd0bb..f567268d964e 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala
@@ -492,7 +492,9 @@ trait CheckAnalysis extends LookupCatalog with 
QueryErrorsBase with PlanToString
         throw SparkException.internalError(
           s"V2Reference should be resolved during analysis: ${r.name}")
 
-      case u: UnresolvedInlineTable if 
unresolvedInlineTableContainsScalarSubquery(u) =>
+      case u: UnresolvedInlineTable
+          if 
!SQLConf.get.getConf(SQLConf.INLINE_TABLE_NON_FOLDABLE_EXPRESSIONS_ENABLED) &&
+            unresolvedInlineTableContainsScalarSubquery(u) =>
         throw QueryCompilationErrors.inlineTableContainsScalarSubquery(u)
 
       case command: V2PartitionCommand =>
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/resolver/Resolver.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/resolver/Resolver.scala
index 458836388a24..61c7170af464 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/resolver/Resolver.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/resolver/Resolver.scala
@@ -639,6 +639,9 @@ class Resolver(
    * Note: By default if all the inline table expressions can be evaluated 
eagerly, the parser
    * would produce a [[LocalRelation]] and the analysis would just skip this 
step and go straight
    * to `resolveLocalRelation` (SPARK-48967, SPARK-49269).
+   *
+   * Inline tables with expressions that are evaluated when the query runs, 
such as `rand()`,
+   * become an [[Expand]], which only the fixed-point Analyzer supports.
    */
   private def resolveInlineTable(unresolvedInlineTable: 
UnresolvedInlineTable): LogicalPlan = {
     val withResolvedExpressions = UnresolvedInlineTable(
@@ -654,6 +657,13 @@ class Resolver(
     val resolvedRelation = EvaluateUnresolvedInlineTable
       .evaluateUnresolvedInlineTable(withResolvedExpressions)
 
+    resolvedRelation match {
+      case _: Expand =>
+        throw new ExplicitlyUnsupportedResolverFeature(
+          "VALUES lists with expressions that are evaluated when the query 
runs")
+      case _ =>
+    }
+
     resolve(resolvedRelation)
   }
 
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/resolver/SubqueryExpressionResolver.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/resolver/SubqueryExpressionResolver.scala
index 574fca29d818..9a1693a3891f 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/resolver/SubqueryExpressionResolver.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/resolver/SubqueryExpressionResolver.scala
@@ -28,6 +28,7 @@ import org.apache.spark.sql.catalyst.expressions.{
 }
 import org.apache.spark.sql.catalyst.plans.logical.LogicalPlan
 import org.apache.spark.sql.errors.QueryCompilationErrors
+import org.apache.spark.sql.internal.SQLConf
 
 /**
  * [[SubqueryExpressionResolver]] resolves specific [[SubqueryExpression]]s, 
such as
@@ -54,6 +55,9 @@ class SubqueryExpressionResolver(expressionResolver: 
ExpressionResolver, resolve
    */
   def resolveScalarSubquery(unresolvedScalarSubquery: ScalarSubquery): 
Expression = {
     traversals.current.parentOperator match {
+      case _: UnresolvedInlineTable
+          if 
conf.getConf(SQLConf.INLINE_TABLE_NON_FOLDABLE_EXPRESSIONS_ENABLED) =>
+        throw new ExplicitlyUnsupportedResolverFeature("scalar subqueries in 
VALUES lists")
       case unresolvedInlineTable: UnresolvedInlineTable =>
         throw 
QueryCompilationErrors.inlineTableContainsScalarSubquery(unresolvedInlineTable)
       case _ =>
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala
index 6765af018b2a..f3cb4653040c 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala
@@ -197,7 +197,10 @@ object UnresolvedRelation {
 
 /**
  * An inline table that has not been resolved yet. Once resolved, it is turned 
by the analyzer into
- * a [[org.apache.spark.sql.catalyst.plans.logical.LocalRelation]].
+ * a [[org.apache.spark.sql.catalyst.plans.logical.LocalRelation]], or into an
+ * [[org.apache.spark.sql.catalyst.plans.logical.Expand]] over a
+ * [[org.apache.spark.sql.catalyst.plans.logical.OneRowRelation]] if some of 
its expressions are
+ * evaluated when the query runs.
  *
  * @param names list of column names
  * @param rows expressions for the data
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/finishAnalysis.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/finishAnalysis.scala
index 7644c1023613..c32995d973a1 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/finishAnalysis.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/finishAnalysis.scala
@@ -92,20 +92,23 @@ object EvalInlineTables extends Rule[LogicalPlan] with 
CastSupport {
 
     def eval(table: ResolvedInlineTable): LocalRelation = {
       val newRows: Seq[InternalRow] =
-        table.rows.map { row => InternalRow.fromSeq(row.map { e =>
-          try {
-            prepareForEval(e).eval()
-          } catch {
-            case NonFatal(ex) =>
-              table.failAnalysis(
-                errorClass = 
"INVALID_INLINE_TABLE.FAILED_SQL_EXPRESSION_EVALUATION",
-                messageParameters = Map("sqlExpr" -> toSQLExpr(e)),
-                cause = ex)
-          }})
-        }
+        table.rows.map(row => 
InternalRow.fromSeq(row.map(evalExpression(table, _))))
 
       LocalRelation(table.output, newRows)
     }
+
+    /** Evaluates one of the expressions of the given inline table. */
+    def evalExpression(table: ResolvedInlineTable, e: Expression): Any = {
+      try {
+        prepareForEval(e).eval()
+      } catch {
+        case NonFatal(ex) =>
+          table.failAnalysis(
+            errorClass = 
"INVALID_INLINE_TABLE.FAILED_SQL_EXPRESSION_EVALUATION",
+            messageParameters = Map("sqlExpr" -> toSQLExpr(e)),
+            cause = ex)
+      }
+    }
 }
 
 /**
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/EvaluateUnresolvedInlineTable.scala
 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/EvaluateUnresolvedInlineTable.scala
index 15c2e74fb4a1..e8d9e9b569ae 100644
--- 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/EvaluateUnresolvedInlineTable.scala
+++ 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/EvaluateUnresolvedInlineTable.scala
@@ -18,12 +18,14 @@ package org.apache.spark.sql.catalyst.util
 
 import org.apache.spark.sql.catalyst.SQLConfHelper
 import org.apache.spark.sql.catalyst.analysis._
-import org.apache.spark.sql.catalyst.expressions.{AliasHelper, EvalHelper, 
Expression}
+import org.apache.spark.sql.catalyst.expressions.{AliasHelper, 
CommonExpressionDef, EvalHelper, Expression, Generator, LeafExpression, 
Literal, OuterReference, ScalarSubquery, SubqueryExpression, Unevaluable, 
WindowExpression, WindowFunction}
+import 
org.apache.spark.sql.catalyst.expressions.aggregate.{AggregateExpression, 
AggregateFunction}
 import org.apache.spark.sql.catalyst.optimizer.EvalInlineTables
-import org.apache.spark.sql.catalyst.plans.logical.LogicalPlan
+import org.apache.spark.sql.catalyst.plans.logical.{Expand, LogicalPlan, 
OneRowRelation}
 import org.apache.spark.sql.catalyst.trees.TreePattern.CURRENT_LIKE
 import org.apache.spark.sql.catalyst.types.DataTypeUtils
 import org.apache.spark.sql.catalyst.util.TypeUtils.{toSQLExpr, toSQLId}
+import org.apache.spark.sql.internal.SQLConf
 import org.apache.spark.sql.types.{StringHelper, StringType, StructField, 
StructType}
 
 /**
@@ -32,6 +34,10 @@ import org.apache.spark.sql.types.{StringHelper, StringType, 
StructField, Struct
  * a child of [[InsertIntoStatement]].
  * Use the method: [[EvaluateUnresolvedInlineTable.evaluate]] as the entry 
point for this
  * transformation.
+ *
+ * If [[SQLConf.INLINE_TABLE_NON_FOLDABLE_EXPRESSIONS_ENABLED]] is true, an 
inline table with
+ * expressions that can only be evaluated when the query runs, such as 
`rand()`, is replaced with
+ * an [[Expand]] over a [[OneRowRelation]] instead, with one projection per 
row.
  */
 object EvaluateUnresolvedInlineTable extends SQLConfHelper
   with AliasHelper with EvalHelper with CastSupport {
@@ -47,7 +53,11 @@ object EvaluateUnresolvedInlineTable extends SQLConfHelper
     validateInputDimension(table)
     validateInputEvaluable(table)
     val resolvedTable = findCommonTypesAndCast(table, ignoreCollation)
-    earlyEvalIfPossible(resolvedTable)
+    if (canEvalAllDuringAnalysis(table)) {
+      earlyEvalIfPossible(resolvedTable)
+    } else {
+      toExpand(resolvedTable)
+    }
   }
 
   /**
@@ -61,6 +71,48 @@ object EvaluateUnresolvedInlineTable extends SQLConfHelper
     if (earlyEvalPossible) EvalInlineTables.eval(table) else table
   }
 
+  /**
+   * Turns an inline table with expressions that are evaluated when the query 
runs into an
+   * [[Expand]] over a [[OneRowRelation]], which produces one output row per 
projection. The
+   * constant expressions are evaluated now (see [[isConstant]]).
+   */
+  private def toExpand(table: ResolvedInlineTable): Expand = {
+    val projections = table.rows.map(_.map {
+      case e if isConstant(e) => 
Literal(EvalInlineTables.evalExpression(table, e), e.dataType)
+      case e => e
+    })
+    Expand(projections, table.output, OneRowRelation())
+  }
+
+  private def runtimeEvaluationEnabled: Boolean =
+    conf.getConf(SQLConf.INLINE_TABLE_NON_FOLDABLE_EXPRESSIONS_ENABLED)
+
+  /** Returns whether all the expressions of the inline table can be evaluated 
during analysis. */
+  private def canEvalAllDuringAnalysis(table: UnresolvedInlineTable): Boolean =
+    table.rows.forall(_.forall(canEvalDuringAnalysis))
+
+  /**
+   * Returns whether the expression can be evaluated during the analysis 
phase: it must be
+   * foldable, or contain CURRENT_LIKE expressions, which are replaced by 
literals at the end of
+   * the analysis phase, but no subqueries, which can only be evaluated when 
the query runs.
+   * Inline tables with only such expressions are evaluated during analysis. If
+   * [[SQLConf.INLINE_TABLE_NON_FOLDABLE_EXPRESSIONS_ENABLED]] is true, the 
expressions with
+   * CURRENT_LIKE expressions must also be deterministic, because a 
[[ResolvedInlineTable]] cannot
+   * contain nondeterministic expressions.
+   */
+  private def canEvalDuringAnalysis(e: Expression): Boolean =
+    (e.containsPattern(CURRENT_LIKE) && !SubqueryExpression.hasSubquery(e) &&
+      (e.deterministic || !runtimeEvaluationEnabled)) ||
+      (e.resolved && trimAliases(prepareForEval(e)).foldable)
+
+  /**
+   * Returns whether the expression is a constant, which is evaluated during 
the analysis phase
+   * even if the inline table becomes an [[Expand]]: it must be foldable, and 
not contain
+   * CURRENT_LIKE expressions, which are replaced with literals at the end of 
the analysis phase.
+   */
+  private def isConstant(e: Expression): Boolean =
+    !e.containsPattern(CURRENT_LIKE) && e.resolved && 
trimAliases(prepareForEval(e)).foldable
+
   /**
    * Validates the input data dimension:
    * 1. All rows have the same cardinality.
@@ -85,18 +137,23 @@ object EvaluateUnresolvedInlineTable extends SQLConfHelper
   }
 
   /**
-   * Validates that all inline table data are valid expressions that can be 
evaluated
-   * (in this they must be foldable).
-   * Note that nondeterministic expressions are not supported since they are 
not foldable.
-   * Exception are CURRENT_LIKE expressions, which are replaced by a literal 
in later stages.
+   * Validates that all inline table data are valid expressions that can be 
evaluated: during the
+   * analysis phase (see [[canEvalDuringAnalysis]]), or, if
+   * [[SQLConf.INLINE_TABLE_NON_FOLDABLE_EXPRESSIONS_ENABLED]] is true, when 
the query runs (see
+   * [[isRuntimeEvaluable]]). In the latter case, all the expressions that are 
not constants are
+   * evaluated when the query runs, including the ones with CURRENT_LIKE 
expressions.
    * This is package visible for unit testing.
    */
   def validateInputEvaluable(table: UnresolvedInlineTable): Unit = {
+    val evaluatedWhenQueryRuns = runtimeEvaluationEnabled && 
!canEvalAllDuringAnalysis(table)
     table.rows.foreach { row =>
       row.foreach { e =>
-        if (e.containsPattern(CURRENT_LIKE)) {
-          // Do nothing.
-        } else if (!e.resolved || !trimAliases(prepareForEval(e)).foldable) {
+        val evaluable = if (evaluatedWhenQueryRuns) {
+          isConstant(e) || (e.resolved && isRuntimeEvaluable(e))
+        } else {
+          canEvalDuringAnalysis(e)
+        }
+        if (!evaluable) {
           e.failAnalysis(
             errorClass = 
"INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE",
             messageParameters = Map("expr" -> toSQLExpr(e)))
@@ -105,6 +162,28 @@ object EvaluateUnresolvedInlineTable extends SQLConfHelper
     }
   }
 
+  /**
+   * Returns whether the resolved expression can be evaluated by itself when 
the query runs.
+   * Aggregate, window and generator functions are not meaningful in a VALUES 
list. Outer
+   * references and subqueries other than uncorrelated scalar subqueries are 
not supported, and
+   * neither are the remaining unevaluable expressions, such as SQL and Python 
UDFs, which would
+   * have to be rewritten into separate operators.
+   */
+  private def isRuntimeEvaluable(e: Expression): Boolean = {
+    !trimAliases(prepareForEval(e)).exists {
+      case _: AggregateExpression | _: AggregateFunction | _: WindowExpression 
|
+           _: WindowFunction | _: Generator | _: OuterReference => true
+      case s: ScalarSubquery => s.isCorrelated
+      case _: SubqueryExpression => true
+      // Replaced with literals at the end of the analysis phase.
+      case l: LeafExpression if l.containsPattern(CURRENT_LIKE) => false
+      // Evaluated by the enclosing `With`.
+      case _: CommonExpressionDef => false
+      case _: Unevaluable => true
+      case _ => false
+    }
+  }
+
   /**
    * This function attempts to coerce inputs into consistent types.
    *
diff --git 
a/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala 
b/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
index 6d532e546e90..1e8236ef67df 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
@@ -1815,6 +1815,19 @@ object SQLConf {
       .booleanConf
       .createWithDefault(true)
 
+  val INLINE_TABLE_NON_FOLDABLE_EXPRESSIONS_ENABLED =
+    buildConf("spark.sql.inlineTable.nonFoldableExpressions.enabled")
+      .internal()
+      .doc("When true, VALUES lists (inline tables) may contain expressions 
that are not " +
+        "constant-foldable, such as nondeterministic functions, Scala/Java 
UDFs, and " +
+        "uncorrelated scalar subqueries. Such an inline table is evaluated on 
executors when " +
+        "the query runs instead of being folded into a local relation during 
analysis. When " +
+        "false, these expressions are rejected during analysis.")
+      .version("4.4.0")
+      .withBindingPolicy(ConfigBindingPolicy.SESSION)
+      .booleanConf
+      .createWithDefault(true)
+
   val ESCAPED_STRING_LITERALS = 
buildConf("spark.sql.parser.escapedStringLiterals")
     .internal()
     .doc("When true, string literals (including regex patterns) remain escaped 
in our SQL " +
diff --git 
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/ResolveInlineTablesSuite.scala
 
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/ResolveInlineTablesSuite.scala
index 50d88e90987c..6cb0a5df968d 100644
--- 
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/ResolveInlineTablesSuite.scala
+++ 
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/ResolveInlineTablesSuite.scala
@@ -20,12 +20,13 @@ package org.apache.spark.sql.catalyst.analysis
 import org.scalatest.BeforeAndAfter
 
 import org.apache.spark.sql.AnalysisException
-import org.apache.spark.sql.catalyst.expressions.{Alias, Cast, CurrentTime, 
CurrentTimestamp, Literal, Rand}
+import org.apache.spark.sql.catalyst.expressions.{Add, Alias, 
AttributeReference, Cast, Concat, CreateArray, CurrentTime, CurrentTimestamp, 
CurrentUser, Exists, Explode, InSubquery, ListQuery, Literal, OuterReference, 
Rand, ScalarSubquery, Uuid}
 import org.apache.spark.sql.catalyst.expressions.aggregate.Count
 import org.apache.spark.sql.catalyst.optimizer.{ComputeCurrentTime, 
EvalInlineTables}
-import org.apache.spark.sql.catalyst.plans.logical.LocalRelation
+import org.apache.spark.sql.catalyst.plans.logical.{Expand, LocalRelation, 
OneRowRelation, Project}
 import org.apache.spark.sql.catalyst.util.EvaluateUnresolvedInlineTable
-import org.apache.spark.sql.types.{LongType, NullType, TimestampNTZType, 
TimestampType, TimeType}
+import org.apache.spark.sql.internal.SQLConf
+import org.apache.spark.sql.types.{DoubleType, IntegerType, LongType, 
NullType, TimestampNTZType, TimestampType, TimeType}
 
 /**
  * Unit tests for [[ResolveInlineTables]]. Note that there are also test cases 
defined in
@@ -37,33 +38,62 @@ class ResolveInlineTablesSuite extends AnalysisTest with 
BeforeAndAfter {
   private def lit(v: Any): Literal = Literal(v)
 
   test("validate inputs are foldable") {
-    EvaluateUnresolvedInlineTable.validateInputEvaluable(
-      UnresolvedInlineTable(Seq("c1", "c2"), Seq(Seq(lit(1)))))
-
-    // Alias is OK
-    EvaluateUnresolvedInlineTable.validateInputEvaluable(
-      UnresolvedInlineTable(Seq("c1", "c2"), Seq(Seq(Alias(lit(1), "a")()))))
-
-    // nondeterministic (rand) should not work
-    intercept[AnalysisException] {
+    withSQLConf(SQLConf.INLINE_TABLE_NON_FOLDABLE_EXPRESSIONS_ENABLED.key -> 
"false") {
       EvaluateUnresolvedInlineTable.validateInputEvaluable(
-        UnresolvedInlineTable(Seq("c1"), Seq(Seq(Rand(1)))))
-    }
-    intercept[AnalysisException] {
+        UnresolvedInlineTable(Seq("c1", "c2"), Seq(Seq(lit(1)))))
+
+      // Alias is OK
       EvaluateUnresolvedInlineTable.validateInputEvaluable(
-        UnresolvedInlineTable(Seq("c1"), Seq(Seq(Cast(Rand(1), 
TimestampNTZType)))))
+        UnresolvedInlineTable(Seq("c1", "c2"), Seq(Seq(Alias(lit(1), "a")()))))
+
+      // nondeterministic (rand) should not work
+      intercept[AnalysisException] {
+        EvaluateUnresolvedInlineTable.validateInputEvaluable(
+          UnresolvedInlineTable(Seq("c1"), Seq(Seq(Rand(1)))))
+      }
+      intercept[AnalysisException] {
+        EvaluateUnresolvedInlineTable.validateInputEvaluable(
+          UnresolvedInlineTable(Seq("c1"), Seq(Seq(Cast(Rand(1), 
TimestampNTZType)))))
+      }
+
+      // aggregate should not work
+      intercept[AnalysisException] {
+        EvaluateUnresolvedInlineTable.validateInputEvaluable(
+          UnresolvedInlineTable(Seq("c1"), Seq(Seq(Count(lit(1))))))
+      }
+
+      // unresolved attribute should not work
+      intercept[AnalysisException] {
+        EvaluateUnresolvedInlineTable.validateInputEvaluable(
+          UnresolvedInlineTable(Seq("c1"), Seq(Seq(UnresolvedAttribute("A")))))
+      }
     }
+  }
 
-    // aggregate should not work
-    intercept[AnalysisException] {
-      EvaluateUnresolvedInlineTable.validateInputEvaluable(
-        UnresolvedInlineTable(Seq("c1"), Seq(Seq(Count(lit(1))))))
+  test("validate inputs are foldable or can be evaluated when the query runs") 
{
+    val oneRow = Project(Seq(Alias(lit(1), "a")()), OneRowRelation())
+    Seq(lit(1), Alias(lit(1), "a")(), Rand(1), Uuid(Some(1L)), 
ScalarSubquery(oneRow)).foreach {
+      e =>
+        EvaluateUnresolvedInlineTable.validateInputEvaluable(
+          UnresolvedInlineTable(Seq("c1"), Seq(Seq(e))))
     }
 
-    // unresolved attribute should not work
-    intercept[AnalysisException] {
-      EvaluateUnresolvedInlineTable.validateInputEvaluable(
-        UnresolvedInlineTable(Seq("c1"), Seq(Seq(UnresolvedAttribute("A")))))
+    val outerReference = OuterReference(AttributeReference("a", IntegerType)())
+    val correlatedPlan = Project(Seq(Alias(outerReference, "b")()), 
OneRowRelation())
+    Seq(
+      Count(lit(1)),
+      Count(lit(1)).toAggregateExpression(),
+      Explode(CreateArray(Seq(lit(1), lit(2)))),
+      outerReference,
+      ScalarSubquery(correlatedPlan, outerAttrs = Seq(outerReference)),
+      Exists(oneRow),
+      InSubquery(Seq(lit(1)), ListQuery(oneRow, numCols = 1)),
+      UnresolvedAttribute("A")
+    ).foreach { e =>
+      intercept[AnalysisException] {
+        EvaluateUnresolvedInlineTable.validateInputEvaluable(
+          UnresolvedInlineTable(Seq("c1"), Seq(Seq(e))))
+      }
     }
   }
 
@@ -156,6 +186,109 @@ class ResolveInlineTablesSuite extends AnalysisTest with 
BeforeAndAfter {
     assert(data.head.getLong(0) == correct)
   }
 
+  test("inline tables with non-foldable expressions are evaluated when the 
query runs") {
+    val table = UnresolvedInlineTable(Seq("c1", "c2"),
+      Seq(Seq(lit(1), Rand(1)), Seq(Add(lit(2), lit(3)), lit(1.5))))
+    val resolved = ResolveInlineTables(table)
+    resolved match {
+      case Expand(projections, output, _: OneRowRelation) =>
+        assert(output.map(_.dataType) == Seq(IntegerType, DoubleType))
+        // Foldable expressions are evaluated during analysis.
+        assert(projections == Seq(Seq(lit(1), Rand(1)), Seq(lit(5), lit(1.5))))
+      case other => fail(s"Expected an Expand, but got $other")
+    }
+    assert(resolved.maxRows.contains(2L))
+    // The optimizer leaves the table in the plan, to be evaluated when the 
query runs.
+    assert(EvalInlineTables(resolved) == resolved)
+  }
+
+  test("inline tables without non-foldable expressions are resolved as 
before") {
+    val flag = SQLConf.INLINE_TABLE_NON_FOLDABLE_EXPRESSIONS_ENABLED.key
+    Seq(
+      Seq(Seq(lit(1), Add(lit(2), lit(3))), Seq(lit(4L), lit(5))),
+      // CURRENT_LIKE expressions are replaced at the end of the analysis 
phase, so the other
+      // expressions of the table are evaluated at that point too.
+      Seq(Seq(CurrentTimestamp(), Add(lit(2), lit(3))))
+    ).foreach { rows =>
+      val table = UnresolvedInlineTable(rows.head.indices.map(i => s"c$i"), 
rows)
+      val resolved = ResolveInlineTables(table)
+      assert(!resolved.isInstanceOf[Expand])
+      comparePlans(
+        resolved, withSQLConf(flag -> "false")(ResolveInlineTables(table)), 
checkAnalysis = false)
+    }
+  }
+
+  test("aliased foldable expressions are evaluated during analysis") {
+    // Column DEFAULT values are substituted as aliased expressions.
+    val table = UnresolvedInlineTable(Seq("c1"), Seq(Seq(Alias(lit(1), 
"a")())))
+    assert(ResolveInlineTables(table).isInstanceOf[LocalRelation])
+
+    val mixed = UnresolvedInlineTable(Seq("c1", "c2"), Seq(Seq(Alias(lit(1), 
"a")(), Rand(1))))
+    ResolveInlineTables(mixed) match {
+      case Expand(Seq(Seq(l: Literal, _: Rand)), _, _) => assert(l == lit(1))
+      case other => fail(s"Expected an Expand, but got $other")
+    }
+  }
+
+  test("CURRENT_LIKE expressions next to non-foldable expressions") {
+    val table = UnresolvedInlineTable(Seq("c1", "c2"),
+      Seq(Seq(CurrentTimestamp(), Rand(1)), Seq(CurrentTimestamp(), Rand(2))))
+    EvalInlineTables(ComputeCurrentTime(ResolveInlineTables(table))) match {
+      case Expand(Seq(Seq(ts1: Literal, _: Rand), Seq(ts2: Literal, _: Rand)), 
_, _) =>
+        // Both CURRENT_TIMESTAMP expressions are evaluated to the same value.
+        assert(ts1 == ts2)
+      case other => fail(s"Expected an Expand, but got $other")
+    }
+  }
+
+  test("nondeterministic expressions next to CURRENT_LIKE expressions in the 
same expression") {
+    // A ResolvedInlineTable cannot contain nondeterministic expressions.
+    val e = Concat(Seq(CurrentUser(), Uuid(Some(1L))))
+    val table = UnresolvedInlineTable(Seq("c1"), Seq(Seq(e)))
+    ResolveInlineTables(table) match {
+      case Expand(Seq(Seq(projection)), _, _: OneRowRelation) => 
assert(projection == e)
+      case other => fail(s"Expected an Expand, but got $other")
+    }
+    withSQLConf(SQLConf.INLINE_TABLE_NON_FOLDABLE_EXPRESSIONS_ENABLED.key -> 
"false") {
+      assert(ResolveInlineTables(table).isInstanceOf[ResolvedInlineTable])
+    }
+  }
+
+  test("subqueries with CURRENT_LIKE expressions are not evaluated during 
analysis") {
+    // The tree patterns of a subquery expression include those of its plan.
+    val currentTimestampRow = Project(Seq(Alias(CurrentTimestamp(), "ts")()), 
OneRowRelation())
+    Seq(
+      ScalarSubquery(currentTimestampRow),
+      CreateArray(Seq(CurrentTimestamp(), ScalarSubquery(currentTimestampRow)))
+    ).foreach { e =>
+      ResolveInlineTables(UnresolvedInlineTable(Seq("c1"), Seq(Seq(e)))) match 
{
+        case Expand(Seq(Seq(projection)), _, _: OneRowRelation) => 
assert(projection == e)
+        case other => fail(s"Expected an Expand, but got $other")
+      }
+    }
+    Seq(
+      Exists(currentTimestampRow),
+      InSubquery(Seq(CurrentTimestamp()), ListQuery(currentTimestampRow, 
numCols = 1))
+    ).foreach { e =>
+      val table = UnresolvedInlineTable(Seq("c1"), Seq(Seq(e)))
+      val ex = intercept[AnalysisException](ResolveInlineTables(table))
+      assert(ex.getCondition == 
"INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE")
+    }
+  }
+
+  test("evaluation errors of foldable expressions next to non-foldable 
expressions") {
+    withSQLConf(SQLConf.ANSI_ENABLED.key -> "true") {
+      val table = UnresolvedInlineTable(Seq("c1", "c2"),
+        Seq(Seq(Cast(lit("abc"), IntegerType), Rand(1))))
+      val e = intercept[AnalysisException](ResolveInlineTables(table))
+      assert(e.getCondition == 
"INVALID_INLINE_TABLE.FAILED_SQL_EXPRESSION_EVALUATION")
+    }
+  }
+
+  test("analysis of inline tables with nondeterministic expressions") {
+    assertAnalysisSuccess(UnresolvedInlineTable(Seq("c1"), Seq(Seq(Rand(1)), 
Seq(Rand(2)))))
+  }
+
   test("nullability inference in convert") {
     val table1 = UnresolvedInlineTable(Seq("c1"), Seq(Seq(lit(1)), 
Seq(lit(2L))))
     val converted1 = 
EvaluateUnresolvedInlineTable.findCommonTypesAndCast(table1)
diff --git 
a/sql/core/src/main/scala/org/apache/spark/sql/execution/ExpandExec.scala 
b/sql/core/src/main/scala/org/apache/spark/sql/execution/ExpandExec.scala
index d68dc0df2aa5..ee31909549e0 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/execution/ExpandExec.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/execution/ExpandExec.scala
@@ -147,9 +147,11 @@ case class ExpandExec(
     // Tracks whether a column has the same output for all rows.
     // Size of sameOutput array should equal N.
     // If sameOutput(i) is true, then the i-th column has the same value for 
all output rows given
-    // an input row.
+    // an input row. Equal nondeterministic expressions can still produce a 
different value for
+    // every output row.
     val sameOutput: Array[Boolean] = output.indices.map { colIndex =>
-      projections.map(p => p(colIndex)).toSet.size == 1
+      val column = projections.map(p => p(colIndex))
+      column.toSet.size == 1 && column.head.deterministic
     }.toArray
 
     // Bind all the branch expressions once up front, so that identical 
expressions appearing in
diff --git 
a/sql/core/src/test/resources/sql-tests/analyzer-results/inline-table.sql.out 
b/sql/core/src/test/resources/sql-tests/analyzer-results/inline-table.sql.out
index 4aa144b59257..f0af900e5499 100644
--- 
a/sql/core/src/test/resources/sql-tests/analyzer-results/inline-table.sql.out
+++ 
b/sql/core/src/test/resources/sql-tests/analyzer-results/inline-table.sql.out
@@ -95,21 +95,7 @@ Project [a#x, b#x]
 -- !query
 select * from values ("one", rand(5)), ("two", 3.0D) as data(a, b)
 -- !query analysis
-org.apache.spark.sql.AnalysisException
-{
-  "errorClass" : 
"INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE",
-  "sqlState" : "42000",
-  "messageParameters" : {
-    "expr" : "\"rand(5)\""
-  },
-  "queryContext" : [ {
-    "objectType" : "",
-    "objectName" : "",
-    "startIndex" : 30,
-    "stopIndex" : 36,
-    "fragment" : "rand(5)"
-  } ]
-}
+[Analyzer test output redacted due to nondeterminism]
 
 
 -- !query
@@ -277,3 +263,97 @@ select * from values (1 as a, 2 as b)
 -- !query analysis
 Project [col1#x, col2#x]
 +- LocalRelation [col1#x, col2#x]
+
+
+-- !query
+select count(distinct u) from values (uuid()), (uuid()), (uuid()) as data(u)
+-- !query analysis
+[Analyzer test output redacted due to nondeterminism]
+
+
+-- !query
+select count(distinct ct), count(distinct r) from values (current_timestamp(), 
rand()), (current_timestamp(), rand()) as data(ct, r)
+-- !query analysis
+[Analyzer test output redacted due to nondeterminism]
+
+
+-- !query
+select a, b is not null from values (1 + 1, rand()), (cast('3' as int), 
rand()) as data(a, b)
+-- !query analysis
+[Analyzer test output redacted due to nondeterminism]
+
+
+-- !query
+select * from values ((select 1) + (select 2), 'a'), ((select max(c) from 
values (4), (5) as t(c)), 'b') as data(a, b)
+-- !query analysis
+Project [a#x, b#x]
++- SubqueryAlias data
+   +- Expand [[(scalar-subquery#x [] + scalar-subquery#x []), a], 
[scalar-subquery#x [], b]], [a#x, b#x]
+      :  :- Project [1 AS 1#x]
+      :  :  +- OneRowRelation
+      :  :- Project [2 AS 2#x]
+      :  :  +- OneRowRelation
+      :  +- Aggregate [max(c#x) AS max(c)#x]
+      :     +- SubqueryAlias t
+      :        +- LocalRelation [c#x]
+      +- OneRowRelation
+
+
+-- !query
+select * from values (1) as t(a), lateral (values (t.a, rand())) as v(b, r)
+-- !query analysis
+org.apache.spark.sql.AnalysisException
+{
+  "errorClass" : 
"INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE",
+  "sqlState" : "42000",
+  "messageParameters" : {
+    "expr" : "\"outer(t.a)\""
+  },
+  "queryContext" : [ {
+    "objectType" : "",
+    "objectName" : "",
+    "startIndex" : 52,
+    "stopIndex" : 54,
+    "fragment" : "t.a"
+  } ]
+}
+
+
+-- !query
+select * from values (exists (select 1), rand()) as data(a, b)
+-- !query analysis
+org.apache.spark.sql.AnalysisException
+{
+  "errorClass" : 
"INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE",
+  "sqlState" : "42000",
+  "messageParameters" : {
+    "expr" : "\"exists()\""
+  },
+  "queryContext" : [ {
+    "objectType" : "",
+    "objectName" : "",
+    "startIndex" : 23,
+    "stopIndex" : 39,
+    "fragment" : "exists (select 1)"
+  } ]
+}
+
+
+-- !query
+select * from values (row_number() over (order by 1), rand()) as data(a, b)
+-- !query analysis
+org.apache.spark.sql.AnalysisException
+{
+  "errorClass" : 
"INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE",
+  "sqlState" : "42000",
+  "messageParameters" : {
+    "expr" : "\"row_number() OVER (ORDER BY 1 ASC NULLS FIRST ROWS BETWEEN 
UNBOUNDED PRECEDING AND CURRENT ROW)\""
+  },
+  "queryContext" : [ {
+    "objectType" : "",
+    "objectName" : "",
+    "startIndex" : 23,
+    "stopIndex" : 52,
+    "fragment" : "row_number() over (order by 1)"
+  } ]
+}
diff --git 
a/sql/core/src/test/resources/sql-tests/analyzer-results/scripting/randomly_generated_scripts.sql.out
 
b/sql/core/src/test/resources/sql-tests/analyzer-results/scripting/randomly_generated_scripts.sql.out
index 793b2b37415d..81aa94f12ec6 100644
--- 
a/sql/core/src/test/resources/sql-tests/analyzer-results/scripting/randomly_generated_scripts.sql.out
+++ 
b/sql/core/src/test/resources/sql-tests/analyzer-results/scripting/randomly_generated_scripts.sql.out
@@ -479,7 +479,7 @@ BEGIN
     END;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -888,7 +888,7 @@ BEGIN
     END FOR supplier_loop;
 END
 -- !query analysis
-LocalRelation <empty>
+LocalRelation [col1#x]
 
 
 -- !query
@@ -994,7 +994,7 @@ BEGIN
     END L4;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -1285,7 +1285,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -1372,7 +1372,7 @@ BEGIN
     END;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -1661,7 +1661,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -2275,7 +2275,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -2351,7 +2351,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -2594,7 +2594,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -2672,7 +2672,7 @@ BEGIN
     END L2;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -2767,7 +2767,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -3210,7 +3210,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -3318,7 +3318,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -3552,7 +3552,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -3863,7 +3863,7 @@ BEGIN
     END;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -4147,7 +4147,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -4642,7 +4642,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation <empty>
+LocalRelation [col1#x]
 
 
 -- !query
@@ -4911,7 +4911,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -5015,7 +5015,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -5154,7 +5154,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -5439,7 +5439,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -5472,7 +5472,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation <empty>
+LocalRelation [col1#x]
 
 
 -- !query
@@ -5529,7 +5529,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -5644,7 +5644,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -5749,7 +5749,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -5990,7 +5990,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -6476,7 +6476,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
@@ -6700,7 +6700,7 @@ BEGIN
     END L0;
 END
 -- !query analysis
-LocalRelation [col1#x]
+LocalRelation <empty>
 
 
 -- !query
diff --git 
a/sql/core/src/test/resources/sql-tests/analyzer-results/udf/udf-inline-table.sql.out
 
b/sql/core/src/test/resources/sql-tests/analyzer-results/udf/udf-inline-table.sql.out
index 786b5ac49b12..9bf3fc541f30 100644
--- 
a/sql/core/src/test/resources/sql-tests/analyzer-results/udf/udf-inline-table.sql.out
+++ 
b/sql/core/src/test/resources/sql-tests/analyzer-results/udf/udf-inline-table.sql.out
@@ -81,21 +81,7 @@ Project [cast(udf(cast(a#x as string)) as string) AS 
udf(a)#x, b#x]
 -- !query
 select udf(a), b from values ("one", rand(5)), ("two", 3.0D) as data(a, b)
 -- !query analysis
-org.apache.spark.sql.AnalysisException
-{
-  "errorClass" : 
"INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE",
-  "sqlState" : "42000",
-  "messageParameters" : {
-    "expr" : "\"rand(5)\""
-  },
-  "queryContext" : [ {
-    "objectType" : "",
-    "objectName" : "",
-    "startIndex" : 38,
-    "stopIndex" : 44,
-    "fragment" : "rand(5)"
-  } ]
-}
+[Analyzer test output redacted due to nondeterminism]
 
 
 -- !query
diff --git a/sql/core/src/test/resources/sql-tests/inputs/inline-table.sql 
b/sql/core/src/test/resources/sql-tests/inputs/inline-table.sql
index bb66e5dc6b9f..641df4a8e0a1 100644
--- a/sql/core/src/test/resources/sql-tests/inputs/inline-table.sql
+++ b/sql/core/src/test/resources/sql-tests/inputs/inline-table.sql
@@ -35,7 +35,7 @@ select * from values ("one", array(0, 1)), ("two", array(2, 
3)) as data(a, b);
 -- decimal and double coercion
 select * from values ("one", 2.0), ("two", 3.0D) as data(a, b);
 
--- error reporting: nondeterministic function rand
+-- nondeterministic function rand
 select * from values ("one", rand(5)), ("two", 3.0D) as data(a, b);
 
 -- error reporting: different number of columns
@@ -75,3 +75,24 @@ select * from values (1 as id, 'a' as name), (2 as id, 'b' 
as name) as t(a, b);
 
 -- aliased expressions without table alias
 select * from values (1 as a, 2 as b);
+
+-- nondeterministic functions have a different seed in every row
+select count(distinct u) from values (uuid()), (uuid()), (uuid()) as data(u);
+
+-- current_timestamp() is evaluated once per query, also next to non-foldable 
expressions
+select count(distinct ct), count(distinct r) from values (current_timestamp(), 
rand()), (current_timestamp(), rand()) as data(ct, r);
+
+-- foldable expressions next to non-foldable expressions
+select a, b is not null from values (1 + 1, rand()), (cast('3' as int), 
rand()) as data(a, b);
+
+-- uncorrelated scalar subqueries
+select * from values ((select 1) + (select 2), 'a'), ((select max(c) from 
values (4), (5) as t(c)), 'b') as data(a, b);
+
+-- error reporting: correlated reference
+select * from values (1) as t(a), lateral (values (t.a, rand())) as v(b, r);
+
+-- error reporting: EXISTS subquery
+select * from values (exists (select 1), rand()) as data(a, b);
+
+-- error reporting: window function
+select * from values (row_number() over (order by 1), rand()) as data(a, b);
diff --git 
a/sql/core/src/test/resources/sql-tests/inputs/udf/udf-inline-table.sql 
b/sql/core/src/test/resources/sql-tests/inputs/udf/udf-inline-table.sql
index 179e528d74ca..daa06b81febf 100644
--- a/sql/core/src/test/resources/sql-tests/inputs/udf/udf-inline-table.sql
+++ b/sql/core/src/test/resources/sql-tests/inputs/udf/udf-inline-table.sql
@@ -32,7 +32,7 @@ select udf(a), b from values ("one", array(0, 1)), ("two", 
array(2, 3)) as data(
 -- decimal and double coercion
 select udf(a), b from values ("one", 2.0), ("two", 3.0D) as data(a, b);
 
--- error reporting: nondeterministic function rand
+-- nondeterministic function rand
 select udf(a), b from values ("one", rand(5)), ("two", 3.0D) as data(a, b);
 
 -- error reporting: different number of columns
diff --git a/sql/core/src/test/resources/sql-tests/results/inline-table.sql.out 
b/sql/core/src/test/resources/sql-tests/results/inline-table.sql.out
index dee75e7338aa..cc6553d8b21b 100644
--- a/sql/core/src/test/resources/sql-tests/results/inline-table.sql.out
+++ b/sql/core/src/test/resources/sql-tests/results/inline-table.sql.out
@@ -107,23 +107,10 @@ two       3.0
 -- !query
 select * from values ("one", rand(5)), ("two", 3.0D) as data(a, b)
 -- !query schema
-struct<>
+struct<a:string,b:double>
 -- !query output
-org.apache.spark.sql.AnalysisException
-{
-  "errorClass" : 
"INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE",
-  "sqlState" : "42000",
-  "messageParameters" : {
-    "expr" : "\"rand(5)\""
-  },
-  "queryContext" : [ {
-    "objectType" : "",
-    "objectName" : "",
-    "startIndex" : 30,
-    "stopIndex" : 36,
-    "fragment" : "rand(5)"
-  } ]
-}
+one    0.02390696427502892
+two    3.0
 
 
 -- !query
@@ -313,3 +300,103 @@ select * from values (1 as a, 2 as b)
 struct<col1:int,col2:int>
 -- !query output
 1      2
+
+
+-- !query
+select count(distinct u) from values (uuid()), (uuid()), (uuid()) as data(u)
+-- !query schema
+struct<count(DISTINCT u):bigint>
+-- !query output
+3
+
+
+-- !query
+select count(distinct ct), count(distinct r) from values (current_timestamp(), 
rand()), (current_timestamp(), rand()) as data(ct, r)
+-- !query schema
+struct<count(DISTINCT ct):bigint,count(DISTINCT r):bigint>
+-- !query output
+1      2
+
+
+-- !query
+select a, b is not null from values (1 + 1, rand()), (cast('3' as int), 
rand()) as data(a, b)
+-- !query schema
+struct<a:int,(b IS NOT NULL):boolean>
+-- !query output
+2      true
+3      true
+
+
+-- !query
+select * from values ((select 1) + (select 2), 'a'), ((select max(c) from 
values (4), (5) as t(c)), 'b') as data(a, b)
+-- !query schema
+struct<a:int,b:string>
+-- !query output
+3      a
+5      b
+
+
+-- !query
+select * from values (1) as t(a), lateral (values (t.a, rand())) as v(b, r)
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.sql.AnalysisException
+{
+  "errorClass" : 
"INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE",
+  "sqlState" : "42000",
+  "messageParameters" : {
+    "expr" : "\"outer(t.a)\""
+  },
+  "queryContext" : [ {
+    "objectType" : "",
+    "objectName" : "",
+    "startIndex" : 52,
+    "stopIndex" : 54,
+    "fragment" : "t.a"
+  } ]
+}
+
+
+-- !query
+select * from values (exists (select 1), rand()) as data(a, b)
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.sql.AnalysisException
+{
+  "errorClass" : 
"INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE",
+  "sqlState" : "42000",
+  "messageParameters" : {
+    "expr" : "\"exists()\""
+  },
+  "queryContext" : [ {
+    "objectType" : "",
+    "objectName" : "",
+    "startIndex" : 23,
+    "stopIndex" : 39,
+    "fragment" : "exists (select 1)"
+  } ]
+}
+
+
+-- !query
+select * from values (row_number() over (order by 1), rand()) as data(a, b)
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.sql.AnalysisException
+{
+  "errorClass" : 
"INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE",
+  "sqlState" : "42000",
+  "messageParameters" : {
+    "expr" : "\"row_number() OVER (ORDER BY 1 ASC NULLS FIRST ROWS BETWEEN 
UNBOUNDED PRECEDING AND CURRENT ROW)\""
+  },
+  "queryContext" : [ {
+    "objectType" : "",
+    "objectName" : "",
+    "startIndex" : 23,
+    "stopIndex" : 52,
+    "fragment" : "row_number() over (order by 1)"
+  } ]
+}
diff --git 
a/sql/core/src/test/resources/sql-tests/results/scripting/randomly_generated_scripts.sql.out
 
b/sql/core/src/test/resources/sql-tests/results/scripting/randomly_generated_scripts.sql.out
index 1583266025ea..160a7f6c4a94 100644
--- 
a/sql/core/src/test/resources/sql-tests/results/scripting/randomly_generated_scripts.sql.out
+++ 
b/sql/core/src/test/resources/sql-tests/results/scripting/randomly_generated_scripts.sql.out
@@ -495,9 +495,9 @@ BEGIN
     END;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Orders processing error
+
 
 
 -- !query
@@ -914,9 +914,9 @@ BEGIN
     END FOR supplier_loop;
 END
 -- !query schema
-struct<>
+struct<col1:string>
 -- !query output
-
+Supplier contact management error
 
 
 -- !query
@@ -1024,9 +1024,9 @@ BEGIN
     END L4;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Customer order calculation error
+
 
 
 -- !query
@@ -1323,9 +1323,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Tenure analysis error
+
 
 
 -- !query
@@ -1412,9 +1412,9 @@ BEGIN
     END;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Order source analysis error
+
 
 
 -- !query
@@ -1707,9 +1707,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Employee demographics error
+
 
 
 -- !query
@@ -2339,9 +2339,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-City shipping error
+
 
 
 -- !query
@@ -2417,9 +2417,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Employee hierarchy error
+
 
 
 -- !query
@@ -2668,9 +2668,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Product price error
+
 
 
 -- !query
@@ -2748,9 +2748,9 @@ BEGIN
     END L2;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Employee diversity error
+
 
 
 -- !query
@@ -2845,9 +2845,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Channel analysis error
+
 
 
 -- !query
@@ -3300,9 +3300,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Order status error
+
 
 
 -- !query
@@ -3410,9 +3410,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Department diversity error
+
 
 
 -- !query
@@ -3652,9 +3652,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Product margin error
+
 
 
 -- !query
@@ -3971,9 +3971,9 @@ BEGIN
     END;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Color diversity error
+
 
 
 -- !query
@@ -4261,9 +4261,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Product size error
+
 
 
 -- !query
@@ -4770,9 +4770,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<>
+struct<col1:string>
 -- !query output
-
+Order item status error
 
 
 -- !query
@@ -5047,9 +5047,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Order-item linkage error
+
 
 
 -- !query
@@ -5155,9 +5155,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Inventory movement error
+
 
 
 -- !query
@@ -5298,9 +5298,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Tenure salary error
+
 
 
 -- !query
@@ -5593,9 +5593,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Type order diversity error
+
 
 
 -- !query
@@ -5628,9 +5628,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<>
+struct<col1:string>
 -- !query output
-
+Offer code error
 
 
 -- !query
@@ -5687,9 +5687,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Dept stagnation error
+
 
 
 -- !query
@@ -5806,9 +5806,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Discount balancing error
+
 
 
 -- !query
@@ -5915,9 +5915,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Order value error
+
 
 
 -- !query
@@ -6162,9 +6162,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Bulk item error
+
 
 
 -- !query
@@ -6666,9 +6666,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Status workflow error
+
 
 
 -- !query
@@ -6898,9 +6898,9 @@ BEGIN
     END L0;
 END
 -- !query schema
-struct<col1:string>
+struct<>
 -- !query output
-Bundle error
+
 
 
 -- !query
diff --git 
a/sql/core/src/test/resources/sql-tests/results/udf/udf-inline-table.sql.out 
b/sql/core/src/test/resources/sql-tests/results/udf/udf-inline-table.sql.out
index 3e84ec09c215..47e048c7e20f 100644
--- a/sql/core/src/test/resources/sql-tests/results/udf/udf-inline-table.sql.out
+++ b/sql/core/src/test/resources/sql-tests/results/udf/udf-inline-table.sql.out
@@ -91,23 +91,10 @@ two 3.0
 -- !query
 select udf(a), b from values ("one", rand(5)), ("two", 3.0D) as data(a, b)
 -- !query schema
-struct<>
+struct<udf(a):string,b:double>
 -- !query output
-org.apache.spark.sql.AnalysisException
-{
-  "errorClass" : 
"INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE",
-  "sqlState" : "42000",
-  "messageParameters" : {
-    "expr" : "\"rand(5)\""
-  },
-  "queryContext" : [ {
-    "objectType" : "",
-    "objectName" : "",
-    "startIndex" : 38,
-    "stopIndex" : 44,
-    "fragment" : "rand(5)"
-  } ]
-}
+one    0.02390696427502892
+two    3.0
 
 
 -- !query
diff --git 
a/sql/core/src/test/scala/org/apache/spark/sql/InlineTableNonFoldableExpressionsSuite.scala
 
b/sql/core/src/test/scala/org/apache/spark/sql/InlineTableNonFoldableExpressionsSuite.scala
new file mode 100644
index 000000000000..d083d699c15d
--- /dev/null
+++ 
b/sql/core/src/test/scala/org/apache/spark/sql/InlineTableNonFoldableExpressionsSuite.scala
@@ -0,0 +1,365 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.spark.sql
+
+import java.util.UUID
+
+import org.apache.spark.{SparkRuntimeException, TaskContext}
+import org.apache.spark.sql.IntegratedUDFTestUtils.{registerTestUDF, 
shouldTestPythonUDFs, TestPythonUDF}
+import org.apache.spark.sql.execution.{ExpandExec, LocalTableScanExec}
+import org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanHelper
+import org.apache.spark.sql.functions.udf
+import org.apache.spark.sql.internal.SQLConf
+import org.apache.spark.sql.test.SharedSparkSession
+import org.apache.spark.sql.types.{DoubleType, LongType, StructField, 
StructType}
+
+/**
+ * End-to-end tests for VALUES lists with expressions that are not foldable, 
which are evaluated
+ * by an `Expand` over a `OneRowRelation` when the query runs.
+ */
+class InlineTableNonFoldableExpressionsSuite
+  extends QueryTest with SharedSparkSession with AdaptiveSparkPlanHelper {
+  import testImplicits._
+
+  private def expandExecs(df: DataFrame): Seq[ExpandExec] =
+    collect(df.queryExecution.executedPlan) { case e: ExpandExec => e }
+
+  private def contextOf(query: String, fragment: String): ExpectedContext = {
+    val start = query.indexOf(fragment)
+    assert(start >= 0, s"'$fragment' is not part of '$query'")
+    ExpectedContext(fragment, start, start + fragment.length - 1)
+  }
+
+  test("non-foldable expressions are evaluated on executors") {
+    withUserDefinedFunction("in_task" -> true) {
+      spark.udf.register("in_task", udf(() => TaskContext.get() != 
null).asNondeterministic())
+      val df = sql("SELECT * FROM VALUES (1, in_task()), (2, in_task()) AS 
t(id, in_task)")
+      checkAnswer(df, Seq(Row(1, true), Row(2, true)))
+      val execs = expandExecs(df)
+      assert(execs.size == 1)
+      // All the rows are evaluated in a single task.
+      assert(execs.head.execute().getNumPartitions == 1)
+    }
+  }
+
+  test("VALUES lists with only foldable expressions still become local 
relations") {
+    val df = sql("SELECT * FROM VALUES (1, 'a'), (2 + 1, upper('b')) AS t(id, 
s)")
+    checkAnswer(df, Seq(Row(1, "a"), Row(3, "B")))
+    assert(expandExecs(df).isEmpty)
+    assert(collect(df.queryExecution.executedPlan) { case l: 
LocalTableScanExec => l }.size == 1)
+  }
+
+  test("VALUES lists that were supported before are evaluated during analysis 
as before") {
+    withUserDefinedFunction("on_driver" -> true) {
+      spark.udf.register("on_driver", udf((s: String) => s != null && 
TaskContext.get() == null))
+      val queries = Seq(
+        "SELECT * FROM VALUES (1, 'a'), (2 + 1, upper('b')) AS t(a, b)",
+        "SELECT * FROM VALUES (current_date(), 1 + 1) AS t(a, b)",
+        // The UDF is not foldable, but it was supported because its input is 
a CURRENT_LIKE
+        // expression. It is evaluated on the driver at the end of the 
analysis phase.
+        "SELECT * FROM VALUES (on_driver(current_user())) AS t(a)")
+      queries.foreach { query =>
+        val df = sql(query)
+        val legacy = withSQLConf(
+            SQLConf.INLINE_TABLE_NON_FOLDABLE_EXPRESSIONS_ENABLED.key -> 
"false") {
+          sql(query)
+        }
+        comparePlans(
+          df.queryExecution.analyzed, legacy.queryExecution.analyzed, 
checkAnalysis = false)
+        checkAnswer(df, legacy)
+        assert(expandExecs(df).isEmpty)
+      }
+      checkAnswer(sql(queries.last), Row(true))
+    }
+
+    // The expressions of inline tables with CURRENT_LIKE expressions are 
evaluated at the end of
+    // the analysis phase, so evaluation errors are reported when the query is 
optimized.
+    withSQLConf(SQLConf.ANSI_ENABLED.key -> "true") {
+      val df = sql("SELECT * FROM VALUES (current_date(), 1 / 0) AS t(a, b)")
+      val e = intercept[AnalysisException](df.collect())
+      assert(e.getCondition == 
"INVALID_INLINE_TABLE.FAILED_SQL_EXPRESSION_EVALUATION")
+    }
+  }
+
+  test("nondeterministic expressions") {
+    // Every expression has its own seed.
+    val uuids = sql("SELECT * FROM VALUES (uuid()), (uuid()), 
(uuid())").collect()
+    assert(uuids.map(_.getString(0)).distinct.length == 3)
+
+    // Each row is evaluated like the only row of a one-row relation.
+    checkAnswer(
+      sql("SELECT * FROM VALUES (rand(5)), (rand(5))"),
+      sql("SELECT rand(5) UNION ALL SELECT rand(5)"))
+
+    // The seeds are chosen during analysis, so executing the plan again 
returns the same values.
+    val df = sql("SELECT * FROM VALUES (rand(), 1), (rand(), 2)")
+    checkAnswer(df, df.collect().toSeq)
+  }
+
+  test("equal nondeterministic expressions are evaluated separately for every 
row") {
+    withUserDefinedFunction("random_uuid" -> true) {
+      spark.udf.register("random_uuid", udf(() => 
UUID.randomUUID().toString).asNondeterministic())
+      Seq(true, false).foreach { codegen =>
+        withSQLConf(SQLConf.WHOLESTAGE_CODEGEN_ENABLED.key -> 
codegen.toString) {
+          val rows = sql("SELECT * FROM VALUES (random_uuid()), 
(random_uuid())").collect()
+          assert(rows.map(_.getString(0)).distinct.length == 2)
+        }
+      }
+    }
+  }
+
+  test("CURRENT_LIKE expressions are evaluated once per query") {
+    val df = sql(
+      """SELECT count(DISTINCT ts), count(DISTINCT r)
+        |FROM VALUES (current_timestamp(), rand()), (current_timestamp(), 
rand()) AS t(ts, r)
+        |""".stripMargin)
+    checkAnswer(df, Row(1, 2))
+  }
+
+  test("nondeterministic expressions next to CURRENT_LIKE expressions in the 
same expression") {
+    checkAnswer(
+      sql(
+        """SELECT count(DISTINCT s), min(startswith(s, current_user()))
+          |FROM VALUES (concat(current_user(), uuid())), 
(concat(current_user(), uuid())) AS t(s)
+          |""".stripMargin),
+      Row(2, true))
+    checkAnswer(
+      sql(
+        """SELECT d BETWEEN current_date() AND date_add(current_date(), 9)
+          |FROM VALUES (date_add(current_date(), CAST(rand() * 10 AS INT))) AS 
t(d)
+          |""".stripMargin),
+      Row(true))
+    withUserDefinedFunction("nd_identity" -> true) {
+      spark.udf.register("nd_identity", udf((s: String) => 
s).asNondeterministic())
+      checkAnswer(
+        sql("SELECT s = current_user() FROM VALUES 
(nd_identity(current_user())) AS t(s)"),
+        Row(true))
+    }
+
+    withSQLConf(SQLConf.INLINE_TABLE_NON_FOLDABLE_EXPRESSIONS_ENABLED.key -> 
"false") {
+      // The inline table is not evaluated when the query runs, so it cannot 
contain them.
+      val e = intercept[AnalysisException](
+        sql("SELECT * FROM VALUES (concat(current_user(), uuid()))"))
+      assert(e.getCondition == "INVALID_NON_DETERMINISTIC_EXPRESSIONS")
+    }
+  }
+
+  test("type coercion and nullability") {
+    val df = sql("SELECT * FROM VALUES (1, rand()), (2L, NULL) AS t(a, b)")
+    assert(df.schema == StructType(Seq(
+      StructField("a", LongType, nullable = false),
+      StructField("b", DoubleType, nullable = true))))
+    checkAnswer(df.select($"a", $"b".isNull), Seq(Row(1L, false), Row(2L, 
true)))
+  }
+
+  test("uncorrelated scalar subqueries") {
+    checkAnswer(sql("SELECT * FROM VALUES ((SELECT 1) + (SELECT 2), 'a')"), 
Row(3, "a"))
+    withTable("t") {
+      sql("CREATE TABLE t(id INT) USING parquet")
+      sql("INSERT INTO t VALUES (1), (2)")
+      sql("INSERT INTO t VALUES ((SELECT max(id) FROM t) + 1)")
+      checkAnswer(sql("SELECT * FROM t"), Seq(Row(1), Row(2), Row(3)))
+    }
+  }
+
+  test("uncorrelated scalar subqueries with CURRENT_LIKE expressions") {
+    // The CURRENT_LIKE expressions inside and next to the subqueries share 
the value of the rest
+    // of the query.
+    checkAnswer(
+      sql("SELECT d = current_date() FROM VALUES ((SELECT current_date())) AS 
t(d)"),
+      Row(true))
+    checkAnswer(
+      sql(
+        """SELECT d = date_add(current_date(), 1)
+          |FROM VALUES (date_add(current_date(), (SELECT 1))) AS t(d)
+          |""".stripMargin),
+      Row(true))
+    withTable("t") {
+      sql("CREATE TABLE t(id INT, ts TIMESTAMP) USING parquet")
+      sql("INSERT INTO t VALUES (1, TIMESTAMP'2000-01-01 00:00:00')")
+      sql(
+        """INSERT INTO t VALUES
+          |  ((SELECT max(id) FROM t WHERE ts < current_timestamp()) + 1, 
current_timestamp())
+          |""".stripMargin)
+      checkAnswer(sql("SELECT id FROM t"), Seq(Row(1), Row(2)))
+    }
+  }
+
+  test("unresolved references in VALUES lists with scalar subqueries") {
+    val unknownColumn = "SELECT * FROM VALUES ((SELECT 1), unknown_col)"
+    checkError(
+      exception = intercept[AnalysisException](sql(unknownColumn)),
+      condition = "UNRESOLVED_COLUMN.WITHOUT_SUGGESTION",
+      parameters = Map("objectName" -> "`unknown_col`"),
+      context = contextOf(unknownColumn, "unknown_col"))
+    val missingTable = "SELECT * FROM VALUES ((SELECT x FROM missing_table))"
+    checkError(
+      exception = intercept[AnalysisException](sql(missingTable)),
+      condition = "TABLE_OR_VIEW_NOT_FOUND",
+      parameters = Map("relationName" -> "`missing_table`"),
+      context = contextOf(missingTable, "missing_table"))
+    withTable("t") {
+      sql("CREATE TABLE t(id INT) USING parquet")
+      val insert = "INSERT INTO t VALUES ((SELECT max(id) FROM missing_table))"
+      checkError(
+        exception = intercept[AnalysisException](sql(insert)),
+        condition = "TABLE_OR_VIEW_NOT_FOUND",
+        parameters = Map("relationName" -> "`missing_table`"),
+        context = contextOf(insert, "missing_table"))
+    }
+  }
+
+  test("correlated references and subqueries other than scalar subqueries are 
rejected") {
+    withTempView("t") {
+      sql("CREATE TEMPORARY VIEW t AS SELECT * FROM VALUES (1) AS t(a)")
+      val query = "SELECT * FROM t, LATERAL (VALUES (t.a, rand())) AS v(b, r)"
+      checkError(
+        exception = intercept[AnalysisException](sql(query)),
+        condition = 
"INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE",
+        parameters = Map("expr" -> "\"outer(t.a)\""),
+        context = contextOf(query, "t.a"))
+
+      // The subquery can only reference the columns of the inline table, 
which has none.
+      val nested = "SELECT * FROM t, LATERAL (VALUES ((SELECT t.a), rand())) 
AS v(b, r)"
+      checkError(
+        exception = intercept[AnalysisException](sql(nested)),
+        condition = "UNRESOLVED_COLUMN.WITHOUT_SUGGESTION",
+        parameters = Map("objectName" -> "`t`.`a`"),
+        context = contextOf(nested, "t.a"))
+    }
+    Seq(
+      "SELECT * FROM VALUES (EXISTS (SELECT 1), rand())",
+      "SELECT * FROM VALUES (1 IN (SELECT 1), rand())",
+      // CURRENT_LIKE expressions don't make them evaluable during analysis.
+      "SELECT * FROM VALUES (EXISTS (SELECT current_date()))",
+      "SELECT * FROM VALUES (current_date() IN (SELECT 
current_date()))").foreach { query =>
+      val e = intercept[AnalysisException](sql(query))
+      assert(e.getCondition == 
"INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE")
+    }
+  }
+
+  test("aggregate, window and generator functions are rejected") {
+    Seq(
+      "SELECT * FROM VALUES (count(1), rand())",
+      "SELECT * FROM VALUES (row_number() OVER (ORDER BY 1), rand())",
+      "SELECT * FROM VALUES (explode(array(1, 2)), 1)",
+      // Also over CURRENT_LIKE expressions, next to expressions evaluated 
when the query runs.
+      "SELECT * FROM VALUES (max(current_date()), rand())",
+      "SELECT * FROM VALUES (row_number() OVER (ORDER BY current_date()), 
rand())",
+      "SELECT * FROM VALUES (explode(array(current_date())), rand())").foreach 
{ query =>
+      val e = intercept[AnalysisException](sql(query))
+      assert(e.getCondition == 
"INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE")
+    }
+  }
+
+  test("SQL UDFs are rejected") {
+    withUserDefinedFunction("plus_one" -> true) {
+      sql("CREATE TEMPORARY FUNCTION plus_one(x INT) RETURNS INT RETURN x + 1")
+      val e = intercept[AnalysisException](sql("SELECT * FROM VALUES 
(plus_one(1))"))
+      assert(e.getCondition == 
"INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE")
+    }
+  }
+
+  test("Python UDFs are rejected") {
+    assume(shouldTestPythonUDFs)
+    withUserDefinedFunction("py_udf" -> true) {
+      registerTestUDF(TestPythonUDF(name = "py_udf"), spark)
+      val e = intercept[AnalysisException](sql("SELECT * FROM VALUES 
(py_udf(1))"))
+      assert(e.getCondition == 
"INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE")
+    }
+  }
+
+  test("evaluation errors are raised when the query runs") {
+    val df = sql("SELECT * FROM VALUES (1, raise_error('boom'))")
+    checkError(
+      exception = intercept[SparkRuntimeException](df.collect()),
+      condition = "USER_RAISED_EXCEPTION",
+      parameters = Map("errorMessage" -> "boom"))
+  }
+
+  test("the expressions of columns that are not read are not evaluated") {
+    // Like the expressions of a SELECT list, the optimizer removes them.
+    checkAnswer(sql("SELECT a FROM VALUES (1, raise_error('boom')) AS t(a, 
b)"), Row(1))
+  }
+
+  test("INSERT INTO ... VALUES with non-foldable expressions and DEFAULT") {
+    withTable("t") {
+      sql("CREATE TABLE t(id INT, s STRING DEFAULT 'x', u STRING) USING 
parquet")
+      sql("INSERT INTO t VALUES (1, DEFAULT, uuid()), (2, 'y', uuid())")
+      checkAnswer(sql("SELECT id, s, length(u) FROM t"), Seq(Row(1, "x", 36), 
Row(2, "y", 36)))
+      checkAnswer(sql("SELECT count(DISTINCT u) FROM t"), Row(2))
+    }
+  }
+
+  test("self-join of an inline table with non-foldable expressions") {
+    val df = sql("SELECT * FROM VALUES (1, rand()), (2, rand()) AS t(id, r)")
+    val joined = df.as("a").join(df.as("b"), $"a.id" === $"b.id")
+    // Both sides are the same plan, so they produce the same values.
+    checkAnswer(joined.select($"a.id", $"a.r" === $"b.r"), Seq(Row(1, true), 
Row(2, true)))
+  }
+
+  test("whole-stage codegen and adaptive query execution") {
+    Seq(true, false).foreach { codegen =>
+      Seq(true, false).foreach { aqe =>
+        withSQLConf(
+            SQLConf.WHOLESTAGE_CODEGEN_ENABLED.key -> codegen.toString,
+            SQLConf.ADAPTIVE_EXECUTION_ENABLED.key -> aqe.toString) {
+          val df = sql(
+            """SELECT a + 1, b IS NOT NULL, c
+              |FROM VALUES (1, rand(), (SELECT 'x')), (2, rand(), 'y') AS t(a, 
b, c)
+              |""".stripMargin)
+          checkAnswer(df, Seq(Row(2, true, "x"), Row(3, true, "y")))
+          assert(expandExecs(df).size == 1)
+        }
+      }
+    }
+  }
+
+  test("SQL metrics") {
+    val df = sql("SELECT * FROM VALUES (rand()), (rand()), (rand())")
+    df.collect()
+    assert(expandExecs(df).head.metrics("numOutputRows").value == 3)
+  }
+
+  test("large inline table with non-foldable expressions") {
+    val rows = (1 to 1000).map(i => s"($i, rand())").mkString(", ")
+    checkAnswer(
+      sql(s"SELECT count(*), count(DISTINCT a), count(DISTINCT b) FROM VALUES 
$rows AS t(a, b)"),
+      Row(1000, 1000, 1000))
+  }
+
+  test("non-foldable expressions are rejected if the feature is disabled") {
+    withSQLConf(SQLConf.INLINE_TABLE_NON_FOLDABLE_EXPRESSIONS_ENABLED.key -> 
"false") {
+      val query = "SELECT * FROM VALUES (1, rand(5))"
+      checkError(
+        exception = intercept[AnalysisException](sql(query)),
+        condition = 
"INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE",
+        parameters = Map("expr" -> "\"rand(5)\""),
+        context = contextOf(query, "rand(5)"))
+
+      Seq("(SELECT 1)", "(SELECT current_date())").foreach { subquery =>
+        val subqueryQuery = s"SELECT * FROM VALUES ($subquery)"
+        checkError(
+          exception = intercept[AnalysisException](sql(subqueryQuery)),
+          condition = 
"UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.SCALAR_SUBQUERY_IN_VALUES",
+          parameters = Map.empty,
+          context = contextOf(subqueryQuery, s"VALUES ($subquery)"))
+      }
+    }
+  }
+}
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala 
b/sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
index 2aa37ba2fa6e..a83059664162 100644
--- a/sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
+++ b/sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
@@ -4965,18 +4965,21 @@ class SQLQuerySuite extends SharedSparkSession with 
AdaptiveSparkPlanHelper
   }
 
   test("SPARK-49659: Unsupported scalar subqueries in VALUES") {
-    checkError(
-      exception = intercept[AnalysisException](
-        sql("SELECT * FROM VALUES ((SELECT 1) + (SELECT 2))")
-      ),
-      condition = 
"UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.SCALAR_SUBQUERY_IN_VALUES",
-      parameters = Map(),
-      context = ExpectedContext(
-        fragment = "VALUES ((SELECT 1) + (SELECT 2))",
-        start = 14,
-        stop = 45
+    withSQLConf(SQLConf.INLINE_TABLE_NON_FOLDABLE_EXPRESSIONS_ENABLED.key -> 
"false") {
+      checkError(
+        exception = intercept[AnalysisException](
+          sql("SELECT * FROM VALUES ((SELECT 1) + (SELECT 2))")
+        ),
+        condition = 
"UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.SCALAR_SUBQUERY_IN_VALUES",
+        parameters = Map(),
+        context = ExpectedContext(
+          fragment = "VALUES ((SELECT 1) + (SELECT 2))",
+          start = 14,
+          stop = 45
+        )
       )
-    )
+    }
+    checkAnswer(sql("SELECT * FROM VALUES ((SELECT 1) + (SELECT 2))"), Row(3))
   }
 
   test("SPARK-49743: OptimizeCsvJsonExpr does not change schema when pruning 
struct") {


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to