srielau commented on code in PR #53664:
URL: https://github.com/apache/spark/pull/53664#discussion_r2707520858
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala:
##########
@@ -870,13 +870,34 @@ trait CheckAnalysis extends LookupCatalog with
QueryErrorsBase with PlanToString
!o.isInstanceOf[MapInPandas] &&
!o.isInstanceOf[MapInArrow] &&
// Lateral join is checked in checkSubqueryExpression.
- !o.isInstanceOf[LateralJoin] =>
+ !o.isInstanceOf[LateralJoin] &&
+ // ResolvedInlineTable allows non-deterministic expressions when
+ // spark.sql.legacy.values.onlyFoldableExpressions is false (the
default).
+ !o.isInstanceOf[ResolvedInlineTable] =>
// The rule above is used to check Aggregate operator.
o.failAnalysis(
errorClass = "INVALID_NON_DETERMINISTIC_EXPRESSIONS",
messageParameters = Map("sqlExprs" ->
o.expressions.map(toSQLExpr(_)).mkString(", "))
)
+ // ResolvedInlineTable with OuterReference is only allowed in
LATERAL contexts
Review Comment:
Good point, fixed:
* Changed from LateralSubquery to SubqueryExpression in the rewrite rule
* Removed CheckAnalysis validation that was too restrictive
* Simplified validateInputEvaluable to allow unresolved attributes that
might become OuterReference
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
