mihailoale-db commented on code in PR #57081:
URL: https://github.com/apache/spark/pull/57081#discussion_r3553780248
##########
common/utils/src/main/resources/error/error-conditions.json:
##########
@@ -8142,8 +8142,8 @@
},
"UNSUPPORTED_EXPR_FOR_OPERATOR" : {
"message" : [
- "A query operator contains one or more unsupported expressions.",
- "Consider to rewrite it to avoid window functions, aggregate functions,
and generator functions in the WHERE clause.",
+ "The query operator <operator> contains one or more unsupported
expressions.",
Review Comment:
It would be possible to create a map if this was an allowlist but it's not:
```
def specialExpressionsInUnsupportedOperator(plan: LogicalPlan):
Seq[Expression] = {
val exprs = plan.expressions
val invalidExpressions = exprs.flatMap { root =>
root.collect {
case e: WindowExpression
if !plan.isInstanceOf[Window] => e
case e: AggregateExpression
if !(plan.isInstanceOf[Aggregate] ||
plan.isInstanceOf[Window] ||
plan.isInstanceOf[CollectMetrics] ||
onlyInLateralSubquery(plan)) => e
case e: Generator
if !(plan.isInstanceOf[Generate] ||
plan.isInstanceOf[BaseEvalPythonUDTF]) => e
}
}
invalidExpressions
}
```
It doesn't make sense to map every single operator to its operation. I would
still keep it as it is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
