ulysses-you commented on code in PR #5862:
URL: https://github.com/apache/incubator-gluten/pull/5862#discussion_r1613067064
##########
gluten-core/src/main/scala/org/apache/gluten/expression/ScalarSubqueryTransformer.scala:
##########
@@ -30,21 +31,7 @@ case class ScalarSubqueryTransformer(substraitExprName:
String, query: ScalarSub
if (TransformerState.underValidationState) {
return ExpressionBuilder.makeLiteral(null, query.dataType, true)
}
- // the first column in first row from `query`.
- val rows = query.plan.executeCollect()
- if (rows.length > 1) {
- throw new IllegalStateException(
- s"more than one row returned by a subquery used as an
expression:\n${query.plan}")
- }
- val result: AnyRef = if (rows.length == 1) {
- assert(
- rows(0).numFields == 1,
- s"Expects 1 field, but got ${rows(0).numFields}; something went wrong
in analysis")
- rows(0).get(0, query.dataType)
- } else {
- // If there is no rows returned, the result should be null.
- null
- }
+ val result = query.eval(InternalRow.empty)
Review Comment:
After this pr, we do not need to execute subquery manually so the exception
behavior is same with vanilla Spark. Note that, this code change is just for
simplify. The subquery has already been materialized before doing transform.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
