szehon-ho commented on code in PR #53360:
URL: https://github.com/apache/spark/pull/53360#discussion_r2617518112
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/AssignmentUtils.scala:
##########
@@ -204,14 +228,71 @@ object AssignmentUtils extends SQLConfHelper with
CastSupport {
}
val updatedFieldExprs = fieldAttrs.zip(fieldExprs).map { case
(fieldAttr, fieldExpr) =>
applyAssignments(fieldAttr, fieldExpr, assignments, addError,
colPath :+ fieldAttr.name,
- coerceNestedTypes)
+ coerceNestedTyptes)
+ }
+ toNamedStruct(structType, updatedFieldExprs)
+
+ case otherType =>
+ addError(
+ "Updating nested fields is only supported for StructType but " +
+ s"'${colPath.quoted}' is of type $otherType")
+ colExpr
+ }
+ }
+
+ private def applyNestedFieldAssignments(
Review Comment:
i just chatted with @aokolnychyi on the difference, and wanted to clarify
here as well. I mean, the existing method `applyFieldAssignments` is obviously
recursive, but it's goal is not to make assignment for every nested field, but
just missing siblings at the existing assignment levels. For example, we have
update x.y.z = source.x.y.z. It will recurse down to evaluate x.y struct, see
the assignment for x.y.z. It will fill missing assignments for z's siblings,
ie if there was x.y.z1 and x.y.z2 it will fill those
The new method on the other hand 'explodes' an assignment for the top level
struct x (if its from UPDATE SET *) into assignment for every single field, ie
not only the missing siblings of existing assignments. Hope that is more clear
on the difference.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
