zhuxiangyi commented on code in PR #8334:
URL: https://github.com/apache/paimon/pull/8334#discussion_r3795039929
##########
paimon-spark/paimon-spark-common/src/main/scala/org/apache/paimon/spark/commands/MergeIntoPaimonDataEvolutionTable.scala:
##########
@@ -541,7 +611,12 @@ case class MergeIntoPaimonDataEvolutionTable(
if (rawBlobUpdateColumns.exists(_.sameRef(attr))) {
Literal(null, attr.dataType)
} else {
- attr
+ prunedByExprId.get(attr.exprId) match {
+ case Some((paths, _)) =>
+ val st = attr.dataType.asInstanceOf[StructType]
+ buildPrunedStruct(st, Nil, paths, p => passthroughExpr(attr,
st, p))
Review Comment:
Good catch. Guarded in copyOutput, and in updateOutput only for the
pure-passthrough case — applying it unconditionally there would make SET
t.nest.a = 5 on a NULL nest silently drop the assignment. Both have regression
tests; Spark 4 copy updated.
Your repro shape mattered here too: the row must be matched but skipped by
its clause, otherwise it keeps its nullness from the base file and doesn't
expose the bug.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
