JingsongLi commented on code in PR #9333:
URL: https://github.com/apache/paimon/pull/9333#discussion_r3827438647
##########
paimon-spark/paimon-spark-common/src/main/scala/org/apache/paimon/spark/commands/MergeIntoPaimonDataEvolutionTable.scala:
##########
@@ -837,7 +837,7 @@ case class MergeIntoPaimonDataEvolutionTable(
mergeFields.filter(field => targetTable.output.exists(attr =>
attr.equals(field)))
val targetReadPlan =
- touchedFileTargetRelation.copy(targetRelation.table,
allReadFieldsOnTarget.toSeq)
+ touchedFileTargetRelation.copy(output = allReadFieldsOnTarget.toSeq)
Review Comment:
Could you apply the same change to
paimon-spark/paimon-spark-4.0/src/main/scala/org/apache/paimon/spark/commands/MergeIntoPaimonDataEvolutionTable.scala?
Spark 4.0 compiles its own copy of this class, and its insertActionInvoke
still calls touchedFileTargetRelation.copy(targetRelation.table, ...). That
replaces the KnownSplitsTable with the original target table, so WHEN NOT
MATCHED continues to scan the full target table on Spark 4.0. A regression
assertion on the insert-side scan or file count would also prevent the two
implementations from drifting again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
