JingsongLi commented on code in PR #8446:
URL: https://github.com/apache/paimon/pull/8446#discussion_r3518657632
##########
paimon-spark/paimon-spark-common/src/main/scala/org/apache/paimon/spark/commands/MergeIntoPaimonDataEvolutionTable.scala:
##########
@@ -71,23 +72,28 @@ case class MergeIntoPaimonDataEvolutionTable(
matchedActions: Seq[MergeAction],
notMatchedActions: Seq[MergeAction],
notMatchedBySourceActions: Seq[MergeAction])
- extends PaimonLeafRunnableCommand
- with WithFileStoreTable
- with ExpressionHelper
+ extends PaimonRowLevelCommand
with Logging {
- private lazy val writer = PaimonSparkWriter(table)
+ override lazy val writer: PaimonSparkWriter = PaimonSparkWriter(table)
Review Comment:
This override bypasses the `PaimonRowLevelCommand` writer, which switches
fixed-bucket non-PK tables to `WRITE_ONLY` to avoid running compaction while
deletion vectors are generated. With the new DELETE support, a MERGE can now
persist deletion vectors and write inserted/updated rows in the same operation
with compaction enabled, so the data file referenced by the new deletion vector
can be compacted away in the same commit. Please keep the inherited writer or
apply the same `WRITE_ONLY` handling here, and mirror the fix in the Spark 4.0
copy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
