junmuz commented on code in PR #10702:
URL: https://github.com/apache/paimon/pull/10702#discussion_r4404486900


##########
paimon-core/src/main/java/org/apache/paimon/schema/SchemaValidation.java:
##########
@@ -205,6 +206,24 @@ public static void validateTableSchema(TableSchema schema, 
Set<String> dynamicOp
                             "Can not set %s on table without primary keys, 
please define primary keys.",
                             CHANGELOG_PRODUCER.key()));
         }
+        List<String> preserveFields = 
options.changelogPreserveFieldOnRetract();
+        if (!preserveFields.isEmpty()) {
+            checkArgument(
+                    options.mergeEngine() == MergeEngine.DEDUPLICATE,
+                    "'%s' is only supported with 'merge-engine' = 
'deduplicate'.",
+                    CHANGELOG_PRODUCER_PRESERVE_FIELD_ON_RETRACT.key());
+            checkArgument(
+                    changelogProducer == ChangelogProducer.LOOKUP,

Review Comment:
   The internal rescale copy in SparkPostponeStagedCommitter now has 
changelog-producer.preserve-field-on-retract removed. That copy forces 
changelog-producer=none and writes no changelog, so the option has no effect 
there. The CREATE/ALTER restriction is unchanged, and the table itself keeps 
the option. This is the only place that forces changelog-producer=none on a 
table copy, so no other internal copies need the same change. 
   
   I added a Spark test to PostponeBucketTableTest. It uses a postpone 
fixed-bucket table with changelog-producer=lookup and preserve-field-on-retract 
enabled. The test writes enough rows to trigger a load-factor rescale from 1 to 
16 buckets. It then upserts an existing key and checks the table contents and 
options. Without the fix, the test fails with the reported validation error 
from the SparkPostponeStagedCommitter constructor. With the fix, the whole 
suite passes on Spark 3.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to