JingsongLi commented on code in PR #10702:
URL: https://github.com/apache/paimon/pull/10702#discussion_r4491890593
##########
paimon-core/src/main/java/org/apache/paimon/schema/SchemaValidation.java:
##########
@@ -205,6 +206,24 @@ public static void validateTableSchema(TableSchema schema,
Set<String> dynamicOp
"Can not set %s on table without primary keys,
please define primary keys.",
CHANGELOG_PRODUCER.key()));
}
+ List<String> preserveFields =
options.changelogPreserveFieldOnRetract();
+ if (!preserveFields.isEmpty()) {
+ checkArgument(
+ options.mergeEngine() == MergeEngine.DEDUPLICATE,
+ "'%s' is only supported with 'merge-engine' =
'deduplicate'.",
+ CHANGELOG_PRODUCER_PRESERVE_FIELD_ON_RETRACT.key());
+ checkArgument(
+ changelogProducer == ChangelogProducer.LOOKUP,
+ "'%s' is only supported with 'changelog-producer' =
'lookup'.",
+ CHANGELOG_PRODUCER_PRESERVE_FIELD_ON_RETRACT.key());
+ for (String field : preserveFields) {
+ checkArgument(
+ schema.fieldNames().contains(field),
Review Comment:
[P2] Keep preservation references aligned with the selected schema on copies
The persisted rename is fixed, but table-copy paths still mix preservation
names from one schema with fields from another. In a valid lookup/deduplicate
primary-key table with `preserve-field-on-retract=event_value` and no
`sequence.field`, write a snapshot, rename that column to `renamed_value`, then
reopen/write: ordinary current reads succeed, but
`current.copy(scan.snapshot-id=old)` throws because the historical fields do
not contain `renamed_value`. Independently, the pre-rename table instance
calling `copyWithLatestSchema()` throws because the latest fields no longer
contain `event_value`. Both fail in this new validation loop.
These are actual filesystem-table reproductions; removing only the
preservation option from the corresponding copy makes each read return the
expected old/current row. The second path is also used by
`CompactRefresher.tryRefresh` when newer-schema files arrive, so an existing
dedicated compactor cannot refresh after this otherwise supported rename.
Please keep the validation and make these copies take the preservation
reference from their selected schema, with regressions for both historical
selection and latest-schema refresh.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]