pallavithegod opened a new pull request, #4446: URL: https://github.com/apache/amoro/pull/4446
## Why are the changes needed? Addresses #1121 for Spark's mixed-table catalog entry points. Spark `ALTER TABLE` currently forwards user property changes to mixed-table commits without restricting internal transaction, partition, and watermark bookkeeping. Changing or removing these values can invalidate table metadata or interfere with consistency tracking. Validate the complete property-change list before loading or mutating a table in each supported mixed-catalog implementation. Reject requests containing an internal bookkeeping property before any part can be committed. This covers both direct mixed-catalog registration and unified-catalog delegation across Spark 3.3, 3.4, and 3.5. ## Brief change log - Add shared validation for `table.partition-properties`, `base.table.max-transaction-id`, `max-txId`, `watermark.table`, and `watermark.base`. - Apply the gate to both SET and UNSET operations in all three Spark catalog implementations. - Preserve supported location, format-version, name-mapping, ordinary configuration, and schema changes. - Keep low-level table commit APIs available to runtime maintenance. - Document the Spark boundary and add shared regression cases exercised through each version's actual catalog entry point. ## How was this patch tested? - [x] Added new unit test cases that verify the changes (`TestMixedCatalogPropertyProtection`). - [x] Verified local builds, test compilation, and the selected tests with Spark 3.3, 3.4, and 3.5; 16 cases passed per profile. - [ ] Tested manually in local environment. Using Java 11: - `./mvnw spotless:apply checkstyle:check` passed. - For each version, ran `./mvnw -Pspark-<version> test -pl amoro-format-mixed/amoro-mixed-spark/v<version>/amoro-mixed-spark-<version> -am -Dtest=TestMixedCatalogPropertyProtection -Dsurefire.failIfNoSpecifiedTests=false`. - The 11 public-entry rejection cases failed against unmodified master: the calls reached catalog access instead of rejecting the internal property first. The five validation-control cases exercise the new shared gate and are not part of that baseline run. This PR addresses the Spark portion of #1121; it does not close the umbrella issue for other interfaces. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
