Aitozi opened a new pull request, #8256: URL: https://github.com/apache/paimon/pull/8256
### Purpose Primary-key table writers currently need to scan existing metadata to initialize the max sequence number. For write-only workloads this can be heavier than necessary because the writer only needs a safe starting sequence number. ### What changed This PR adds a `sequence.generation.mode` option with two modes: - `scan`: keep the existing behavior and scan restored files for the max sequence number. - `snapshot`: persist the max sequence number in snapshot properties and use it to initialize later write-only writers. For write-only primary-key tables in `snapshot` mode, the writer can skip loading previous files once the latest snapshot carries the max sequence property. If the latest snapshot does not have the property yet, the writer scans once to bootstrap the snapshot property safely. ### Tests - `git diff --check` - `mvn -s ~/.m2/apache-community.xml -o package -Pgenerate-docs -pl paimon-docs -nsu -DskipTests -am` - `mvn -s ~/.m2/apache-community.xml -o -pl paimon-core -am -Pfast-build -DfailIfNoTests=false -Dtest=KeyValueFileStoreWriteTest test` - `mvn -s ~/.m2/apache-community.xml -o -pl paimon-flink/paimon-flink-common -am -Pfast-build -DfailIfNoTests=false -Dtest=PrimaryKeyFileStoreTableITCase#testWriteOnlySnapshotSequenceOverwritePreviousValue test` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
