jordepic opened a new pull request, #9762:
URL: https://github.com/apache/paimon/pull/9762
### Purpose
Primary-key compaction chooses its rewriter inside a private factory. An
external engine currently has to reconstruct the writer setup to replace the
file read/merge/write work with its own implementation.
Add `TableWrite.withCompactRewriterFactory(...)`, backed by the existing
`CompactRewriter` boundary. Each partition/bucket receives its own rewriter,
and the factory receives Paimon's normal rewriter so implementations can
delegate unsupported operations. Paimon retains input selection, background
scheduling, and commit coordination. This PR adds the extension point; it does
not add a native compaction engine.
```java
write.withCompactRewriterFactory((partition, bucket, defaultRewriter) ->
new CustomCompactRewriter(partition, bucket, defaultRewriter));
```
The API documents ownership and recovery: the returned rewriter owns the
default, failed creation closes the default, and the factory must be installed
before bucket creation and again on recovered writers. Write-only mode does not
invoke the factory; append, postpone, and primary-key clustering writers reject
it. The Java writing guide includes the integration contract.
cc @JingsongLi — feedback on exposing this boundary for external/native
engines would be appreciated.
### Tests
24 focused Core tests passed on JDK 11, with normal Checkstyle, Spotless,
and enforcer checks enabled:
```bash
mvn -pl paimon-core -am -Pflink1 \
-DwildcardSuites=none \
-Dtest=CompactRewriterFactoryTest,MergeTreeCompactManagerFactoryTest,TableWriteTest,KeyValueFileStoreWriteTest
\
-DfailIfNoTests=false -Dsurefire.failIfNoSpecifiedTests=false test
```
Coverage includes real rewrites and metadata upgrades across reopened
writers; fallback selection for none/input/lookup/full-compaction and deletion
vectors; final table contents; rewriter closure; write-only behavior;
unsupported append writers; late installation; and cleanup/suppressed
exceptions when factory creation fails.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
