QuakeWang opened a new pull request, #7598:
URL: https://github.com/apache/paimon/pull/7598
### Purpose
Fix the dedicated-format writeBundle path so bundle writes remain correct
when data is fanned out to main/blob/vector writers.
This change:
- materializes only non-replayable bundles in the dedicated fan-out path
instead of always materializing them;
- introduces explicit opt-in bundle capabilities in common
(ReplayableBundleRecords and ProjectableBundleRecords) without changing the
base BundleRecords contract;
- allows projection to preserve typed bundle implementations such as
ArrowBundleRecords;
- preserves row-level side effects in bundle writes, including sequence
number updates and file-index writing.
### Tests
Added or updated tests in paimon-core:
- ProjectedFileWriterTest
- BundleAwareRowDataRollingFileWriterTest
- DedicatedFormatRollingFileWriterTest
- DedicatedFormatRollingFileWriterVectorTest
These cover:
- replayable/projectable bundle pass-through and fallback behavior;
- sequence-number side effects for bundle writes;
- dedicated-format bundle writes with main-file index side effects;
- external-storage blob fallback for bundle writes;
- single-use bundle writes for blob/vector dedicated paths.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
