lszskye opened a new pull request, #350:
URL: https://github.com/apache/paimon-cpp/pull/350
### Purpose
Linked issue: #174
Improve INPUT changelog write performance by writing data and changelog
files concurrently.
This change:
- writes changelog files asynchronously while data files are written on
the caller thread;
- waits for both write pipelines before publishing their results;
- aborts both writers when either pipeline fails;
- adds `changelog-producer.input.parallel-write`, enabled by default;
- preserves the previous sequential behavior when the option is disabled;
- extracts common sort-merge writing logic;
- reuses a shared helper when creating data and changelog rolling writers.
### Tests
- Extended `CoreOptionsTest` to verify the default and configured values
of the new option.
- Updated `MergeTreeWriterTest.TestInputChangelog` to support
nondeterministic file numbering during parallel writes.
- Added `MergeTreeWriterTest.TestInputChangelogWithParallelWriteDisabled`
to verify the sequential fallback and file creation order.
### API and Format
This change adds the public option
`changelog-producer.input.parallel-write`.
It does not change the storage format or protocol.
### Generative AI tooling
Generated-by: OpenAI Codex (GPT-5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
