ifndef-SleePy opened a new issue, #8220: URL: https://github.com/apache/paimon/issues/8220
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Motivation This is the umbrella issue for [PIP-30](https://cwiki.apache.org/confluence/display/PAIMON/PIP-30%3A+Improvement+For+Paimon+Committer+In+Flink) — moving Paimon's Flink committer from a downstream `CommitterOperator` to a JobManager-side `OperatorCoordinator`. The target is Flink jobs writing to unaware-bucket append tables, where this enables region failover and removes the commit-side fan-in shuffle for large-parallelism Flink-to-Paimon write jobs. Recent design discussion (with updated diagrams that didn't render cleanly on cwiki): - Mailing-list thread: https://lists.apache.org/thread/5n1mwc7wc3cfjogz56w9k5ql3zyl00lw - Updated design doc: https://docs.google.com/document/d/1asWKzoytfeB1D8bS_yRIAHnpR40frLf0fnPn2-WSL74/edit ### Solution The work will be split across a series of PRs to keep each one independently reviewable. Tentative breakdown (subject to change as reviews progress): - [ ] Decouple `Committer.Context` from operator-only assumptions. - [ ] Introduce `WriteOperatorCoordinator` as the JobManager-side commit driver. - [ ] Wire the unaware-bucket append table to coordinator-based commit. - [ ] Follow-ups (performance polish, documentation) as needed. Links to per-PR issues will be added here as they are opened. ### Anything else? _No response_ ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
