Hi devs, I would like to start a discussion about PIP-30: Improvement For Paimon Committer In Flink [1].
Currently Flink writes data to Paimon based on Two-Phase Commit which will generate a global committer node and connect all tasks in one region. If any task fails, it will lead to a global failover in Flink job. To solve this issue, we would like to introduce a Paimon Writer Coordinator to perform table commit operation, enabling Flink paimon jobs to support region failover and improving stability. Looking forward to hearing from you, thanks! [1] https://cwiki.apache.org/confluence/display/PAIMON/PIP-30%3A+Improvement+For+Paimon+Committer+In+Flink Best, Fang Yong
