eskabetxe commented on PR #20097: URL: https://github.com/apache/flink/pull/20097#issuecomment-1322026302
Hi @wanglijie95 thanks for reviewing this.. 1. The idea for JdbcProducer is to allow that we can generate a new producer if needed, for example have a clickhouse producer based on BalancedClickhouseDataSource. So based on this, we will have by default JdbcProducer/NonXaJdbcProducer, JdbcProducer/XaJdbcProducer allowing other implementations without rewriting again common code. I will check the use of JdbcOutputFormat for NonXa. 2. It correct, its there by design but it not use them. It should be the implementation of JdbcProducer that guarantee the delivery guarantee.. So for NonXa it will ignore the TwoPhaseCommittingSink or use them to commit what is pending, as it should give AtLeastOnce guarantee and for XA it will use them to give the ExactlyOnce guarantee 3. Yes is in current plan, but the current implementation of XA used on SinkFunction use the RuntimeContext that is not available on Sink2, so we will need to reimplement it. I talk to @MartijnVisser about this and he ask me if we can use the [pre/proCommit](https://cwiki.apache.org/confluence/display/FLINK/FLIP-191%3A+Extend+unified+Sink+interface+to+support+small+file+compaction) introduced on Sink2, and as Im not an expert on XA neither on the changes introduced in pre/proCommits here we are.. If this approach is wrong we could go with the 2 sinks approach, as you comment. But I believe that allowing to extend/modify the logic more related to the JdbcClient could be beneficial in some cases. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
