nicoloboschi opened a new pull request, #18017: URL: https://github.com/apache/pulsar/pull/18017
### Motivation Usually in high load systems the JDBC drivers are meant to used with the JDBC Batch API. Currently it's not implemented in the jdbc sink ### Modifications * New flag `useJdbcBatch` to turn on this feature (default to false) * If useJdbcBatch is on, then the sink will use ´addBatch()´ and `executeBatch()` methods * In case in the same messages batch (defined by batchSize or timeoutMs) there are different kind of operations, the batches are split in different batches operations but still in the same transaction. This means that if you use transactions the behaviour is semantically the same. If transactions are disabled (autocommit=true) the behaviour could be a little different since more statements could be executed than without this option. For example, if in the same batch you have two messages and first statement fail: A) no jdbc batch -> the first fails, the second fails. B) with jdbc batch -> the first goes in, the second fails. this is due to the JDBC batch nature. However it's recommended to turn on transacions if the driver has support for them. * Added more specific tests to verify error scenario behaviours and ensure that commits and msg ack are executed correctly for each message. ### Documentation <!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. --> - [x] `doc` <!-- Your PR contains doc changes. Please attach the local preview screenshots (run `sh start.sh` at `pulsar/site2/website`) to your PR description, or else your PR might not get merged. --> - [ ] `doc-required` <!-- Your PR changes impact docs and you will update later --> - [ ] `doc-not-needed` <!-- Your PR changes do not impact docs --> - [ ] `doc-complete` <!-- Docs have been already added --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
