BewareMyPower opened a new pull request #7996:
URL: https://github.com/apache/pulsar/pull/7996
### Motivation
Support key based batching for C++ client. This is a catch up work on #4435.
In addition, currently the implementation of `BatchMessageContainer` is
coupling to `ProducerImpl` tightly. The batch message container registers a
timer to producer's executor and the timeout callback is also producer's
method. Even its `add` method could call `sendMessage` to send batch to
producer's pending queue. These should be producer's work.
### Modifications
- Add a `MessageAndCallbackBatch` to store a `MessageImpl` of serialized
single messages and a callback list.
- Add a `BatchMessageContainerBase` to provide interface methods and methods
like update/clear message number/bytes, create `OpSendMsg`.
- Let `ProducerImpl` manage the batch timer and determine whether to create
`OpSendMsg` from `BatchMessageContainerBase` and send.
- Make `BatchMessageContainer` inherit `BatchMessageContainerBase`, it only
manages a `MessageAndCallbackBatch`.
- Add a `BatchMessageKeyBasedContainer` that inherits
`BatchMessageContainerBase`, it manages a map of message key and
`MessageAndCallbackBatch`.
- Add a producer config to change batching type.
- Add some units tests for key based batching and a unit test for key shared
subscription.
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
This change added tests and can be verified as follows:
- Current tests affected by default batching type, like
`BatchMessageTest.*` and `BasicEndToEndTest.*`
- Newly added tests: `KeyBasedBatchingTest.*` and
`KeySharedConsumerTest.testKeyBasedBatching`
### Documentation
- Does this pull request introduce a new feature? (yes)
- If yes, how is the feature documented? (docs)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
