zebehringer commented on issue #21794: URL: https://github.com/apache/pulsar/issues/21794#issuecomment-1874728613
I am able to get more consistent results with fewer timeouts if I change the _publisher_ config to flush batches after 100 milliseconds rather than the default of 1 millisecond. I think this creates a better guarantee that each batch always has 1000 messages, which shares a common factor with the batch size of the consumer. Since these messages are batched, ACKing batch messages requires (I think) all messages in a batch to be ACKed together before the batch can be removed. If that is true, then having a common factor for batch sizes across producer and consumer results in less partially-ACKed batches. However, I don't think that is a great workaround, since a consumer shouldn't have to know such things about the producer(s). I still think it is something with ACKing, because (with the default producer batch flush of 1ms) I still see `unackedMessages` = 50000 in topics stats right after the call to ACK all the first 50000 received messages -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
