nodece commented on PR #22380: URL: https://github.com/apache/pulsar/pull/22380#issuecomment-2026854789
> I know that it feels wrong, but changing this is a breaking change. https://github.com/apache/pulsar/pull/4273/files#diff-16c47c755ed17a62606a6b9a6b1dc65d5fe0baed21f6e3541df264bec81b83f6R328 introduces rate limit for replication, perhaps not considering the batch messages. We can discuss this on the dev mailing list. > Rate limiting by message count matters mainly for consumers since the number of messages impact the load on client side message processing. > In the case of replication, that is not a concern at all. Producer rate limiting is important, and which impacts the consumer, but for some users who use the full-mesh replication, the users care about this. When we use the batch size to request the token from the replication rate limiter, this will only reduce throughput, I think this is acceptable. > Isn't this a metrics issue? Can we fix this instead? `pulsar_replication_rate_out` from the local cluster, if an entry is a batch message, which includes multiple messages, but only records 1 msg by `msgOut.recordEvent(headersAndPayload.readableBytes());`. `pulsar_replication_rate_in` from the remote cluster, when receiving this batch messages, which counts the how many messages in the batch messages. so causes pulsar_replication_rate_in > pulsar_replication_rate_out -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
