Jayer23 commented on issue #24010:
URL: https://github.com/apache/pulsar/issues/24010#issuecomment-2684199660
> You probably have some naughty partitions with heavy write throughput, you
can check by following Prometheus query:
>
> ```
> pulsar_throughput_in{cluster="YOUR_CLUSTER_NAME"} > 5 * 1024^2
> ```
>
> If you discover partitions with excessive write loads, consider increasing
the number of partitions for the affected topics. This will help distribute the
write throughput more evenly across your bookie instances.
>
> Notes about Pulsar message publishing:
>
> * Each partition maintains exactly one writable ledger at a time
> * With configuration `ensembleSize=2, writeQuorum=2`:
>
> * A partition with `100MB/s` write throughput will generate
approximately `100MB/s` write load on each bookie in the ensemble
> * Bookies not in the current ensemble will not receive any writes from
this ledger
1. There are only 3 partitions in the cluster with write rates exceeding
2MB/s
2. The producer has Lz4 compression . Does the Prometheus query threshold
need to be adjusted?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
