michaeljmarshall opened a new issue #12069:
URL: https://github.com/apache/pulsar/issues/12069
**Is your feature request related to a problem? Please describe.**
The current Pulsar metrics implementation has very fine grained histogram
buckets. This leads to more metrics data, which means more compute to calculate
the metrics, more memory to store them, and then more cost downstream on the
service storing the historical metrics. Since different installations have
different requirements, it seems to me that we could make these buckets
configurable.
**Describe the solution you'd like**
Make the histogram buckets for metrics configurable. A user would simply
provide a comma delimited list of bucket values. We would need several new
configuration values for each type of histogram that is to be configured. It
looks like the affect values are: `ENTRY_LATENCY_BUCKETS_USEC`,
`ENTRY_SIZE_BUCKETS_BYTES`, and `WRITE_LATENCY_BUCKETS_USEC`. If we're
concerned about proliferation of configurations, we could combine the two
latency related bucket arrays in a single shared configuration.
**Describe alternatives you've considered**
Other solutions might include server side filters by metrics providers.
However, that would still mean that Pulsar has to calculate the metrics, which
would be unnecessary.
**Additional context**
Here are two examples of the current bucket definitions:
```java
public static final long[] ENTRY_LATENCY_BUCKETS_USEC = { 500, 1_000,
5_000, 10_000, 20_000, 50_000, 100_000,
200_000, 1000_000 };
public static final long[] ENTRY_SIZE_BUCKETS_BYTES = { 128, 512, 1024,
2048, 4096, 16_384, 102_400, 1_232_896 };
```
They have 9 and 8 buckets, respectively. The default values for these
configurations would be the current values to ensure backwards compatibility.
If this feature is accepted, I will follow up with a proposal to add a
similar configuration option to Apache Bookkeeper.
I'm happy to help contribute this feature.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
