lhotari opened a new pull request, #22494:
URL: https://github.com/apache/pulsar/pull/22494
Fixes #22477
### Motivation
There are multiple problems in the /metrics endpoint:
- requests are handled one-by-one and added into a queue
- when requests have timed out, they get processed regardless. time outs
are detected in the later phases of the processing. Instead, timed out requests
should be short-circuited in the beginning of the processing.
- processing is single threaded and therefore the throughput is low
- the metricsBufferResponse mode added in #14453 improves things, but it
contains some race conditions where a buffer can get released when it is in
use.
### Modifications
- fix the request timeout issue
- enable multithreading. currently fixed to 4 threads since higher
concurrency requires more memory and leads to OOM issues.
- by default, combine concurrent requests to reuse the same result. this
improves throughput even when `metricsBufferResponse` mode isn't enabled
- remove the previous `metricsBufferResponse` solution since the TimeWindow
and WindowWrap classes arean't needed at all. The concurrent request combining
solution can also be used for `metricsBufferResponse` solution.
- optimize some details of stats generation which were allocating a lot of
objects.
### Documentation
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
- [ ] `doc` <!-- Your PR contains doc changes. -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update
later -->
- [x] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
