GWphua opened a new issue, #19505:
URL: https://github.com/apache/druid/issues/19505
### Affected Version
v32 (Caused by #15757)
### Description
We are deploying our clusters using the MM-less Kubernetes extension.
Recently, as we are upgrading our clusters from v27 to v37, we experienced a
query degradation in `segmentMetadata` queries. We have yet to establish
whether other query types are affected too.
#### Peon Querying Performance
- After upgrading to v37, performance degradation found when querying on
realtime tasks.
- Currently, the cause is found on `segmentMetadata` queries, it might be
possible to face performance degradation on other query types.
- Identified cause to be [PR
#15757](https://github.com/apache/druid/pull/15757), created to address memory
load when serving `GroupBy` queries.
- While [PR #17170](https://github.com/apache/druid/pull/17170) tried to
mitigate the issue, the performance is still sub-optimal.
The following benchmark is done locally on
`SinkQuerySegmentWalkerBenchmark`, measuing ms/op for differing number of
hydrants per sink.
| | 10 | 50 | 100 | 200 |
| :----------------------------- | :---- | :---- | :---- | :---- |
| v27 | 0.089 | 0.162 | 0.265 | 0.493 |
| before-pr-15757 | 0.089 | 0.165 | 0.274 | 0.479 |
| after-pr-15757-before-pr-17170 | 0.519 | 2.351 | 4.847 | 9.112 |
| after-pr-17170 | 0.231 | 0.784 | 1.443 | 2.791 |
| v37 | 0.228 | 0.825 | 1.499 | 2.930 |
At 200 hydrants, #15757 regresses this benchmark (~19x slower) from 0.479
ms/op to 9.112 ms/op; #17170 improves it to 2.791 ms/op, but that is still
about 5.8x slower than before #15757.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
