elephone-184 opened a new pull request, #5041:
URL: https://github.com/apache/rocketmq-dashboard/pull/5041
## What is the purpose of the change
In RocketMQ, producers often use custom `MessageQueueSelector` logic (e.g.
`SelectMessageQueueByHash`) to route partitioned or ordered messages. Over
time, skewed hash keys or low-cardinality routing IDs cause severe traffic
hotspots on a single queue (e.g. 95% of traffic hitting queue 3 while queues
0-2 sit idle). This leads to localized broker disk saturation, consumer thread
starvation, and unpredictable latency.
This PR introduces an automated `TopicTrafficSkewDetector`:
1. `TopicTrafficSkewReportVO` & `QueueTrafficDistributionVO`: Domain report
capturing total topic message volumes, Gini inequality coefficient (0.0 to
1.0), standard deviation, hotspot indicators, and traffic share percentages.
2. `TopicTrafficSkewDetector`: Queries real `TopicQueueStatsVO` across all
broker partitions. It calculates the statistical Gini coefficient of message
distribution, classifies skew severity (`NORMAL`, `MODERATE_SKEW` >= 0.40,
`SEVERE_HOTSPOT` >= 0.70), flags specific hot partitions (traffic share > 2.5x
expected average), and generates remediation advice (e.g. salting keys,
reviewing queue selector entropy).
3. Service & Controller wiring: Integrates through
`MetadataService.auditTopicTrafficSkew` and exposes
`/api/topics/{name}/traffic-skew` in `TopicController`.
4. Comprehensive unit test suite covering balanced distributions, severe
hotspots, moderate skew, single-queue topics, zero-message topics, and
controller/service integration.
## Brief changelog
- Add `TopicTrafficSkewReportVO.java` domain model.
- Add `TopicTrafficSkewDetector.java` statistical Gini skew detection engine.
- Add `auditTopicTrafficSkew` in `MetadataService`.
- Expose `/api/topics/{name}/traffic-skew` endpoint in `TopicController`.
- Add `TopicTrafficSkewDetectorTest.java` and expand
`MetadataServiceTest.java` and `TopicControllerTest.java`.
## Verifying this change
- `TopicTrafficSkewDetectorTest`: verifies Gini coefficient math, standard
deviation, single queue edge case, balanced distribution (Gini=0), and hotspot
thresholding.
- `MetadataServiceTest`: verifies metadata service delegation to skew
detector.
- `TopicControllerTest`: verifies `/api/topics/{name}/traffic-skew` HTTP
REST endpoint mapping.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
