orangeCatDeveloper opened a new pull request, #4328:
URL: https://github.com/apache/hertzbeat/pull/4328
Fixes #1275
After restarting HertzBeat, website monitors report multi-second response
times for a long stretch even though the monitored sites are fine.
Root cause: a restart re-adds every cyclic job in the same instant, and each
one gets the same first-run delay (its full interval). From then on all
monitors collect at the exact same second, every cycle, forever — the burst of
concurrent requests contends for collector threads and target capacity,
inflating every measured response time.
The fix gives interval-scheduled jobs a random first-run phase inside their
interval, so a restart spreads them across the cycle instead of stacking them.
Two deliberate exclusions: cron jobs keep their meaningful fixed phase, and
already-executed / re-added jobs (config updates) keep their current phase.
`TimerDispatcherTest` pins the jitter range and both exclusions.
Reproduced end-to-end — 25 website monitors, 10s interval, against a
single-threaded local server (200ms/request), restarting the backend with and
without the fix:
```
before restart (no fix): all 25 monitors fire in the same second every cycle
n=1068 p50=2865ms p95=6801ms max=7132ms
after restart (with fix): same load, phases spread across the interval
n=1600 p50= 617ms p95=1497ms max=4931ms
```
Timeline below — the dashed line is the restart with the fix; the residual
sparse outliers on the right are ordinary queueing at 50% utilization of the
deliberately weak test server, not re-clustering (per-second dispatch counts
stay spread).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
