| Gehel added a comment. |
Looking at Grafana I can see spikes in batch progress that correlate with drops in lag. Zooming in, I can even see negative drops into batch progress, which should not happen. I suspect our metrics are skewed by the non monotonic nature of kafka updates (just a guess). Since we alert on Lag, having those drops is problematic, since it makes the alert flap and does not play nice with acknowledgments. And obviously, what we report is probably wrong, or at least unexpected.
TASK DETAIL
EMAIL PREFERENCES
To: Gehel
Cc: Stashbot, Smalyshev, Mathew.onipe, Gehel, Aklapper, Nandana, AndyTan, Davinaclare77, Qtn1293, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Jay8g, fgiunchedi
Cc: Stashbot, Smalyshev, Mathew.onipe, Gehel, Aklapper, Nandana, AndyTan, Davinaclare77, Qtn1293, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Jay8g, fgiunchedi
_______________________________________________ Wikidata-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
