[Wikidata-bugs] [Maniphest] [Commented On] T171928: Wikidata and dewiki databases locked

[jcrespo]

jcrespo added a comment.

I have started working on more complete monitoring, useful if we go over the route of human monitoring rather than automation, here is one example:

$ ./check_mariadb.py --icinga -h db1052.eqiad.wmnet --check_read_only=0
Version 10.0.28-MariaDB, Uptime 16295390s, read_only: False, s1 lag: 0.00s, 34 client(s), 886.72 QPS, connection latency: 0.075968s, query latency: 0.001053s

$ ./check_mariadb.py --icinga -h db1052.eqiad.wmnet --check_read_only=1
CRIT: read_only: "False", expected "True"; OK: Version 10.0.28-MariaDB, Uptime 16295413s, s1 lag: 0.00s, 40 client(s), 615.42 QPS, connection latency: 0.069848s, query latency: 0.000987s

It is configurable, so it can be adapted to the best possible monitoring we decide:

./check_mariadb.py --icinga -h db1052.eqiad.wmnet --check_read_only=0 --check_warn_lag=-1
WARN: s1 lag is 0.31s; OK: Version 10.0.28-MariaDB, Uptime 16295662s, read_only: False, 32 client(s), 761.35 QPS, connection latency: 0.066061s, query latency: 0.001263s

Code coming soon.

TASK DETAIL

https://phabricator.wikimedia.org/T171928

EMAIL PREFERENCES

https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: jcrespo
Cc: mark, Marostegui, Elitre, Joe, jcrespo, greg, Mbch331, Smalyshev, MisterSynergy, TerraCodes, Jay8g, Liuxinyu970226, Lydia_Pintscher, Aklapper, Esc3300, PokestarFan, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, Zppix, Johan, Izno, Luke081515, Wikidata-bugs, aude, ArielGlenn, faidon, He7d3r, TheDJ, fgiunchedi, mmodell

_______________________________________________
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs