elukey added a subscriber: aaron.
elukey added a comment.
Another big SET bursts to mc1022 caused timeouts for the MW appservers..
Don't meant to pressure you guys but is there any plan forward to fix this?
The SET bursts should be due to a key either being evicted or expired:
elukey@mc1022:~$ cat slab_140 | grep -i wikidata
ITEM
wikibase_shared/1_33_0-wmf_20-wikidatawiki-hhvm:CacheAwarePropertyInfoStore
[100621 b; 1552491470 s]
ITEM
wikibase_shared/1_33_0-wmf_20-wikidatawiki-hhvm:WikiPageEntityRevisionLookup:Q56701805
[99611 b; 1552430549 s]
elukey@mc1022:~$ date -d @1552491470
Wed Mar 13 15:37:50 UTC 2019 <================= TTL 24h should
match with this being just re-created
elukey@mc1022:~$ date -d @1552430549
Tue Mar 12 22:42:29 UTC 2019
elukey@mc1022:~$ date
Tue Mar 12 15:39:31 UTC 2019
Now I am wondering how the key gets re-set, it might be something to tune in
the usage of the MWWanCache. Adding @aaron as FYI :)
Mcrouter timeouts/TKO registered:
https://grafana.wikimedia.org/d/000000549/mcrouter?panelId=9&fullscreen&orgId=1&var-source=eqiad%20prometheus%2Fops&var-cluster=appserver&var-instance=All&from=1552402906212&to=1552405067164
Slab 140 SET traffic:
https://grafana.wikimedia.org/d/000000317/memcache-slabs?panelId=49&fullscreen&orgId=1&from=1552403014459&to=1552404751954&var-datasource=eqiad%20prometheus%2Fops&var-cluster=memcached&var-instance=mc1022&var-slab=All
TASK DETAIL
https://phabricator.wikimedia.org/T97368
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Addshore, elukey
Cc: aaron, elukey, Lydia_Pintscher, Jonas, Stashbot, gerritbot,
Jdforrester-WMF, Joe, mark, Addshore, jijiki, akosiaris, Krinkle, JanZerebecki,
thiemowmde, aude, daniel, Aklapper, hoo, alaa_wmde, Legado_Shulgin, CucyNoiD,
Nandana, NebulousIris, thifranc, AndyTan, Gaboe420, Versusxo,
Majesticalreaper22, Giuliamocci, Davinaclare77, Adrian1985, Qtn1293, Cpaulf30,
Imarlier, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis,
GoranSMilovanovic, Adik2382, Th3d3v1ls, Hfbn0, Ramalepe, Liugev6, QZanden,
LawExplorer, Vali.matei, Lewizho99, Zppix, Maathavan, _jensen, rosalieper,
Wong128hk, Wikidata-bugs, ArielGlenn, faidon, He7d3r, Mbch331, Jay8g, fgiunchedi
_______________________________________________
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
