ema added a comment.
`node_vmstat_nr_slab_unreclaimable` is going up indefinitely on nodes
affected by the issue, following a pattern that matches the general memory
usage. However, the actual amount of "lost" memory does not match the size of
unreclaimable slabs, which is only ~2G on mw1357:
root@mw1357:~# grep SUnreclaim /proc/meminfo
SUnreclaim: 2097548 kB
F32102914: Screenshot from 2020-08-13 15-04-23.png
<https://phabricator.wikimedia.org/F32102914>
We discussed two things to try out next:
- reboot 4.9 with memory accounting disabled (`cgroup.memory=nokmem`) to see
if we ran into
https://lwn.net/ml/linux-kernel/[email protected]/, in
particular considering that mem_cgroup_try_charge is featured often in the
`memcg_schedule_kmem_cache_create` traces that @cdanis suggested to gather (see
P12251 <https://phabricator.wikimedia.org/P12251> and P12252
<https://phabricator.wikimedia.org/P12252>)
- reboot with 4.19 from stretch-backports because it's a low hanging fruit
and 4.9 is very old
TASK DETAIL
https://phabricator.wikimedia.org/T260281
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: ema
Cc: Addshore, Majavah, Ladsgroup, JMeybohm, ema, Joe, RhinosF1, ArielGlenn,
jijiki, Aklapper, CDanis, lmata, wkandek, Akuckartz, darthmon_wmde, WDoranWMF,
holger.knust, EvanProdromou, Legado_Shulgin, Nandana, Klaas_Z4us_V,
Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, GoranSMilovanovic, Th3d3v1ls,
Hfbn0, QZanden, LawExplorer, Zppix, elukey, _jensen, rosalieper, Agabi10,
Scott_WUaS, Pchelolo, Wong128hk, Wikidata-bugs, aude, faidon, Mbch331, Rxy,
Jay8g, fgiunchedi, Dzahn
_______________________________________________
Wikidata-bugs mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
