% slabtop -s c --once |head -10
Active / Total Objects (% used) : 185625 / 205302 (90.4%)
Active / Total Slabs (% used) : 16756 / 16757 (100.0%)
Active / Total Caches (% used) : 101 / 182 (55.5%)
Active / Total Size (% used) : 856865.31K / 859451.65K (99.7%)
Minimum / Average / Maximum Object : 0.02K / 4.19K / 4096.00K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
384 384 100% 2048.00K 384 1 786432K size-2097152
26276 26250 99% 1.00K 6569 4 26276K ext4_inode_cache
181 181 100% 32.12K 181 1 11584K kmem_cache
very different indeed.
(btw size-2097152 sounds like one of the default name used by hugectl (or
hugeadm) from the hugetlbfs tools). is that mounted in your case? and are there
any hugepages reserved? )
Not that I'd know of. But wasn't there a new feature called "transparent hugepage
support" in 6.1?
thp shows in /proc/meminfo as AnonHugePages
]# grep Huge /proc/meminfo
AnonHugePages: 18432 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
you show check those there too to see where the come from
stijn
Cheers,
Stephan
stijn
On 09/05/2011 05:10 PM, Stephan Wiesand wrote:
Hi Stijn,
On Sep 5, 2011, at 16:24, Stijn De Weirdt wrote:
hi all,
we are having an "issue" with some SL61 nodes. after a reboot, free reports
1.4GB of memory in use, of which 24+163=187MB buffers+cache.
i'm unable to identify what is holding the memory, and i'd like to know if
others see this too and how i could proceed to find the culprit.
yes, we see this as well. On a 48 GB system without users or special processes:
# free -m
total used free shared buffers cached
Mem: 48388 1374 47013 0 30 186
-/+ buffers/cache: 1157 47231
In /proc/meminfo, I find that the difference to what I'd consider reasonable
(and see on a 48GB SL5 system) is due to slabs.
A "slabtop -s c" reveals that it's a "size-2097152" pool accounting for this.
Do you see this as well?
Cheers,
Stephan
(it is a 32core/64GB machine; kernel commandline has crashkernel=128M@16M (but
no difference then eg crashkernel=auto and kdump is off))
many thanks,
stijn
free
# free -m
total used free shared buffers cached
Mem: 64554 1604 62949 0 24 166
-/+ buffers/cache: 1413 63140
Swap: 16394 0 16394
mem sorted top
top - 16:13:52 up 13 min, 1 user, load average: 0.00, 0.01, 0.01
Tasks: 694 total, 1 running, 693 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 66103768k total, 1643336k used, 64460432k free, 25164k buffers
Swap: 16787916k total, 0k used, 16787916k free, 170552k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2788 root 20 0 37988 25m 2876 S 0.0 0.0 0:00.06 pbs_mom
2653 root 20 0 159m 12m 1472 S 0.0 0.0 0:00.19 ncm-cdispd
2643 root 20 0 138m 5604 840 S 0.0 0.0 0:00.00 cdp-listend
3276 root 20 0 120m 4156 3232 S 0.0 0.0 0:00.07 sshd
2620 root 20 0 745m 3788 1764 S 0.0 0.0 0:00.12 automount
3102 nslcd 20 0 427m 2936 488 S 0.0 0.0 0:00.00 nslcd
3301 root 20 0 103m 1688 1336 S 0.0 0.0 0:00.05 bash
3623 root 20 0 13528 1604 844 R 0.3 0.0 0:00.14 top
1 root 20 0 21416 1544 1240 S 0.0 0.0 0:06.23 init
2482 root 20 0 194m 1484 1108 S 0.0 0.0 0:00.14 qlgc_dsc
2325 root 20 0 242m 1412 928 S 0.0 0.0 0:00.04 rsyslogd
2459 rpcuser 20 0 23112 1168 884 S 0.0 0.0 0:00.00 rpc.statd
2606 root 18 -2 10956 1144 412 S 0.0 0.0 0:00.03 udevd
3164 nscd 20 0 583m 1132 788 S 0.0 0.0 0:00.02 nscd
2697 root 20 0 62040 1064 464 S 0.0 0.0 0:00.00 sshd
943 root 16 -4 10960 1052 316 S 0.0 0.0 0:00.12 udevd
2607 root 18 -2 10956 1052 320 S 0.0 0.0 0:00.00 udevd
2723 root 20 0 112m 1012 380 S 0.0 0.0 0:00.00 crond
2707 root 20 0 22488 992 752 S 0.0 0.0 0:00.03 xinetd
2439 rpc 20 0 18940 908 672 S 0.0 0.0 0:00.04 rpcbind
2568 dbus 20 0 23448 876 604 S 0.0 0.0 0:00.01 dbus-daemon
2972 nagios 20 0 37096 796 452 S 0.0 0.0 0:00.00 nrpe
