Re: memory usage displsy
In response to Per olof Ljungmark p...@intersonic.se: Dan Nelson wrote: In the last episode (Sep 01), Bill Moran said: In response to Per olof Ljungmark p...@intersonic.se: What is a good way to find out how memory is used? Have a 6.4 box where memory is used by something but I fail to see what is using it - tried different switches to ps(1), tried the stat tools but a big chunk of memory does not show at all. A proper tool for analyzing memory usage live, this is a production box? I've always been able to get what I need from top. You can do -o res to sort by resident memory usage, which helps. ps will sort by memory usage when given the -m flag. Also check ipcs -a to see if there are any sysv shared memory segments hanging arnound. If you don't see anything using the memory, where are you seeing that something is using it? ...and here is top output after I stopped Postfix, slapd and Cyrus-IMAP. Still over 3G Active. snip You did not sort by res and there are only 40 processes showing, which means your output is truncated and may have truncated the problematic process. Please use top -o res to get the output sorted by memory usage, or don't truncate the output (former preferred). Also, please provide the output of ipcs -a -- Bill Moran http://www.potentialtech.com http://people.collaborativefusion.com/~wmoran/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: memory usage displsy
Bill Moran wrote: In response to Per olof Ljungmark p...@intersonic.se: Dan Nelson wrote: In the last episode (Sep 01), Bill Moran said: In response to Per olof Ljungmark p...@intersonic.se: What is a good way to find out how memory is used? Have a 6.4 box where memory is used by something but I fail to see what is using it - tried different switches to ps(1), tried the stat tools but a big chunk of memory does not show at all. A proper tool for analyzing memory usage live, this is a production box? I've always been able to get what I need from top. You can do -o res to sort by resident memory usage, which helps. ps will sort by memory usage when given the -m flag. Also check ipcs -a to see if there are any sysv shared memory segments hanging arnound. If you don't see anything using the memory, where are you seeing that something is using it? ...and here is top output after I stopped Postfix, slapd and Cyrus-IMAP. Still over 3G Active. snip You did not sort by res and there are only 40 processes showing, which means your output is truncated and may have truncated the problematic process. Please use top -o res to get the output sorted by memory usage, or don't truncate the output (former preferred). Also, please provide the output of ipcs -a There was no more processes... ipcs -a Message Queues: T ID KEY MODEOWNERGROUPCREATOR CGROUP CBYTES QNUM QBYTES LSPIDLRPID STIMERTIMECTIME Shared Memory: T ID KEY MODEOWNERGROUPCREATOR CGROUP NATTCHSEGSZ CPID LPID ATIME DTIMECTIME Semaphores: T ID KEY MODEOWNERGROUPCREATOR CGROUP NSEMS OTIMECTIME ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: memory usage displsy
In response to Per olof Ljungmark p...@intersonic.se: Bill Moran wrote: In response to Per olof Ljungmark p...@intersonic.se: Dan Nelson wrote: In the last episode (Sep 01), Bill Moran said: In response to Per olof Ljungmark p...@intersonic.se: What is a good way to find out how memory is used? Have a 6.4 box where memory is used by something but I fail to see what is using it - tried different switches to ps(1), tried the stat tools but a big chunk of memory does not show at all. A proper tool for analyzing memory usage live, this is a production box? I've always been able to get what I need from top. You can do -o res to sort by resident memory usage, which helps. ps will sort by memory usage when given the -m flag. Also check ipcs -a to see if there are any sysv shared memory segments hanging arnound. If you don't see anything using the memory, where are you seeing that something is using it? ...and here is top output after I stopped Postfix, slapd and Cyrus-IMAP. Still over 3G Active. snip You did not sort by res and there are only 40 processes showing, which means your output is truncated and may have truncated the problematic process. Please use top -o res to get the output sorted by memory usage, or don't truncate the output (former preferred). Also, please provide the output of ipcs -a There was no more processes... From your top output: 45 processes: 1 running, 44 sleeping There were 40 processes listed, so there were 5 not shown. ipcs -a OK, this verifies that nothing is tied up in shared memory. -- Bill Moran http://www.potentialtech.com http://people.collaborativefusion.com/~wmoran/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: memory usage displsy
On Tuesday 01 September 2009 23:19:23 Michael David Crawford wrote: Per olof Ljungmark wrote: Well, my problem is that if I add up all I *can* see in top or ps it never gets near the by now 3G plus memory shown as Active. Maybe one gig is accounted for, I'm not that familiar with FreeBSD yet, but the kernel uses memory which might not be charged against any process. For example, to map some virtual memory requires memory to store the mappings in. Open files have kernel structures, as do filesystems. If top or ps were only to show userspace memory allocations, then you're right, a lot of memory would be unaccounted for. It doesn't for the Active to Free states. For individual processes, everything is shown that the process allocates. So for a file descriptor, an int would be allocated, where the kernel holds the real info. This is one cause for filled Active memory: a process polling multiple file descriptors, like a File Alteration Monitor under current desktops. The other, as Dan Nelson described, is file cache. If you want to be sure it's this, then reboot the machine and run: /etc/periodic/security/100.chksetuid You should see memory usage going up. If this causes a performance problem (i.e. You sometimes are subject to heavily increasing loads on a mailserver, that causes a lot of forks and file cache memory isn't unloaded fast enough), then you should either disable the security check or properly seperate data from binaries using partitions and mount data partitions with nosuid/noexec, so that these are omitted from the daily checks. -- Mel ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: memory usage displsy
Mel Flynn wrote: On Tuesday 01 September 2009 23:19:23 Michael David Crawford wrote: Per olof Ljungmark wrote: Well, my problem is that if I add up all I *can* see in top or ps it never gets near the by now 3G plus memory shown as Active. Maybe one gig is accounted for, I'm not that familiar with FreeBSD yet, but the kernel uses memory which might not be charged against any process. For example, to map some virtual memory requires memory to store the mappings in. Open files have kernel structures, as do filesystems. If top or ps were only to show userspace memory allocations, then you're right, a lot of memory would be unaccounted for. It doesn't for the Active to Free states. For individual processes, everything is shown that the process allocates. So for a file descriptor, an int would be allocated, where the kernel holds the real info. This is one cause for filled Active memory: a process polling multiple file descriptors, like a File Alteration Monitor under current desktops. The other, as Dan Nelson described, is file cache. If you want to be sure it's this, then reboot the machine and run: /etc/periodic/security/100.chksetuid You should see memory usage going up. If this causes a performance problem (i.e. You sometimes are subject to heavily increasing loads on a mailserver, that causes a lot of forks and file cache memory isn't unloaded fast enough), then you should either disable the security check or properly seperate data from binaries using partitions and mount data partitions with nosuid/noexec, so that these are omitted from the daily checks. Thank you all for the informative answers, helped a lot to understand better what is going on. I cannot run 100.chksetuid on a production server but I will definitely do it on the testing one. Cheers, -- per ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: memory usage displsy
In response to Per olof Ljungmark p...@intersonic.se: What is a good way to find out how memory is used? Have a 6.4 box where memory is used by something but I fail to see what is using it - tried different switches to ps(1), tried the stat tools but a big chunk of memory does not show at all. A proper tool for analyzing memory usage live, this is a production box? I've always been able to get what I need from top. You can do -o res to sort by resident memory usage, which helps. -- Bill Moran http://www.potentialtech.com http://people.collaborativefusion.com/~wmoran/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: memory usage displsy
In the last episode (Sep 01), Bill Moran said: In response to Per olof Ljungmark p...@intersonic.se: What is a good way to find out how memory is used? Have a 6.4 box where memory is used by something but I fail to see what is using it - tried different switches to ps(1), tried the stat tools but a big chunk of memory does not show at all. A proper tool for analyzing memory usage live, this is a production box? I've always been able to get what I need from top. You can do -o res to sort by resident memory usage, which helps. ps will sort by memory usage when given the -m flag. Also check ipcs -a to see if there are any sysv shared memory segments hanging arnound. If you don't see anything using the memory, where are you seeing that something is using it? -- Dan Nelson dnel...@allantgroup.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: memory usage displsy
Per olof Ljungmark wrote: Well, my problem is that if I add up all I *can* see in top or ps it never gets near the by now 3G plus memory shown as Active. Maybe one gig is accounted for, I'm not that familiar with FreeBSD yet, but the kernel uses memory which might not be charged against any process. For example, to map some virtual memory requires memory to store the mappings in. Open files have kernel structures, as do filesystems. If top or ps were only to show userspace memory allocations, then you're right, a lot of memory would be unaccounted for. Mike -- Michael David Crawford m...@prgmr.com prgmr.com - We Don't Assume You Are Stupid. Xen-Powered Virtual Private Servers: http://prgmr.com/xen ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: memory usage displsy
Bill Moran wrote: In response to Per olof Ljungmark p...@intersonic.se: What is a good way to find out how memory is used? Have a 6.4 box where memory is used by something but I fail to see what is using it - tried different switches to ps(1), tried the stat tools but a big chunk of memory does not show at all. A proper tool for analyzing memory usage live, this is a production box? I've always been able to get what I need from top. You can do -o res to sort by resident memory usage, which helps. Well, my problem is that if I add up all I *can* see in top or ps it never gets near the by now 3G plus memory shown as Active. Maybe one gig is accounted for, ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: memory usage displsy
Dan Nelson wrote: In the last episode (Sep 01), Bill Moran said: In response to Per olof Ljungmark p...@intersonic.se: What is a good way to find out how memory is used? Have a 6.4 box where memory is used by something but I fail to see what is using it - tried different switches to ps(1), tried the stat tools but a big chunk of memory does not show at all. A proper tool for analyzing memory usage live, this is a production box? I've always been able to get what I need from top. You can do -o res to sort by resident memory usage, which helps. ps will sort by memory usage when given the -m flag. Also check ipcs -a to see if there are any sysv shared memory segments hanging arnound. If you don't see anything using the memory, where are you seeing that something is using it? What I see is a slapd process using about 150M, then around a hundred imap processes 5-10M each. If the server is restarted, 70-80% will be free, now, after three months we're at 11% free loosing about 20% per month. The exact sum VSZ right now as shown by ps is 1073632k but top says Mem: 3111M Active, 311M Inact, 230M Wired, 144M Cache, 112M Buf, 27M Free Clearly something is grabbing memory and not releasing it. Stopping and starting various programs makes very little difference. No SYSV mem at all. Thanks, ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: memory usage displsy
Dan Nelson wrote: In the last episode (Sep 01), Bill Moran said: In response to Per olof Ljungmark p...@intersonic.se: What is a good way to find out how memory is used? Have a 6.4 box where memory is used by something but I fail to see what is using it - tried different switches to ps(1), tried the stat tools but a big chunk of memory does not show at all. A proper tool for analyzing memory usage live, this is a production box? I've always been able to get what I need from top. You can do -o res to sort by resident memory usage, which helps. ps will sort by memory usage when given the -m flag. Also check ipcs -a to see if there are any sysv shared memory segments hanging arnound. If you don't see anything using the memory, where are you seeing that something is using it? ...and here is top output after I stopped Postfix, slapd and Cyrus-IMAP. Still over 3G Active. last pid: 10278; load averages: 0.03, 0.02, 0.00 up 93+02:50:16 01:57:35 45 processes: 1 running, 44 sleeping CPU: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Mem: 3057M Active, 312M Inact, 228M Wired, 144M Cache, 112M Buf, 81M Free Swap: 4096M Total, 80K Used, 4096M Free PID USERNAME THR PRI NICE SIZERES STATETIME WCPU COMMAND 647 root4 200 3372K 1508K kserel 110:35 0.00% apcupsd 831 root3 200 5008K 1920K kserel 58:48 0.00% bacula-fd 480 root1 960 1416K 932K select 25:23 0.00% syslogd 596 bind1 40 6400K 5160K kqread 23:05 0.00% named 709 root1 960 2780K 1484K select 4:26 0.00% ntpd 661 root1 40 3372K 1972K accept 0:53 0.00% saslauthd 660 root1 200 3372K 1972K lockf0:53 0.00% saslauthd 662 root1 200 3372K 1972K lockf0:53 0.00% saslauthd 659 root1 200 3372K 1972K lockf0:53 0.00% saslauthd 657 root1 200 3372K 1972K lockf0:52 0.00% saslauthd 913 root1 80 1412K 900K nanslp 0:22 0.00% cron 91648 peo 1 960 11372K 7572K select 0:04 0.00% sshd 3419 nagios 1 960 1380K 960K select 0:01 0.00% nrpe2 91656 root1 200 3880K 1952K pause0:00 0.00% csh 10243 root1 960 2516K 1604K RUN 0:00 0.00% top 95511 root1 50 4120K 2156K ttyin0:00 0.00% csh 95504 peo 1 960 6296K 2544K select 0:00 0.00% sshd 95502 root1 40 6300K 2540K sbwait 0:00 0.00% sshd 91646 root1 40 6300K 2476K sbwait 0:00 0.00% sshd 10223 root1 40 6300K 2660K sbwait 0:00 0.00% sshd 10232 root1 50 3880K 2044K ttyin0:00 0.00% csh 91650 peo 1 200 3836K 1848K pause0:00 0.00% csh 95506 peo 1 200 3940K 1916K pause0:00 0.00% csh 10227 peo 1 200 3836K 1976K pause0:00 0.00% csh 906 root1 960 3552K 2016K select 0:00 0.00% sshd 10225 peo 1 960 6296K 2664K select 0:00 0.00% sshd 429 root1 960 528K 284K select 0:00 0.00% devd 91654 peo 1 80 1804K 1112K wait 0:00 0.00% su 95510 peo 1 80 1804K 1168K wait 0:00 0.00% su 10231 peo 1 80 1804K 1244K wait 0:00 0.00% su 961 root1 50 1352K 784K ttyin0:00 0.00% getty 962 root1 50 1352K 784K ttyin0:00 0.00% getty 968 root1 50 1352K 784K ttyin0:00 0.00% getty 964 root1 50 1352K 784K ttyin0:00 0.00% getty 966 root1 50 1352K 784K ttyin0:00 0.00% getty 963 root1 50 1352K 784K ttyin0:00 0.00% getty 965 root1 50 1352K 784K ttyin0:00 0.00% getty 967 root1 50 1352K 784K ttyin0:00 0.00% getty 943 root1 1110 1444K 840K select 0:00 0.00% inetd 138 root1 200 1260K 636K pause0:00 0.00% adjkerntz ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: memory usage displsy
In the last episode (Sep 02), Per olof Ljungmark said: Dan Nelson wrote: In the last episode (Sep 01), Bill Moran said: In response to Per olof Ljungmark p...@intersonic.se: What is a good way to find out how memory is used? Have a 6.4 box where memory is used by something but I fail to see what is using it - tried different switches to ps(1), tried the stat tools but a big chunk of memory does not show at all. A proper tool for analyzing memory usage live, this is a production box? I've always been able to get what I need from top. You can do -o res to sort by resident memory usage, which helps. ps will sort by memory usage when given the -m flag. Also check ipcs -a to see if there are any sysv shared memory segments hanging arnound. If you don't see anything using the memory, where are you seeing that something is using it? What I see is a slapd process using about 150M, then around a hundred imap processes 5-10M each. If the server is restarted, 70-80% will be free, now, after three months we're at 11% free loosing about 20% per month. The exact sum VSZ right now as shown by ps is 1073632k but top says Mem: 3111M Active, 311M Inact, 230M Wired, 144M Cache, 112M Buf, 27M Free Clearly something is grabbing memory and not releasing it. Disk cache, most likely. I would expect Free memory as reported by top to drop down to under 100MB a few hours after a system is rebooted. The difference between Active-Inact-Cache-Buf is more an indication of how long ago a particular page has been touched (and how much work it is to map the page back into a processes memory space), and doesn't really say what the block is being used for. If you are not actively swapping, there is no need for panic. Even a couple hundred MB of used swap is fine, as long as you're not constantly having to pull it back into memory. http://www.freebsd.org/doc/en/books/arch-handbook/vm.html has a good rundown of how the VM system works. -- Dan Nelson dnel...@allantgroup.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
