Re: FreeBSD vice OS X memory management
Wojciech Puchar wojtek.tensor.gdynia.pl> writes: > > > does OS X kernel share any code with FreeBSD kernel's memory management > > subsystem ? > > IMHO no. OSX is somehow-microkernel based, they did take things from > FreeBSD but not this IMHO. > > anyway - who cares > Well, I quoted the source in my 2nd post in this thread. But I will repeat it once again: "I'm quite sure that the memory manager of OSX wasn't derived from BSD, but from Mach. Actually, FreeBSD has adapted that memory manager, so it's rather the other way around" If so, both OS X and FreeBSD share the same MM subsys base. jb ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: FreeBSD vice OS X memory management
Wojciech Puchar wojtek.tensor.gdynia.pl> writes: > > > > > "2) Inactive memory (which is memory that has been recently used but is no > > longer) is supposed to be seamlessly reclaimed automatically by the OS when > > needed for new programs. In practice, I?ve found that this isn?t the case, > > and > > my system slows to a crawl and starts paging out to disk when free memory > > drops > > to zero, even as half of the available RAM (which is a lot) is marked as > > inactive. ..." > > > > Well, this is not a case of a "BSD is dying" troll (you can safely ignore > > those). > > > yes it is, just search a bit to know what "inactive" memory in FreeBSD is. His description (the quoted text) is at least of intuitive nature, and in fact in may be correct as it referrs to OS X MM subsys, which may be based on least-recently used pageout algorithm (as FreeBSD originally used to be too). jb ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: FreeBSD vice OS X memory management
is relatively new. My guess is that if there is a problem it's ZFS specific. If it were a more general problem I think we'd see a lot more complaints, whereas ZFS already has a reputation for needing lots of memory. you may precisely set up a limits of memory that ZFS would use at most. or just don't use it which i do. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: FreeBSD vice OS X memory management
If you really are having a problem with FreeBSD you are going to have to do a lot better than this in terms of providing some data points which define the problem. I am in agreement with Adam here: either you can work the problem or you can troll. I don't see any indication yet of any real problem analysis, only a wild mix of stuff non-related to FreeBSD sprinkled with some magic 'memory management' dust. The fact that FreeBSD DOES NOT page excessively on the same workload relative to other OS (linux, netbsd) is one of most important thing i decided to use it. If his system is heavily paging then simply he have too large working set. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: FreeBSD vice OS X memory management
"2) Inactive memory (which is memory that has been recently used but is no longer) is supposed to be seamlessly reclaimed automatically by the OS when needed for new programs. In practice, I?ve found that this isn?t the case, and my system slows to a crawl and starts paging out to disk when free memory drops to zero, even as half of the available RAM (which is a lot) is marked as inactive. ..." Well, this is not a case of a "BSD is dying" troll (you can safely ignore those). yes it is, just search a bit to know what "inactive" memory in FreeBSD is. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: FreeBSD vice OS X memory management
most importantly networking but certainly not memory subsystem. On Wed, 25 Apr 2012, Chuck Swiger wrote: On Apr 25, 2012, at 5:31 AM, jb wrote: does OS X kernel share any code with FreeBSD kernel's memory management subsystem ? The simple answer is no. A more complex answer: % grep -ri freebsd xnu-1699.24.23 | wc -l 520 % grep -ril freebsd xnu-1699.24.23 | sort | uniq ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: FreeBSD vice OS X memory management
does OS X kernel share any code with FreeBSD kernel's memory management subsystem ? IMHO no. OSX is somehow-microkernel based, they did take things from FreeBSD but not this IMHO. anyway - who cares Something is deeply broken in OS X memory management http://workstuff.tumblr.com/post/20464780085/something-is-deeply-broken-in-os-x- memory-management One of the problems that caught my eyes was inactive memory reclamation. I remember some time ago there was a thread here with similar topic. http://lists.freebsd.org/pipermail/freebsd-questions/2012-March/239121.html jb ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org" ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: FreeBSD vice OS X memory management
RW googlemail.com> writes: > ... > > ... > > "2) Inactive memory (which is memory that has been recently used but > > is no longer) is supposed to be seamlessly reclaimed automatically by > > the OS when needed for new programs. In practice, I’ve found that > > this isn’t the case, and my system slows to a crawl and starts paging > > out to disk when free memory drops to zero, even as half of the > > available RAM (which is a lot) is marked as inactive. ..." > > That's not a good description of inactive memory, most of which > contains useful data. The situation described is undesirable, but not > abnormal. It can happen when your physical memory is spread thinly, but > most of it isn't being frequently accessed. In that case the inactive > queue can be dominated by dirty swap-backed pages. > ... Would implementing the VM pageout algorithm in such a way that it would mix in equal proportion the current least-actively used algo and the old least-recently used algo help the situation ? jb ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: FreeBSD vice OS X memory management
On Thu, 26 Apr 2012 08:32:39 + (UTC) jb wrote: > Adam Vande More gmail.com> writes: > > > ... > > http://workstuff.tumblr.com/post/19036310553/two-things-that-really-helped- > > speed-up-my-mac-and > > http://dywypi.org/2012/02/back-on-linux.html > > > > "2) Inactive memory (which is memory that has been recently used but > is no longer) is supposed to be seamlessly reclaimed automatically by > the OS when needed for new programs. In practice, I’ve found that > this isn’t the case, and my system slows to a crawl and starts paging > out to disk when free memory drops to zero, even as half of the > available RAM (which is a lot) is marked as inactive. ..." That's not a good description of inactive memory, most of which contains useful data. The situation described is undesirable, but not abnormal. It can happen when your physical memory is spread thinly, but most of it isn't being frequently accessed. In that case the inactive queue can be dominated by dirty swap-backed pages. > The above and the past FreeBSD thread here, both I referred to, have > something in common - the system seems to progressively come under > stress due to what one user experienced as "missing memory", The FreeBSD link involved ZFS which manages its own disk caching and is relatively new. My guess is that if there is a problem it's ZFS specific. If it were a more general problem I think we'd see a lot more complaints, whereas ZFS already has a reputation for needing lots of memory. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: FreeBSD vice OS X memory management
Adam Vande More wrote: > On Thu, Apr 26, 2012 at 12:04 AM, jb wrote: > >> If so, should FreeBSD adopt NetBSD's MM subsys, or just improve itself >> surgically ? >> > > You ought first establish there is a problem. What you have cited is > recently reinvigorated trend that has taken on the air of the "BDS is > dying" troll. What you have is a set of computer users with no > understanding of kernel internals attempting to diagnose some sort of > possibly legitimate problem by reaching conclusion via rumor and > guesswork. These people can be taken about as seriously as those who > insist the moon landing was fake and other bizarre ignorant > pseudo-science. > > http://workstuff.tumblr.com/post/19036310553/two-things-that-really-helped- speed-up-my-mac-and > http://dywypi.org/2012/02/back-on-linux.html > > When you have a test case illustrating your feared FreeBSD VM > shortcomings, you may at that point begin to attract developer interest. > To the OP: A potential first test case where the symptom is "my system slows to a crawl and starts paging out to disk" might be to build a kernel with the SCHED_4BSD scheduler. There have been a couple of edge/corner cases that sound like this. That is, if you really have a problem and want to try eliminating one possibility. Another thing that shows up in things like top is it breaks and does not report accurate values for anything when userland and kernel are out of sync, that is if it runs at all without segfaulting. World and kernel being out of sync would be operator error. In this case the values you are using to somehow relate the symptom to memory management would be false. As far as all the rest, such as something being "deeply broken in OS X memory management", mentions of NetBSD memory management, etc, are all irrelevant. It is this wild mix of stuff seemingly non-related to any problem in FreeBSD per se, that makes this look like a troll. If you really are having a problem with FreeBSD you are going to have to do a lot better than this in terms of providing some data points which define the problem. I am in agreement with Adam here: either you can work the problem or you can troll. I don't see any indication yet of any real problem analysis, only a wild mix of stuff non-related to FreeBSD sprinkled with some magic 'memory management' dust. Sorry if this comes across the wrong way, but this really looks like troll material to me too - it has a great resemblance to a pattern trolls have used for many years. -Mike ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: FreeBSD vice OS X memory management
Adam Vande More gmail.com> writes: > ... > http://workstuff.tumblr.com/post/19036310553/two-things-that-really-helped- > speed-up-my-mac-and > http://dywypi.org/2012/02/back-on-linux.html > "2) Inactive memory (which is memory that has been recently used but is no longer) is supposed to be seamlessly reclaimed automatically by the OS when needed for new programs. In practice, I’ve found that this isn’t the case, and my system slows to a crawl and starts paging out to disk when free memory drops to zero, even as half of the available RAM (which is a lot) is marked as inactive. ..." Well, this is not a case of a "BSD is dying" troll (you can safely ignore those). The above and the past FreeBSD thread here, both I referred to, have something in common - the system seems to progressively come under stress due to what one user experienced as "missing memory", and other two users experienced (as shown here above) as inefficient (or lack of) early reclamation of inactive pages. We just want the devs and users make aware of things. jb ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: FreeBSD vice OS X memory management
On Thu, Apr 26, 2012 at 12:04 AM, jb wrote: > If so, should FreeBSD adopt NetBSD's MM subsys, or just improve itself > surgically ? > You ought first establish there is a problem. What you have cited is recently reinvigorated trend that has taken on the air of the "BDS is dying" troll. What you have is a set of computer users with no understanding of kernel internals attempting to diagnose some sort of possibly legitimate problem by reaching conclusion via rumor and guesswork. These people can be taken about as seriously as those who insist the moon landing was fake and other bizarre ignorant pseudo-science. http://workstuff.tumblr.com/post/19036310553/two-things-that-really-helped-speed-up-my-mac-and http://dywypi.org/2012/02/back-on-linux.html When you have a test case illustrating your feared FreeBSD VM shortcomings, you may at that point begin to attract developer interest. -- Adam Vande More ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: FreeBSD vice OS X memory management
jb gmail.com> writes: > ... > "The related implementation in FreeBSD seems to have a similar problem: > > NetBSD users have also reported that UVM’s im- provements have had a positive > effect on their applica- tions. This is most noticeable when physical memory > becomes scarce and the VM system must page out data to free up memory. > Under BSD VM this type of paging causes the system to become highly > unresponsive, while > under UVM the system slows while paging but does not become unresponsive. > http://static.usenix.org/event/usenix99/full_papers/cranor/cranor.p... > > Should be easy to fix: just start to page out some stuff in time before > there is no memory left." > ... Would this mean that FreeBSD's (and Mach's ?) MM subsys are behind NetBSD's ? If so, should FreeBSD adopt NetBSD's MM subsys, or just improve itself surgically ? jb ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: FreeBSD vice OS X memory management
Chuck Swiger mac.com> writes: > > On Apr 25, 2012, at 5:31 AM, jb wrote: > > does OS X kernel share any code with FreeBSD kernel's memory management subsystem ? > > The simple answer is no. A more complex answer: > > % grep -ri freebsd xnu-1699.24.23 | wc -l > 520 > > % grep -ril freebsd xnu-1699.24.23 | sort | uniq > > > > % grep -ril freebsd xnu-1699.24.23 | sort | uniq > ~/Downloads > xnu-1699.24.23/EXTERNAL_HEADERS/stdbool.h > xnu-1699.24.23/bsd/bsm/audit.h > ... Right, MM subsys is not part of BSD import. But XNU kernel is a combo of Mach and old BSD kernel parts. There was some discussion here: http://www.osnews.com/comments/25861 where two comments are of interest: "I'm quite sure that the memory manager of OSX wasn't derived from BSD, but from Mach. Actually, FreeBSD has adapted that memory manager, so it's rather the other way around. But Apple might learn from the way FreeBSD does things. If it is feasible, as the kernel is quite different." "The related implementation in FreeBSD seems to have a similar problem: NetBSD users have also reported that UVM’s im- provements have had a positive effect on their applica- tions. This is most noticeable when physical memory becomes scarce and the VM system must page out data to free up memory. Under BSD VM this type of paging causes the system to become highly unresponsive, while under UVM the system slows while paging but does not become unresponsive. http://static.usenix.org/event/usenix99/full_papers/cranor/cranor.p... Should be easy to fix: just start to page out some stuff in time before there is no memory left." When I browsed the USENIX paper (dated 1999) I understood that it is indeed possible that FreeBSD may have imported some Mach's MM code in those early BSD VM days. And over time since then some ideas (if not exact code) may have migrated between OS X and FreeBSD MM subsystems. If so, the problems experienced may be similar or identical even today. jb ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: FreeBSD vice OS X memory management
On Apr 25, 2012, at 5:31 AM, jb wrote: > does OS X kernel share any code with FreeBSD kernel's memory management > subsystem ? The simple answer is no. A more complex answer: % grep -ri freebsd xnu-1699.24.23 | wc -l 520 % grep -ril freebsd xnu-1699.24.23 | sort | uniq % grep -ril freebsd xnu-1699.24.23 | sort | uniq ~/Downloads xnu-1699.24.23/EXTERNAL_HEADERS/stdbool.h xnu-1699.24.23/bsd/bsm/audit.h xnu-1699.24.23/bsd/bsm/audit_domain.h xnu-1699.24.23/bsd/bsm/audit_errno.h xnu-1699.24.23/bsd/bsm/audit_fcntl.h xnu-1699.24.23/bsd/bsm/audit_kevents.h xnu-1699.24.23/bsd/crypto/aes/gen/aesopt.h xnu-1699.24.23/bsd/crypto/blowfish/bf_enc.c xnu-1699.24.23/bsd/crypto/blowfish/bf_locl.h xnu-1699.24.23/bsd/crypto/blowfish/bf_pi.h xnu-1699.24.23/bsd/crypto/blowfish/bf_skey.c xnu-1699.24.23/bsd/crypto/blowfish/blowfish.h xnu-1699.24.23/bsd/crypto/cast128/cast128.c xnu-1699.24.23/bsd/crypto/cast128/cast128.h xnu-1699.24.23/bsd/crypto/cast128/cast128_subkey.h xnu-1699.24.23/bsd/crypto/des/des.h xnu-1699.24.23/bsd/crypto/des/des_ecb.c xnu-1699.24.23/bsd/crypto/des/des_enc.c xnu-1699.24.23/bsd/crypto/des/des_locl.h xnu-1699.24.23/bsd/crypto/des/des_setkey.c xnu-1699.24.23/bsd/crypto/des/podd.h xnu-1699.24.23/bsd/crypto/des/sk.h xnu-1699.24.23/bsd/crypto/des/spr.h xnu-1699.24.23/bsd/crypto/rc4/rc4.c xnu-1699.24.23/bsd/crypto/rc4/rc4.h xnu-1699.24.23/bsd/crypto/sha2/sha2.c xnu-1699.24.23/bsd/crypto/sha2/sha2.h xnu-1699.24.23/bsd/dev/dtrace/blist.c xnu-1699.24.23/bsd/dev/dtrace/blist.h xnu-1699.24.23/bsd/dev/memdev.c xnu-1699.24.23/bsd/dev/vn/vn.c xnu-1699.24.23/bsd/hfs/hfs_lookup.c xnu-1699.24.23/bsd/hfs/hfscommon/headers/RedBlackTree.h xnu-1699.24.23/bsd/kern/kern_event.c xnu-1699.24.23/bsd/kern/kern_mib.c xnu-1699.24.23/bsd/kern/kern_newsysctl.c xnu-1699.24.23/bsd/kern/kern_resource.c xnu-1699.24.23/bsd/kern/makesyscalls.sh xnu-1699.24.23/bsd/kern/sys_pipe.c xnu-1699.24.23/bsd/kern/syscalls.master xnu-1699.24.23/bsd/kern/tty.c xnu-1699.24.23/bsd/kern/uipc_socket.c xnu-1699.24.23/bsd/kern/uipc_socket2.c xnu-1699.24.23/bsd/libkern/strsep.c xnu-1699.24.23/bsd/man/man2/aio_cancel.2 xnu-1699.24.23/bsd/man/man2/aio_error.2 xnu-1699.24.23/bsd/man/man2/aio_read.2 xnu-1699.24.23/bsd/man/man2/aio_return.2 xnu-1699.24.23/bsd/man/man2/aio_suspend.2 xnu-1699.24.23/bsd/man/man2/aio_write.2 xnu-1699.24.23/bsd/man/man2/audit.2 xnu-1699.24.23/bsd/man/man2/auditctl.2 xnu-1699.24.23/bsd/man/man2/auditon.2 xnu-1699.24.23/bsd/man/man2/getaudit.2 xnu-1699.24.23/bsd/man/man2/getauid.2 xnu-1699.24.23/bsd/man/man2/getdtablesize.2 xnu-1699.24.23/bsd/man/man2/getlcid.2 xnu-1699.24.23/bsd/man/man2/getpgrp.2 xnu-1699.24.23/bsd/man/man2/getsid.2 xnu-1699.24.23/bsd/man/man2/i386_get_ldt.2 xnu-1699.24.23/bsd/man/man2/issetugid.2 xnu-1699.24.23/bsd/man/man2/kqueue.2 xnu-1699.24.23/bsd/man/man2/mmap.2 xnu-1699.24.23/bsd/man/man2/mprotect.2 xnu-1699.24.23/bsd/man/man2/msync.2 xnu-1699.24.23/bsd/man/man2/read.2 xnu-1699.24.23/bsd/man/man2/semctl.2 xnu-1699.24.23/bsd/man/man2/semget.2 xnu-1699.24.23/bsd/man/man2/semop.2 xnu-1699.24.23/bsd/man/man2/sendfile.2 xnu-1699.24.23/bsd/man/man2/setaudit.2 xnu-1699.24.23/bsd/man/man2/setauid.2 xnu-1699.24.23/bsd/man/man2/setlcid.2 xnu-1699.24.23/bsd/man/man2/setregid.2 xnu-1699.24.23/bsd/man/man2/setreuid.2 xnu-1699.24.23/bsd/man/man2/sigaction.2 xnu-1699.24.23/bsd/man/man2/undelete.2 xnu-1699.24.23/bsd/man/man2/utimes.2 xnu-1699.24.23/bsd/man/man2/write.2 xnu-1699.24.23/bsd/man/man3/queue.3 xnu-1699.24.23/bsd/man/man4/aio.4 xnu-1699.24.23/bsd/man/man4/audit.4 xnu-1699.24.23/bsd/man/man4/auditpipe.4 xnu-1699.24.23/bsd/man/man4/bpf.4 xnu-1699.24.23/bsd/man/man4/divert.4 xnu-1699.24.23/bsd/man/man4/dummynet.4 xnu-1699.24.23/bsd/man/man4/faith.4 xnu-1699.24.23/bsd/man/man4/gif.4 xnu-1699.24.23/bsd/man/man4/ifmib.4 xnu-1699.24.23/bsd/man/man4/inet6.4 xnu-1699.24.23/bsd/man/man4/ipfirewall.4 xnu-1699.24.23/bsd/man/man4/ipsec.4 xnu-1699.24.23/bsd/man/man4/stf.4 xnu-1699.24.23/bsd/man/man4/tty.4 xnu-1699.24.23/bsd/man/man9/copy.9 xnu-1699.24.23/bsd/man/man9/fetch.9 xnu-1699.24.23/bsd/man/man9/intro.9 xnu-1699.24.23/bsd/man/man9/store.9 xnu-1699.24.23/bsd/man/man9/style.9 xnu-1699.24.23/bsd/miscfs/devfs/README xnu-1699.24.23/bsd/miscfs/devfs/devfs.h xnu-1699.24.23/bsd/miscfs/devfs/devfs_tree.c xnu-1699.24.23/bsd/miscfs/devfs/devfs_vfsops.c xnu-1699.24.23/bsd/miscfs/devfs/devfs_vnops.c xnu-1699.24.23/bsd/miscfs/devfs/devfsdefs.h xnu-1699.24.23/bsd/net/bpf.c xnu-1699.24.23/bsd/net/bpf.h xnu-1699.24.23/bsd/net/bpf_compat.h xnu-1699.24.23/bsd/net/bpf_filter.c xnu-1699.24.23/bsd/net/bpfdesc.h xnu-1699.24.23/bsd/net/bridgestp.c xnu-1699.24.23/bsd/net/bridgestp.h xnu-1699.24.23/bsd/net/if.c xnu-1699.24.23/bsd/net/if.h xnu-1699.24.23/bsd/net/if_arp.h xnu-1699.24.23/bsd/net/if_bridge.c xnu-1699.24.23/bsd/net/if_bridgevar.h xnu-1699.24.23/bsd/net/if_dl.h xnu-1699.24.23/bsd/net/if_gif.c xnu-1699.24.23/bsd/net/if_loop.c xnu-1699.24
FreeBSD vice OS X memory management
Hi, does OS X kernel share any code with FreeBSD kernel's memory management subsystem ? Something is deeply broken in OS X memory management http://workstuff.tumblr.com/post/20464780085/something-is-deeply-broken-in-os-x- memory-management One of the problems that caught my eyes was inactive memory reclamation. I remember some time ago there was a thread here with similar topic. http://lists.freebsd.org/pipermail/freebsd-questions/2012-March/239121.html jb ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: virtual memory management
[LoN]Kamikaze wrote: >> Don't forget that the system also pages to swap space and it takes the >> attitude of parking as much as possible out there in case it comes in >> to demand again. Ten if it really needs the space for something, it >> invalidates the oldest stuff and uses that space. >> >> So, you should really expect that your swap space should be >> nearly maxed all the time if things are working well. > > If this is the case something is really wrong on my system: > > Swap: 4096M Total, 4096M Free > > either top doesn't show the precautionary swapping or this is not happening. "Precautionary" swapping does exist, but it's not that often :) What the poster probably meant is that it's possible to have a perfectly working system which looks like it's using a lot of swap if the swapin/swapout rate is low enough. signature.asc Description: OpenPGP digital signature
Re: virtual memory management
> Don't forget that the system also pages to swap space and it takes the > attitude of parking as much as possible out there in case it comes in > to demand again. Ten if it really needs the space for something, it > invalidates the oldest stuff and uses that space. > > So, you should really expect that your swap space should be > nearly maxed all the time if things are working well. If this is the case something is really wrong on my system: Swap: 4096M Total, 4096M Free either top doesn't show the precautionary swapping or this is not happening. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: virtual memory management
Dear all, | Also remember that swap usage itself is not a bad thing; it just means Problem solved. I should have thought about that earlier. Yesterday I was playing with HotSaNIC software to use it on this box. In the end I decided I didn't like it and I didn't really need it so I removed it from the system stopping rrdtool first. Only today did I notice that I have an unusually high (for my setup) number of sleeping processes. ps ax showed me that I had about 150 perl processes that were started by HotSaNIC but never really stopped. I killed them (what a joy :) and voila: last pid: 62059; load averages: 0.04, 0.12, 0.22 up 51+00:27:42 23:21:25 81 processes: 1 running, 78 sleeping, 2 zombie CPU states: 2.7% user, 0.0% nice, 0.8% system, 0.8% interrupt, 95.7% idle Mem: 161M Active, 4348K Inact, 111M Wired, 128K Cache, 60M Buf, 216M Free Swap: 512M Total, 81M Used, 431M Free, 15% Inuse Thank you all for your support! My experience with FBSD is pretty much 51 days old :). But I read most of the posts hoping to learn new things. The great adventure now is to upgrade to 6.2 from 6.1. Never been there before :) Warm regards, -- Zbigniew Szalbot ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: virtual memory management
On Sat, Jan 20, 2007 at 08:57:27AM +0100, Zbigniew Szalbot wrote: > Dear all, > > Is there a FBSD command to manage virtual memory? I think my swap size is > now a bit too much used: > > last pid: 19824; load averages: 0.06, 0.05, 0.02 up 50+10:00:17 > 08:54:00 > 230 processes: 1 running, 227 sleeping, 2 zombie > CPU states: 0.0% user, 0.0% nice, 0.4% system, 0.8% interrupt, 98.8% idle > Mem: 232M Active, 27M Inact, 91M Wired, 212K Cache, 60M Buf, 142M Free > Swap: 512M Total, 482M Used, 29M Free, 94% Inuse > > The swap size usage grow so big probably because I started wget to > download an iso image and then WinSCP to grab it from the FBSD machine to > my laptop. When I started wget, the swap usage was around 19% and had been > like that for many days. > > Is there any way to handle swap size usage other than restarting the box? Don't forget that the system also pages to swap space and it takes the attitude of parking as much as possible out there in case it comes in to demand again. Ten if it really needs the space for something, it invalidates the oldest stuff and uses that space. So, you should really expect that your swap space should be nearly maxed all the time if things are working well. Someone else can give you more accurate detailed information if you want it. jerry > > Thank you very much in advance! > > -- > Zbigniew Szalbot > > ___ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "[EMAIL PROTECTED]" > ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: virtual memory management
In the last episode (Jan 20), Zbigniew Szalbot said: > >> I see lots of them; every one in that list is contributinig. If > >> you add up all those process sizes you'll see where the space is > >> going. > > > > By which I mean the difference between size and res, which indicates > > the amount of process memory allocated but not currently resident in > > RAM. This isn't a foolproof method (see e.g. the FAQ entry on > > rpc.statd), but it's true in your case. > > > >> Basically you are just overloading your system by trying to run > >> too much at once. Reduce the load or add more RAM. > > The problem is I cannot add more RAM (too old machine to do that) but > I know what to do to decrease the load a bit. So thanks for the > pointer! I appreciate it! Also remember that swap usage itself is not a bad thing; it just means the system has moved some unused process data to disk. What /is/ bad is when the system is so low on RAM that is it constantly shuffling data to and from swap just to keep running. This is called thrashing, and you can track it by watching the "##Kb In, ##Kb Out" values on the Swap: line in top, and the "pi" and "po" columns in vmstat output. As long as you don't see constant swapping activity, you're okay. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: virtual memory management
Zbigniew Szalbot wrote: > hello, > >>> The problem is I cannot add more RAM (too old machine to do that) but I >>> know what to do to decrease the load a bit. So thanks for the pointer! I >>> appreciate it! >> You might also want to stop using mod_php in apache and convert to >> fastcgi setup - this way you'll get all Apache processes to use a much >> more reasonable amount, like ~~5MB - 8MB and a small number of php-cgi >> processes that use ~~20MB or more, saving you memory in the end. > > Does this mean recompiling Apache? Or is it a question of httpd.conf? From > what I understand it probably involves recompilation? At most it would require recompilation of PHP (the main port, not the extensions) and installing mod_fcgid. If you enabled CGI and FastCGI during PHP build, you only need mod_fcgid. See http://fastcgi.coremail.cn/configuration.htm#PHP for documentation. (don't forget to remove mod_php from httpd.conf). Note that using FastCGI is different in some important aspects from mod_php, so read up on it if you haven't used it before. signature.asc Description: OpenPGP digital signature
Re: virtual memory management
On Saturday 20 January 2007 08:57, Zbigniew Szalbot wrote: > Is there any way to handle swap size usage other than restarting the box? Yes, you can add swap while the system is running with swapon(8). If you don't have an empty partition available you could create one with mdconfig(8). -Pieter ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: virtual memory management
hello, >> The problem is I cannot add more RAM (too old machine to do that) but I >> know what to do to decrease the load a bit. So thanks for the pointer! I >> appreciate it! > > You might also want to stop using mod_php in apache and convert to > fastcgi setup - this way you'll get all Apache processes to use a much > more reasonable amount, like ~~5MB - 8MB and a small number of php-cgi > processes that use ~~20MB or more, saving you memory in the end. Does this mean recompiling Apache? Or is it a question of httpd.conf? From what I understand it probably involves recompilation? Thank you! -- Zbigniew Szalbot ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: virtual memory management
Zbigniew Szalbot wrote: > The problem is I cannot add more RAM (too old machine to do that) but I > know what to do to decrease the load a bit. So thanks for the pointer! I > appreciate it! You might also want to stop using mod_php in apache and convert to fastcgi setup - this way you'll get all Apache processes to use a much more reasonable amount, like ~~5MB - 8MB and a small number of php-cgi processes that use ~~20MB or more, saving you memory in the end. signature.asc Description: OpenPGP digital signature
Re: virtual memory management
Dear Kris and all, >> I see lots of them; every one in that list is contributinig. If you >> add up all those process sizes you'll see where the space is going. > > By which I mean the difference between size and res, which indicates > the amount of process memory allocated but not currently resident in > RAM. This isn't a foolproof method (see e.g. the FAQ entry on > rpc.statd), but it's true in your case. > >> Basically you are just overloading your system by trying to run too >> much at once. Reduce the load or add more RAM. The problem is I cannot add more RAM (too old machine to do that) but I know what to do to decrease the load a bit. So thanks for the pointer! I appreciate it! Warm regards, -- Zbigniew Szalbot ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: virtual memory management
On Sat, Jan 20, 2007 at 03:51:38AM -0500, Kris Kennaway wrote: > On Sat, Jan 20, 2007 at 09:13:48AM +0100, Zbigniew Szalbot wrote: > > Hello again, > > > > >> The swap size usage grow so big probably because I started wget to > > >> download an iso image and then WinSCP to grab it from the FBSD machine > > >> to my laptop. When I started wget, the swap usage was around 19% and > > >> had been like that for many days. > > > > > > That should not cause such a thing (wget does not try to fit the whole > > > file in memory, so it won't be pushing stuff out to swap). Look at the > > > process sizes in top to see what is using the swap space - something(s) > > > that is still running is using that 482MB. > > > > I do not see such a process: > > > > PID USERNAME THR PRI NICE SIZERES STATETIME WCPU COMMAND > > 21442 root 2 200 224M 26128K kserel 0:06 0.00% java > > 896 mysql 16 200 70756K 14764K kserel 255:35 0.00% mysqld > > 98693 bind 1 960 32812K 32172K select 0:22 0.00% dnscache > > 5035 www 1 40 28372K 15660K accept 5:05 1.86% httpd > > 5026 www 1 40 28240K 15104K accept 4:46 0.00% httpd > > 5065 www 1 40 28128K 15196K accept 4:29 0.00% httpd > > 5030 www 1 40 27892K 15144K accept 4:21 0.00% httpd > > 5126 www 1 40 27784K 14864K accept 4:20 0.00% httpd > > 5029 www 1 40 27760K 14644K accept 4:22 0.00% httpd > > 5027 www 1 40 27740K 15140K accept 4:30 0.00% httpd > > 5028 www 1 40 27516K 14812K accept 4:03 0.00% httpd > > 95977 www 1 40 27216K 14532K accept 2:21 0.00% httpd > > 703 root 1 960 16412K 2296K select 4:35 0.00% httpd > > 91014 mailman 1 80 11492K 1600K nanslp 6:00 0.00% python > > 91012 mailman 1 80 11024K 1560K nanslp 5:32 0.00% python > > 91010 mailman 1 80 11008K 1568K nanslp 5:23 0.00% python > > 91009 mailman 1 80 11008K 1552K nanslp 5:20 0.00% python > > I see lots of them; every one in that list is contributinig. If you > add up all those process sizes you'll see where the space is going. By which I mean the difference between size and res, which indicates the amount of process memory allocated but not currently resident in RAM. This isn't a foolproof method (see e.g. the FAQ entry on rpc.statd), but it's true in your case. > Basically you are just overloading your system by trying to run too > much at once. Reduce the load or add more RAM. > > Kris pgpTxEofIrvtn.pgp Description: PGP signature
Re: virtual memory management
On Sat, Jan 20, 2007 at 09:13:48AM +0100, Zbigniew Szalbot wrote: > Hello again, > > >> The swap size usage grow so big probably because I started wget to > >> download an iso image and then WinSCP to grab it from the FBSD machine > >> to my laptop. When I started wget, the swap usage was around 19% and > >> had been like that for many days. > > > > That should not cause such a thing (wget does not try to fit the whole > > file in memory, so it won't be pushing stuff out to swap). Look at the > > process sizes in top to see what is using the swap space - something(s) > > that is still running is using that 482MB. > > I do not see such a process: > > PID USERNAME THR PRI NICE SIZERES STATETIME WCPU COMMAND > 21442 root 2 200 224M 26128K kserel 0:06 0.00% java > 896 mysql 16 200 70756K 14764K kserel 255:35 0.00% mysqld > 98693 bind 1 960 32812K 32172K select 0:22 0.00% dnscache > 5035 www 1 40 28372K 15660K accept 5:05 1.86% httpd > 5026 www 1 40 28240K 15104K accept 4:46 0.00% httpd > 5065 www 1 40 28128K 15196K accept 4:29 0.00% httpd > 5030 www 1 40 27892K 15144K accept 4:21 0.00% httpd > 5126 www 1 40 27784K 14864K accept 4:20 0.00% httpd > 5029 www 1 40 27760K 14644K accept 4:22 0.00% httpd > 5027 www 1 40 27740K 15140K accept 4:30 0.00% httpd > 5028 www 1 40 27516K 14812K accept 4:03 0.00% httpd > 95977 www 1 40 27216K 14532K accept 2:21 0.00% httpd > 703 root 1 960 16412K 2296K select 4:35 0.00% httpd > 91014 mailman 1 80 11492K 1600K nanslp 6:00 0.00% python > 91012 mailman 1 80 11024K 1560K nanslp 5:32 0.00% python > 91010 mailman 1 80 11008K 1568K nanslp 5:23 0.00% python > 91009 mailman 1 80 11008K 1552K nanslp 5:20 0.00% python I see lots of them; every one in that list is contributinig. If you add up all those process sizes you'll see where the space is going. Basically you are just overloading your system by trying to run too much at once. Reduce the load or add more RAM. Kris pgpA2qhyW02hO.pgp Description: PGP signature
Re: virtual memory management
Hello again, >> The swap size usage grow so big probably because I started wget to >> download an iso image and then WinSCP to grab it from the FBSD machine >> to my laptop. When I started wget, the swap usage was around 19% and >> had been like that for many days. > > That should not cause such a thing (wget does not try to fit the whole > file in memory, so it won't be pushing stuff out to swap). Look at the > process sizes in top to see what is using the swap space - something(s) > that is still running is using that 482MB. I do not see such a process: PID USERNAME THR PRI NICE SIZERES STATETIME WCPU COMMAND 21442 root 2 200 224M 26128K kserel 0:06 0.00% java 896 mysql 16 200 70756K 14764K kserel 255:35 0.00% mysqld 98693 bind 1 960 32812K 32172K select 0:22 0.00% dnscache 5035 www 1 40 28372K 15660K accept 5:05 1.86% httpd 5026 www 1 40 28240K 15104K accept 4:46 0.00% httpd 5065 www 1 40 28128K 15196K accept 4:29 0.00% httpd 5030 www 1 40 27892K 15144K accept 4:21 0.00% httpd 5126 www 1 40 27784K 14864K accept 4:20 0.00% httpd 5029 www 1 40 27760K 14644K accept 4:22 0.00% httpd 5027 www 1 40 27740K 15140K accept 4:30 0.00% httpd 5028 www 1 40 27516K 14812K accept 4:03 0.00% httpd 95977 www 1 40 27216K 14532K accept 2:21 0.00% httpd 703 root 1 960 16412K 2296K select 4:35 0.00% httpd 91014 mailman 1 80 11492K 1600K nanslp 6:00 0.00% python 91012 mailman 1 80 11024K 1560K nanslp 5:32 0.00% python 91010 mailman 1 80 11008K 1568K nanslp 5:23 0.00% python 91009 mailman 1 80 11008K 1552K nanslp 5:20 0.00% python I have just restarted the java program as it used to hold 224M but it did not help (and it is quite stable and never given me any problem). After using Squirrel Mail for some time swap use went down to 71% Mem: 258M Active, 49M Inact, 108M Wired, 16M Cache, 60M Buf, 61M Free Swap: 512M Total, 440M Used, 71M Free, 86% Inuse Thank you very much for your suggestion Kris! -- Zbigniew Szalbot ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: virtual memory management
On Sat, Jan 20, 2007 at 08:57:27AM +0100, Zbigniew Szalbot wrote: > Dear all, > > Is there a FBSD command to manage virtual memory? I think my swap size is > now a bit too much used: > > last pid: 19824; load averages: 0.06, 0.05, 0.02 up 50+10:00:17 > 08:54:00 > 230 processes: 1 running, 227 sleeping, 2 zombie > CPU states: 0.0% user, 0.0% nice, 0.4% system, 0.8% interrupt, 98.8% idle > Mem: 232M Active, 27M Inact, 91M Wired, 212K Cache, 60M Buf, 142M Free > Swap: 512M Total, 482M Used, 29M Free, 94% Inuse > > The swap size usage grow so big probably because I started wget to > download an iso image and then WinSCP to grab it from the FBSD machine to > my laptop. When I started wget, the swap usage was around 19% and had been > like that for many days. That should not cause such a thing (wget does not try to fit the whole file in memory, so it won't be pushing stuff out to swap). Look at the process sizes in top to see what is using the swap space - something(s) that is still running is using that 482MB. Probably you have one or more processes that are using a large amount of virtual memory, which is too big to fit in RAM. That's the situation you need to address. > Is there any way to handle swap size usage other than restarting the box? kill(1). Kris pgpuL9xkWePKe.pgp Description: PGP signature
virtual memory management
Dear all, Is there a FBSD command to manage virtual memory? I think my swap size is now a bit too much used: last pid: 19824; load averages: 0.06, 0.05, 0.02 up 50+10:00:17 08:54:00 230 processes: 1 running, 227 sleeping, 2 zombie CPU states: 0.0% user, 0.0% nice, 0.4% system, 0.8% interrupt, 98.8% idle Mem: 232M Active, 27M Inact, 91M Wired, 212K Cache, 60M Buf, 142M Free Swap: 512M Total, 482M Used, 29M Free, 94% Inuse The swap size usage grow so big probably because I started wget to download an iso image and then WinSCP to grab it from the FBSD machine to my laptop. When I started wget, the swap usage was around 19% and had been like that for many days. Is there any way to handle swap size usage other than restarting the box? Thank you very much in advance! -- Zbigniew Szalbot ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: memory management
On Tue, 4 Apr 2006 10:53:05 +0100 (BST) hossein ghahghaei <[EMAIL PROTECTED]> wrote: > how manage freebsd the memory ? The answer is a bit too complex to provide via a mailing list. I recommend you get a copy of _The_Design_and_Implementation_of_ _FreeBSD_. It has all the details you could ever want. That and a copy of the source code (which comes with FreeBSD) should keep you busy studying for some time. -- Bill Moran Collaborative Fusion Inc. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
memory management
how manage freebsd the memory ? Send instant messages to your online friends http://uk.messenger.yahoo.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
