Re: limit on PV entries
On Wed, 1 Feb 2012 10:17:21 +0100 n dhert wrote: > FreeBSD 8.2-RELEASE > >From time to time, I get in /var/log/messages > kernel: Approaching the limit on PV entries, consider increasing > either the vm.pmap.shpgperproc or the vm.pmap.pv_entry_max sysctl. > > this started a few weeks ago, never had that before, don't have it on > other FreeBSD 8.2-RELEASE systems. > > - What does this mean? > - And how to increase either of the two and to what level ? > > $ sysctl vm.pmap.shpgperproc > vm.pmap.shpgperproc: 200 > $ sysctl vm.pmap.pv_entry_max > vm.pmap.pv_entry_max: 3256966 If you want to know the meaning of a particular sysctl, then 'sysctl -d' is your friend: # sysctl -d vm.pmap.shpgperproc vm.pmap.shpgperproc: Page share factor per proc # sysctl -d vm.pmap.pv_entry_maxvm.pmap.pv_entry_max vm.pmap.pv_entry_max: Max number of PV entries (OK, granted, you may wish to know more than is provided by just these terse single-line entries, but it's a start along the clue path, at least) Your mileage may vary, but for me personally, I like to have *some* sense that I'm approaching this sort of problem in an organized, methodical manner, rather than simply pulling numbers out of a hat, so to speak. So, since computers are based on the binary number system, I naturally gravitate towards numbers that are nice, neat multiples of some power of 2, such as 64, 128, 256, 512, 1024, 2048 and so on. There's really no hard-and-fast rule for determining a good setting for many of these numeric-type sysctls. The method I generally use is: First, try to find a similarly named sysctl within that same hierarchy that represents the current working value for the sysctl in question, by lopping off the last element of the sysctl and scanning the results, using 'sysctl -d' again on the most likely candidate(s) to be sure. For instance, since we're dealing here with two members of the vm.pmap family: # sysctl vm.pmap vm.pmap.pat_works: 1 vm.pmap.pg_ps_enabled: 1 vm.pmap.pv_entry_max: 7573622 vm.pmap.shpgperproc: 1024 vm.pmap.pde.demotions: 14959 vm.pmap.pde.mappings: 1055 vm.pmap.pde.p_failures: 255640 vm.pmap.pde.promotions: 23441 vm.pmap.pdpe.demotions: 5 vm.pmap.pv_entry_count: 270500 <--- this looks like a likely candidate vm.pmap.pc_chunk_count: 2590 vm.pmap.pc_chunk_allocs: 1551162 vm.pmap.pc_chunk_frees: 1548572 vm.pmap.pc_chunk_tryfail: 0 vm.pmap.pv_entry_frees: 244822815 vm.pmap.pv_entry_allocs: 245093315 vm.pmap.pv_entry_spare: 164620 vm.pmap.pmap_collect_inactive: 0 vm.pmap.pmap_collect_active: 0 # sysctl -d vm.pmap.pv_entry_count vm.pmap.pv_entry_count: Current number of pv entries Comparing the current working value to the defined maximum value here, it's pretty obvious that the problem doesn't lie with vm.pmap.pv_entry_max, so it's more likely that (were I experiencing a problem) the value of vm.pmap.shpgperproc needs to be increased. How I choose the value also depends on what the number represents. If it's determining the size of a buffer or some other chunk of memory, I bump the value up to the next (or second) nearest multiple of, again, some reasonable power of 2 (64, 128, 256, 512, 1024), since computers "like" such nice, "neat" numbers. In the case of items that set a limit on the number of entries in some sort of array, such as vm.pmap.pv_entry_max, I usually just double the default setting and see how that works, perhaps backing it off a bit later by some "sensible" factor. If the problem persists, just bump it up again by a similar factor until it goes away. You may, in time, devise your own methodology, but the one I just described is "comfortable" for me, and does feel a lot better than just haphazardly plugging in random numbers and hoping for the best. HTH -- Conrad J. Sabatier conr...@cox.net ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: limit on PV entries
Did you explicitly disabled superpages? What is the output of $ sysctl vm.pmap.pg_ps_enabled ? On 02/01/2012 10:17, n dhert wrote: FreeBSD 8.2-RELEASE From time to time, I get in /var/log/messages kernel: Approaching the limit on PV entries, consider increasing either the vm.pmap.shpgperproc or the vm.pmap.pv_entry_max sysctl. this started a few weeks ago, never had that before, don't have it on other FreeBSD 8.2-RELEASE systems. - What does this mean? - And how to increase either of the two and to what level ? $ sysctl vm.pmap.shpgperproc vm.pmap.shpgperproc: 200 $ sysctl vm.pmap.pv_entry_max vm.pmap.pv_entry_max: 3256966 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org" -- No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
limit on PV entries
FreeBSD 8.2-RELEASE >From time to time, I get in /var/log/messages kernel: Approaching the limit on PV entries, consider increasing either the vm.pmap.shpgperproc or the vm.pmap.pv_entry_max sysctl. this started a few weeks ago, never had that before, don't have it on other FreeBSD 8.2-RELEASE systems. - What does this mean? - And how to increase either of the two and to what level ? $ sysctl vm.pmap.shpgperproc vm.pmap.shpgperproc: 200 $ sysctl vm.pmap.pv_entry_max vm.pmap.pv_entry_max: 3256966 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Approaching the limit on PV entries, consider increasing either the vm.pmap.shpgperproc or the vm.pmap.pv_entry_max tunable
Hello, We have an HP Proliant DL380G5 with 4GB of RAM and FreeBSD 7.0 which runs PostgreSQL 8.3 for more than a year now. No problems, except that two days ago I noticed those messages in my kernel logs : "Approaching the limit on PV entries, consider increasing either the vm.pmap.shpgperproc or the vm.pmap.pv_entry_max tunable." Since vm.pmap.shpgperproc and vm.pmap.pv_entry_max require a reboot I want to be sure that I put "good" values (for ex: I read that increasing too much vm.pmap.shpgperproc could result in a panic at boot time or later). I have the following in /boot/loader.conf : kern.ipc.semmni=1024 kern.ipc.semmns=16384 kern.ipc.semmnu=16384 vm.pmap.shpgperproc=225 vm.pmap.pv_entry_max=4134816 and the following in /etc/sysctl.conf : kern.ipc.shmall=262144 kern.ipc.shmmax=1073741824 kern.ipc.semmap=16384 kern.ipc.shm_use_phys=1 Do you think that those values for vm.pmap.pv_entry_max and vm.pmap.shpgperproc are OK ? Is it OK to increase vm.pmap.pv_entry_max as long as vm.kvm_free is not too low ? Thanks, Julien -- Julien Cigar Belgian Biodiversity Platform http://www.biodiversity.be Université Libre de Bruxelles (ULB) Campus de la Plaine CP 257 Bâtiment NO, Bureau 4 N4 115C (Niveau 4) Boulevard du Triomphe, entrée ULB 2 B-1050 Bruxelles Mail: jci...@ulb.ac.be @biobel: http://biobel.biodiversity.be/person/show/471 Tel : 02 650 57 52 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: kernel: Approaching the limit on PV entries...
> Thanks. I'll see what happens. > > In amd64/7.0 is there any chance running out of pv_entrys would show > up as failures in interprocess communication rather than a panic? The > original symptom was that certain web pages (or jailed servers, I'm > not sure) were unreachable, as if the firewall were misconfigured, > until the system was rebooted. I didn't get to look at the system > before it was rebooted, and I find very little in the logs to explain > what was going on. Sometimes pv_entry allocation can fail even below the pv_entry_high_water if there is no more free pages available to allocate for a pv_entry chunk. Operations (such temp mappings, copies) will not occur if the pv_entry_high_water (90% if pv_entry_max) has been reached OR a page is not immediately available for allocation. The first situation is quiet, you could put a sysctl counter. Maybe PV_STAT(pc_chunk_tryfail++) should be moved from get_pv_entry() to the "else" case of the routine pmap_try_insert_pv_entry(). On the other hand, some allocations (permanent mapping) are unconditional and can occur even when pv_entry_max has been exceeded (amd64). The code will also wait for page allocations. --Mark Tinguely. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: kernel: Approaching the limit on PV entries...
On 10/10/08, Mark Tinguely <[EMAIL PROTECTED]> wrote: >> > vm.pmap.pv_entry_count: 583006 >> > vm.pmap.shpgperproc: 200 >> > vm.pmap.pv_entry_max: 2243305 >> > >> > The system: >> > FreeBSD 7.0-RELEASE-p5 FreeBSD 7.0-RELEASE-p5 #0: Wed Oct 1 >> > 07:51:58 UTC 2008 >> > [EMAIL PROTECTED]:/usr/obj/usr/src/sys/GENERIC amd64 >> > >> > Can someone briefly explain what this is telling me and how to decide >> > which sysctl to increase? I have found some old postings that predate >> > the sysctls that suggested increasing shpgperproc in the kernel >> > configuration, about 50 at a time until the problem goes away, but I >> > still have no clue what that is accomplishing. > > what (simplified): > the pv_entry helps the virtual memory system track physical pages, so a > physical page can be shared with another process or another virtual address. > > In the i386/amd64 the pv_entry entries are allocated in page size "chunks" > on a per process memory map basis. This helps reduce redundant pointers > and overall saves memory. > > "shpgperproc" can be read as "the number of shared pages per proceess". > pv_entry_max is calculated from shpgperproc (and on the amd64, shpgperproc > can be derived from setting the vm.pmap.pv_entry_max). > > On the amd64, the values can be adjusted by sysctl, but on the i386 > the values must be compiled into the kernel. > > There are some automatic adjustments in the calculation of the number > of pv_entry, but the warnings are given early enough to help aid in the > tweaking of the value. The advice of slowly increasing vm.pmap.shpgperproc > is probably the best solution. I would adjust up slower than 50 (25% > increase seems to be pretty high). > > --Mark Tinguely. > Thanks. I'll see what happens. In amd64/7.0 is there any chance running out of pv_entrys would show up as failures in interprocess communication rather than a panic? The original symptom was that certain web pages (or jailed servers, I'm not sure) were unreachable, as if the firewall were misconfigured, until the system was rebooted. I didn't get to look at the system before it was rebooted, and I find very little in the logs to explain what was going on. Thanks again, - Bob [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: kernel: Approaching the limit on PV entries...
> > vm.pmap.pv_entry_count: 583006 > > vm.pmap.shpgperproc: 200 > > vm.pmap.pv_entry_max: 2243305 > > > > The system: > > FreeBSD 7.0-RELEASE-p5 FreeBSD 7.0-RELEASE-p5 #0: Wed Oct 1 > > 07:51:58 UTC 2008 > > [EMAIL PROTECTED]:/usr/obj/usr/src/sys/GENERIC amd64 > > > > Can someone briefly explain what this is telling me and how to decide > > which sysctl to increase? I have found some old postings that predate > > the sysctls that suggested increasing shpgperproc in the kernel > > configuration, about 50 at a time until the problem goes away, but I > > still have no clue what that is accomplishing. what (simplified): the pv_entry helps the virtual memory system track physical pages, so a physical page can be shared with another process or another virtual address. In the i386/amd64 the pv_entry entries are allocated in page size "chunks" on a per process memory map basis. This helps reduce redundant pointers and overall saves memory. "shpgperproc" can be read as "the number of shared pages per proceess". pv_entry_max is calculated from shpgperproc (and on the amd64, shpgperproc can be derived from setting the vm.pmap.pv_entry_max). On the amd64, the values can be adjusted by sysctl, but on the i386 the values must be compiled into the kernel. There are some automatic adjustments in the calculation of the number of pv_entry, but the warnings are given early enough to help aid in the tweaking of the value. The advice of slowly increasing vm.pmap.shpgperproc is probably the best solution. I would adjust up slower than 50 (25% increase seems to be pretty high). --Mark Tinguely. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: kernel: Approaching the limit on PV entries...
Bob Johnson wrote: A web server with several jailed copies of Apache is having problems that seem to be caused by incorrect IPFW rules, but in the process of working on that, I find in the log the following repeated many times: Oct 8 23:29:50 spider kernel: Approaching the limit on PV entries, consider increasing either the vm.pmap.shpgperproc or the vm.pmap.pv_entry_max sysctl. Oct 8 23:30:52 spider kernel: Approaching the limit on PV entries, consider increasing either the vm.pmap.shpgperproc or the vm.pmap.pv_entry_max sysctl. sysctl gives me: # sysctl vm.pmap vm.pmap.pmap_collect_active: 0 vm.pmap.pmap_collect_inactive: 0 vm.pmap.pv_entry_spare: 45818 vm.pmap.pv_entry_allocs: 595716945 vm.pmap.pv_entry_frees: 595133939 vm.pmap.pc_chunk_tryfail: 0 vm.pmap.pc_chunk_frees: 3543052 vm.pmap.pc_chunk_allocs: 3546795 vm.pmap.pc_chunk_count: 3743 vm.pmap.pv_entry_count: 583006 vm.pmap.shpgperproc: 200 vm.pmap.pv_entry_max: 2243305 The system: FreeBSD 7.0-RELEASE-p5 FreeBSD 7.0-RELEASE-p5 #0: Wed Oct 1 07:51:58 UTC 2008 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/GENERIC amd64 Can someone briefly explain what this is telling me and how to decide which sysctl to increase? I have found some old postings that predate the sysctls that suggested increasing shpgperproc in the kernel configuration, about 50 at a time until the problem goes away, but I still have no clue what that is accomplishing. Also, the system has been rebooted since I collected those messages, and they aren't happening any more, but I expect they will reappear eventually. Until then I probably can't actually test anything. Thanks for your time, -- Bob Johnson [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]" For what it's worth, I just came across a machine with the same exact problem, except when I do finally run out of entries, I get a kernel panic. FBSD 7-RELEASE-p4 ~Paul ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
kernel: Approaching the limit on PV entries...
A web server with several jailed copies of Apache is having problems that seem to be caused by incorrect IPFW rules, but in the process of working on that, I find in the log the following repeated many times: Oct 8 23:29:50 spider kernel: Approaching the limit on PV entries, consider increasing either the vm.pmap.shpgperproc or the vm.pmap.pv_entry_max sysctl. Oct 8 23:30:52 spider kernel: Approaching the limit on PV entries, consider increasing either the vm.pmap.shpgperproc or the vm.pmap.pv_entry_max sysctl. sysctl gives me: # sysctl vm.pmap vm.pmap.pmap_collect_active: 0 vm.pmap.pmap_collect_inactive: 0 vm.pmap.pv_entry_spare: 45818 vm.pmap.pv_entry_allocs: 595716945 vm.pmap.pv_entry_frees: 595133939 vm.pmap.pc_chunk_tryfail: 0 vm.pmap.pc_chunk_frees: 3543052 vm.pmap.pc_chunk_allocs: 3546795 vm.pmap.pc_chunk_count: 3743 vm.pmap.pv_entry_count: 583006 vm.pmap.shpgperproc: 200 vm.pmap.pv_entry_max: 2243305 The system: FreeBSD 7.0-RELEASE-p5 FreeBSD 7.0-RELEASE-p5 #0: Wed Oct 1 07:51:58 UTC 2008 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/GENERIC amd64 Can someone briefly explain what this is telling me and how to decide which sysctl to increase? I have found some old postings that predate the sysctls that suggested increasing shpgperproc in the kernel configuration, about 50 at a time until the problem goes away, but I still have no clue what that is accomplishing. Also, the system has been rebooted since I collected those messages, and they aren't happening any more, but I expect they will reappear eventually. Until then I probably can't actually test anything. Thanks for your time, -- Bob Johnson [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"
