Re: OpenBSD 6.0-stable: uvm_mapent_alloc: out of static map entries
mxb wrote:
> Hey,
> seeing following in dmesg:
>
> uvm_mapent_alloc: out of static map entries
>
> Wasn’t it fixed so system dynamically adjusted this or do I stil need to
> increase and re-compile kernel ?
uvm amaps got restructured that way for 6.0. What you are seeing is in
a different part of uvm.
>From uvm_mapent_alloc in sys/uvm/uvm_map.c: Looks like the allocation
succeeds, but your system is allocating map entries at a too high rate.
Can't tell why that is though.
if (map->flags & VM_MAP_INTRSAFE || cold) {
mtx_enter(_kmapent_mtx);
if (SLIST_EMPTY(_free)) {
ne = km_alloc(PAGE_SIZE, _page, _dirty,
_nowait);
if (ne == NULL)
panic("uvm_mapent_alloc: cannot allocate map "
"entry");
for (i = 0; i < PAGE_SIZE / sizeof(*ne); i++) {
SLIST_INSERT_HEAD(_free,
[i], daddrs.addr_kentry);
}
if (ratecheck(_kmapent_last_warn_time,
_kmapent_warn_rate))
printf("uvm_mapent_alloc: out of static "
"map entries\n");
}
me = SLIST_FIRST(_free);
SLIST_REMOVE_HEAD(_free, daddrs.addr_kentry);
uvmexp.kmapent++;
mtx_leave(_kmapent_mtx);
me->flags = UVM_MAP_STATIC;
...
> P.S.
> Have plenty of RAM (15G free) on this box.
>
>
> //mxb
OpenBSD 6.0-stable: uvm_mapent_alloc: out of static map entries
Hey, seeing following in dmesg: uvm_mapent_alloc: out of static map entries Wasn’t it fixed so system dynamically adjusted this or do I stil need to increase and re-compile kernel ? P.S. Have plenty of RAM (15G free) on this box. //mxb
uvm_mapent_alloc: out of static map entries
Hi, I've had today this message uvm_mapent_alloc: out of static map entries in my quite current 5.4 GENERIC.MP#193 i386 (Mon Jan 6) When it begun I've also had e-mails from mrtg failing to contact snmpd. I've found an old post and a site http://www.saigonist.com/content/openbsd-uvmmapentalloc-out-static-map-entries telling to play with NKMEMPAGES_MAX, NKMEMPAGES, MAX_KMAPENT Should I bother with these tips or not? Thanks G
Re: uvm_mapent_alloc: out of static map entries
i have been seeing this on the server for some time and i thought this will be corrected on last releases but i was wrong i could not find a solution on archives but (correct me if am wrong) it seems that this is just a message and that the system auto increase this when this occurs, better explanation on archives Date: Mon, 20 Jan 2014 11:37:46 +0200 From: bil...@edu.physics.uoc.gr To: misc@openbsd.org Subject: uvm_mapent_alloc: out of static map entries Hi, I've had today this message uvm_mapent_alloc: out of static map entries in my quite current 5.4 GENERIC.MP#193 i386 (Mon Jan 6) When it begun I've also had e-mails from mrtg failing to contact snmpd. I've found an old post and a site http://www.saigonist.com/content/openbsd-uvmmapentalloc-out-static-map-entrie s telling to play with NKMEMPAGES_MAX, NKMEMPAGES, MAX_KMAPENT Should I bother with these tips or not? Thanks G
uvm_mapent_alloc: out of static map entries
i have read on archives but too many opinions on this subject since 4 and many of them are saying to restart server, restart process, wait to be fixed a big diff but the problem its that the diff its for 4.3 and i have 4.8 and of course i have the problem any new info about this and i havent found the solution to this. P.D. Oh lord listen to my prays i hope the folks on openbsd misc can please enlighten me as i am a dummy and leave me a message at least whatever message will be well something
Re: uvm_mapent_alloc: out of static map entries
Carlos, We are now on OpenBSD 5.3 and going forward. Please try that first. carlos albino garcia grijalba [genesi...@hotmail.com] wrote: i have read on archives but too many opinions on this subject since 4 and many of them are saying to restart server, restart process, wait to be fixed a big diff but the problem its that the diff its for 4.3 and i have 4.8 and of course i have the problem any new info about this and i havent found the solution to this. P.D. Oh lord listen to my prays i hope the folks on openbsd misc can please enlighten me as i am a dummy and leave me a message at least whatever message will be well something -- I'm not being defensive. Maybe you're the one that's being defensive. Maybe you should look at yourself once in awhile.
Re: uvm_mapent_alloc: out of static map entries
carlos albino garcia grijalba [genesi...@hotmail.com] wrote: ok problem of mine again i run again on a fast solution since i have just seen that there have been a lot of changes on uvm lets go 4.8 - 4.9 - 5.0 - 5.1 - 5.2 - 5.3 ant thanks this is actually an aswer will do that and let folks know what happen Just install 5.3. You don't need to upgrade to each version.
Re: uvm_mapent_alloc: out of static map entries
carlos albino garcia grijalba [genesi...@hotmail.com] wrote: it is a server on production m a little concerned about fail after upgrade from 4.8 to 5.3 has some services on it Just upgrade to 5.3, pkg_add -r, and fix the fallout from ports changes. Read the faq/current.html too
Re: uvm_mapent_alloc: out of static map entries
On 28 May 2013 21:39, Chris Cappuccio ch...@nmedia.net wrote: carlos albino garcia grijalba [genesi...@hotmail.com] wrote: it is a server on production m a little concerned about fail after upgrade from 4.8 to 5.3 has some services on it Just upgrade to 5.3, pkg_add -r, and fix the fallout from ports changes. Read the faq/current.html too If he is upgrading to 5.3, he should read faq/upgrade53.html instead :) -- Sincerely, Ville Valkonen
Re: uvm_mapent_alloc: out of static map entries
ok problem of mine again i run again on a fast solution since i have just seen that there have been a lot of changes on uvm lets go 4.8 - 4.9 - 5.0 - 5.1 - 5.2 - 5.3 ant thanks this is actually an aswer will do that and let folks know what happen Date: Tue, 28 May 2013 09:54:00 -0700 From: ch...@nmedia.net To: genesi...@hotmail.com CC: misc@openbsd.org Subject: Re: uvm_mapent_alloc: out of static map entries Carlos, We are now on OpenBSD 5.3 and going forward. Please try that first. carlos albino garcia grijalba [genesi...@hotmail.com] wrote: i have read on archives but too many opinions on this subject since 4 and many of them are saying to restart server, restart process, wait to be fixed a big diff but the problem its that the diff its for 4.3 and i have 4.8 and of course i have the problem any new info about this and i havent found the solution to this. P.D. Oh lord listen to my prays i hope the folks on openbsd misc can please enlighten me as i am a dummy and leave me a message at least whatever message will be well something -- I'm not being defensive. Maybe you're the one that's being defensive. Maybe you should look at yourself once in awhile.
Re: uvm_mapent_alloc: out of static map entries
ok let u know what happen thank u very much actually u are the only folk that answer all my other mails have been kicked by the way where do i have to send mail to know why my laptop has to be rebooted so that the fan work on the first boot i just never work Date: Tue, 28 May 2013 11:39:53 -0700 From: ch...@nmedia.net To: genesi...@hotmail.com CC: misc@openbsd.org Subject: Re: uvm_mapent_alloc: out of static map entries carlos albino garcia grijalba [genesi...@hotmail.com] wrote: it is a server on production m a little concerned about fail after upgrade from 4.8 to 5.3 has some services on it Just upgrade to 5.3, pkg_add -r, and fix the fallout from ports changes. Read the faq/current.html too
Re: uvm_mapent_alloc: out of static map entries
carlos albino garcia grijalba [genesi...@hotmail.com] wrote: ok let u know what happen thank u very much actually u are the only folk that answer all my other mails have been kicked by the way where do i have to send mail to know why my laptop has to be rebooted so that the fan work on the first boot i just never work I don't know about your laptop and its fan, but if you think the problem is specific to OpenBSD, you may want to try a 5.3-current snapshot on it. (see ftp://ftp.openbsd.org/pub/OpenBSD/snapshots/ ) ACPI parsing had a significant fix committed last week, one that affects many different implementations.
Re: uvm_mapent_alloc: out of static map entries
it is a server on production m a little concerned about fail after upgrade from 4.8 to 5.3 has some services on it Date: Tue, 28 May 2013 11:19:18 -0700 From: ch...@nmedia.net To: genesi...@hotmail.com CC: misc@openbsd.org Subject: Re: uvm_mapent_alloc: out of static map entries carlos albino garcia grijalba [genesi...@hotmail.com] wrote: ok problem of mine again i run again on a fast solution since i have just seen that there have been a lot of changes on uvm lets go 4.8 - 4.9 - 5.0 - 5.1 - 5.2 - 5.3 ant thanks this is actually an aswer will do that and let folks know what happen Just install 5.3. You don't need to upgrade to each version.
uvm_mapent_alloc: out of static map entries (on a notebook)
hi there, i have just noticed in /var/log/messages: Jul 26 21:57:59 hatvan /bsd: uvm_mapent_alloc: out of static map entries Jul 26 22:14:26 hatvan /bsd: uvm_mapent_alloc: out of static map entries Jul 26 22:26:38 hatvan /bsd: uvm_mapent_alloc: out of static map entries archives mostly show 4.3 or older posts about servers with high loads and all kinds of (surely heretic) knob fiddling.. this is on my notebook that apart of firefox's inhuman requirements is hardly tormented at all. how would i go about finding out what caused this and why? $ netstat -m 74 mbufs in use: 42 mbufs allocated to data 4 mbufs allocated to packet headers 28 mbufs allocated to socket names and addresses 40/212/6144 mbuf 2048 byte clusters in use (current/peak/max) 0/8/6144 mbuf 4096 byte clusters in use (current/peak/max) 0/8/6144 mbuf 8192 byte clusters in use (current/peak/max) 0/8/6144 mbuf 9216 byte clusters in use (current/peak/max) 0/8/6144 mbuf 12288 byte clusters in use (current/peak/max) 0/8/6144 mbuf 16384 byte clusters in use (current/peak/max) 0/8/6144 mbuf 65536 byte clusters in use (current/peak/max) 700 Kbytes allocated to network (14% in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines OpenBSD 5.0-beta (GENERIC.MP) #22: Thu Jul 21 21:23:21 MDT 2011 dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC.MP cpu0: Intel(R) Core(TM) Duo CPU L2400 @ 1.66GHz (GenuineIntel 686-class) 1.67 GHz cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,VMX,EST,TM2,xTPR,PDCM real mem = 2137387008 (2038MB) avail mem = 2092339200 (1995MB) mainbus0 at root bios0 at mainbus0: AT/286+ BIOS, date 04/18/07, BIOS32 rev. 0 @ 0xfd690, SMBIOS rev. 2.4 @ 0xe0010 (67 entries) bios0: vendor LENOVO version 7BETC9WW (2.10 ) date 04/18/2007 bios0: LENOVO 1705CTO acpi0 at bios0: rev 2 acpi0: sleep states S0 S3 S4 S5 acpi0: tables DSDT FACP SSDT ECDT TCPA APIC MCFG HPET SLIC BOOT SSDT SSDT SSDT SSDT acpi0: wakeup devices LID_(S3) SLPB(S3) DURT(S3) EXP0(S4) EXP1(S4) EXP2(S4) EXP3(S4) PCI1(S4) USB0(S3) USB1(S3) USB2(S3) USB7(S3) HDEF(S4) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpiec0 at acpi0 acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: apic clock running at 166MHz cpu1 at mainbus0: apid 1 (application processor) cpu1: Intel(R) Core(TM) Duo CPU L2400 @ 1.66GHz (GenuineIntel 686-class) 1.67 GHz cpu1: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,VMX,EST,TM2,xTPR,PDCM ioapic0 at mainbus0: apid 1 pa 0xfec0, version 20, 24 pins ioapic0: misconfigured as apic 2, remapped to apid 1 acpimcfg0 at acpi0 addr 0xf000, bus 0-63 acpihpet0 at acpi0: 14318179 Hz acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus -1 (AGP_) acpiprt2 at acpi0: bus 2 (EXP0) acpiprt3 at acpi0: bus 3 (EXP1) acpiprt4 at acpi0: bus 4 (EXP2) acpiprt5 at acpi0: bus 12 (EXP3) acpiprt6 at acpi0: bus 21 (PCI1) acpicpu0 at acpi0: C3, C2, C1, PSS acpicpu1 at acpi0: C3, C2, C1, PSS acpipwrres0 at acpi0: PUBS acpitz0 at acpi0: critical temperature is 127 degC acpitz1 at acpi0: critical temperature is 97 degC acpibtn0 at acpi0: LID_ acpibtn1 at acpi0: SLPB acpibat0 at acpi0: BAT0 model 42T4629 serial 327 type LION oem SANYO acpibat1 at acpi0: BAT1 not present acpibat2 at acpi0: BAT2 not present acpiac0 at acpi0: AC unit online acpithinkpad0 at acpi0 acpidock0 at acpi0: GDCK not docked (0) bios0: ROM list: 0xc/0xea00! 0xcf000/0x1000 0xd/0x1000 0xdc000/0x4000! 0xe/0x1! cpu0: Enhanced SpeedStep 1663 MHz: speeds: 1667, 1333, 1000 MHz pci0 at mainbus0 bus 0: configuration mode 1 (bios) pchb0 at pci0 dev 0 function 0 Intel 82945GM Host rev 0x03 vga1 at pci0 dev 2 function 0 Intel 82945GM Video rev 0x03 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) intagp0 at vga1 agp0 at intagp0: aperture at 0xd000, size 0x1000 inteldrm0 at vga1: apic 1 int 16 drm0 at inteldrm0 Intel 82945GM Video rev 0x03 at pci0 dev 2 function 1 not configured azalia0 at pci0 dev 27 function 0 Intel 82801GB HD Audio rev 0x02: msi azalia0: codecs: Analog Devices AD1981HD, 0x/0x, using Analog Devices AD1981HD audio0 at azalia0 ppb0 at pci0 dev 28 function 0 Intel 82801GB PCIE rev 0x02: apic 1 int 20 pci1 at ppb0 bus 2 em0 at pci1 dev 0 function 0 Intel PRO/1000MT (82573L) rev 0x00: msi, address 00:16:d3:b6:19:57 ppb1 at pci0 dev 28 function 1 Intel 82801GB PCIE rev 0x02: apic 1 int 21 pci2 at ppb1 bus 3 ath0 at pci2 dev 0 function 0 Atheros AR5212 (IBM MiniPCI) rev 0x01: apic 1 int 17 ath0: AR5424 10.3 phy 6.1 rf5424 10.2, WOR2W, address 00:19:7e:4c:f7:1f ppb2 at pci0 dev 28 function 2 Intel 82801GB PCIE rev 0x02: apic 1 int 22 pci3 at ppb2 bus 4 ppb3 at pci0 dev 28 function 3 Intel 82801GB PCIE rev 0x02: apic 1 int 23 pci4
Re: uvm_mapent_alloc: out of static map entries (on a notebook)
On Wed, Jul 27, 2011, frantisek holop wrote: hi there, i have just noticed in /var/log/messages: Jul 26 21:57:59 hatvan /bsd: uvm_mapent_alloc: out of static map entries Jul 26 22:14:26 hatvan /bsd: uvm_mapent_alloc: out of static map entries Jul 26 22:26:38 hatvan /bsd: uvm_mapent_alloc: out of static map entries The only static map used today is kmem_map, mostly used by malloc but also the i386 pmap code. vmstat -m shows the biggest consumer of malloc is probably acpi and drm, which are probably doing a good job of fragmenting memory. The messsage is mostly harmless, just be thankful it's no longer a panic.
bsd: uvm_mapent_alloc: out of static map entries
436K idle pause 0:00 0.00% ksh 12220 user 180 552K 432K idle pause 0:00 0.00% ksh 1004 user 180 528K 428K idle pause 0:00 0.00% ksh 1796 root 20 548K 412K idle netio 0:00 0.00% pflogd 22458 user 180 564K 420K idle pause 0:00 0.00% sh $ cat /etc/sysctl.conf # $OpenBSD: sysctl.conf,v 1.47 2009/06/09 11:52:54 sthen Exp $ # # This file contains a list of sysctl options the user wants set at # boot time. See sysctl(3) and sysctl(8) for more information on # the many available variables. # #net.inet.ip.forwarding=1 # 1=Permit forwarding (routing) of IPv4 packets #net.inet.ip.mforwarding=1 # 1=Permit forwarding (routing) of IPv4 multicast packets #net.inet.ip.multipath=1# 1=Enable IP multipath routing #net.inet.icmp.rediraccept=1# 1=Accept ICMP redirects #net.inet6.icmp6.rediraccept=0 # 0=Don't accept IPv6 ICMP redirects #net.inet6.ip6.forwarding=1 # 1=Permit forwarding (routing) of IPv6 packets #net.inet6.ip6.mforwarding=1# 1=Permit forwarding (routing) of IPv6 multicast packets #net.inet6.ip6.multipath=1 # 1=Enable IPv6 multipath routing #net.inet6.ip6.accept_rtadv=1 # 1=Permit IPv6 autoconf (forwarding must be 0) #net.inet.tcp.rfc1323=0 # 0=Disable TCP RFC1323 extensions (for if tcp is slow) #net.inet.tcp.rfc3390=0 # 0=Disable RFC3390 for TCP window increasing net.inet.esp.enable=0 # 0=Disable the ESP IPsec protocol net.inet.esp.udpencap=0 #net.inet.ah.enable=0 # 0=Disable the AH IPsec protocol #net.inet.esp.udpencap=0# 0=Disable ESP-in-UDP encapsulation #net.inet.ipcomp.enable=1 # 1=Enable the IPCOMP protocol #net.inet.etherip.allow=1 # 1=Enable the Ethernet-over-IP protocol #net.inet.tcp.ecn=1 # 1=Enable the TCP ECN extension #net.inet.carp.preempt=1# 1=Enable carp(4) preemption #net.inet.carp.log=1# 1=Enable logging of carp(4) packets net.inet.tcp.recvspace=65535 ddb.panic=1 # 0=Do not drop into ddb on a kernel panic #ddb.console=1 # 1=Permit entry of ddb from the console #fs.posix.setuid=0 # 0=Traditional BSD chown() semantics #vm.swapencrypt.enable=0# 0=Do not encrypt pages that go to swap #vfs.nfs.iothreads=4# Number of nfsio kernel threads #net.inet.ip.mtudisc=0 # 0=Disable tcp mtu discovery #kern.usercrypto=0 # 0=Disable userland use of /dev/crypto #kern.splassert=2 # 2=Enable with verbose error messages #kern.nosuidcoredump=2 # 2=Put suid coredumps in /var/crash #kern.watchdog.period=32# 0=Enable hardware watchdog(4) timer if available #kern.watchdog.auto=0 # 0=Disable automatic watchdog(4) retriggering kern.bufcachepercent=40 kern.shminfo.shmall=32768 # needed by Chromium machdep.allowaperture=2 # See xf86(4) #machdep.apmhalt=1 # 1=powerdown hack, try if halt -p doesn't work #machdep.kbdreset=1 # permit console CTRL-ALT-DEL to do a nice halt #machdep.lidsuspend=1 # laptop lid closes cause a suspend #machdep.userldt=1 # allow userland programs to play with ldt, # required by some ports #kern.emul.aout=1 # enable running dynamic OpenBSD a.out bins #kern.emul.freebsd=1# enable running FreeBSD binaries kern.emul.linux=1 # enable running Linux binaries #kern.emul.svr4=1 # enable running SVR4 binaries $ $ tail -20 /var/log/messages Oct 12 07:32:35 host savecore: no core dump Oct 12 07:39:00 host ntpd[18149]: 0 out of 3 peers valid Oct 12 07:39:00 host ntpd[18149]: bad peer from pool pool.ntp.org (93.174.192.20) Oct 12 07:39:00 host ntpd[18149]: bad peer from pool pool.ntp.org (213.28.138.38) Oct 12 07:39:00 host ntpd[18149]: bad peer from pool pool.ntp.org (93.174.192.21) Oct 12 08:00:01 host syslogd: restart Oct 12 08:14:34 host vpnc[21328]: truncated in: 1427 - -5 Oct 12 08:14:55 host last message repeated 3 times Oct 12 08:16:32 host last message repeated 8 times Oct 12 08:33:00 host vpnc[21328]: truncated in: 1427 - -5 Oct 12 08:33:45 host last message repeated 5 times Oct 12 08:50:04 host last message repeated 6 times Oct 12 09:05:32 host vpnc[21328]: truncated in: 1427 - -5 Oct 12 09:06:18 host last message repeated 5 times Oct 12 09:24:33 host last message repeated 24 times Oct 12 09:35:23 host last message repeated 4 times Oct 12 09:42:01 host last message repeated 13 times Oct 12 09:56:17 host /bsd: uvm_mapent_alloc: out of static map entries Oct 12 09:58:10 host /bsd: uvm_mapent_alloc: out of static map entries Oct 12 10:00:01 host syslogd: restart $ $ cat /etc/X11/xorg.conf Section ServerLayout Identifier X.org Configured Screen 0 Screen0 0 0 InputDeviceMouse0 CorePointer InputDeviceKeyboard0 CoreKeyboard EndSection Section Files ModulePath
Re: bsd: uvm_mapent_alloc: out of static map entries
On 2010-10-12, Tomas Bodzar tomas.bod...@gmail.com wrote: subject says a lot, but I will of course provide some details. I'm reading trough archives, but I don't have server and I can't see problems with numbers in outputs as in those cases. It's plain workstation and when that happen my X die and I will end in console so I need to startx again. How certain are you that the two events (log entry and X crashing) are connected? I think it's more likely to be something else, out of static map entries is fairly common on i386. Please have a read of Xorg.0.log after a crash and see if anything relevant is mentioned, it will probably be helpful to have xenocara built from source with debug symbols and obtain a backtrace, see 'How to build something with debug information' and 'How to get a core file out of the X server' in /usr/xenocara/README.
Re: uvm_mapent_alloc: out of static map entries
Hello Alastair. I have looked on the web and found various OpenBSD maillist archive references to this but the impression is that this was fixed ages ago. In my understanding of OpenBSD community: stable system should work fine on general system without any changes of kernel parameters and new software that can change something. Also it seems ok to have from time to time uvm_mapent_alloc: out of static map entries. So it can be hard to find connection between that message and hung. Anyway you can look to: http://blogs.helion-prime.com/2010/02/25/tuning-of-postgresql-under-openbsd.h tml (OpenBSD kernel parameters) and increase value somehow to see what happens. And yes, kernel hackers will say that you will randomly push button as it's impossible to find documentation about kernel parameters and they interaction on the web. have a nice day. -- Vasiliy On Sun, Apr 25, 2010 at 7:04 PM, Alastair Johnson att...@googlemail.com wrote: Last week I setup an 4.6 i386 OpenBSD server. The hardware is Dell Poweredge 1850 with 2GB RAM. Its fully updated 4.6 stable. In only a few days it has twice hung with the error on screen: uvm_mapent_alloc: out of static map entries I have looked on the web and found various OpenBSD maillist archive references to this but the impression is that this was fixed ages ago. The server should be a very lightly used mail relay running exim, ssh and nothing much else. I certainly dont want to randomly push buttons: http://kerneltrap.org/mailarchive/openbsd-misc/2008/5/16/1842014 but 2 years ago this seemed under control: http://kerneltrap.org/mailarchive/openbsd-misc/2008/5/16/1841134 But it's not really alarming, unless it continues to print that continuously. Its not doing it continuously - just once and then hang. Below is dmesg output. Please let me know if i can provide any more useful information. Many thanks, Alastair Johnson [r...@relayb..com /etc]# dmesg OpenBSD 4.6-stable (GENERIC) #0: Thu Apr 22 22:41:04 BST 2010 r...@relayb:/usr/src/sys/arch/i386/compile/GENERIC cpu0: Intel(R) Xeon(TM) CPU 3.20GHz (GenuineIntel 686-class) 3.20 GHz cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,CNXT-ID,CX16,xTPR real mem = 2146795520 (2047MB) avail mem = 2067058688 (1971MB) mainbus0 at root bios0 at mainbus0: AT/286+ BIOS, date 01/09/06, BIOS32 rev. 0 @ 0xffe90, SMBIOS rev. 2.3 @ 0xf9920 (87 entries) bios0: vendor Dell Computer Corporation version A05 date 01/09/2006 bios0: Dell Computer Corporation PowerEdge 1850 acpi0 at bios0: rev 0 acpi0: tables DSDT FACP APIC SPCR HPET MCFG acpi0: wakeup devices PCI0(S5) PALO(S5) PBLO(S5) VPR0(S5) PBHI(S5) VPR1(S5) PICH(S5) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: apic clock running at 199MHz cpu at mainbus0: not configured ioapic0 at mainbus0: apid 2 pa 0xfec0, version 20, 24 pins ioapic0: misconfigured as apic 0, remapped to apid 2 ioapic1 at mainbus0: apid 3 pa 0xfec8, version 20, 24 pins ioapic1: misconfigured as apic 0, remapped to apid 3 ioapic2 at mainbus0: apid 4 pa 0xfec83000, version 20, 24 pins ioapic2: misconfigured as apic 0, remapped to apid 4 acpihpet0 at acpi0: 14318179 Hz acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 1 (PALO) acpiprt2 at acpi0: bus 2 (DOBA) acpiprt3 at acpi0: bus 3 (DOBB) acpiprt4 at acpi0: bus 4 (PBLO) acpiprt5 at acpi0: bus 8 (VPR0) acpiprt6 at acpi0: bus 5 (PBHI) acpiprt7 at acpi0: bus 6 (PXB1) acpiprt8 at acpi0: bus 7 (PXB2) acpiprt9 at acpi0: bus 9 (PICH) acpicpu0 at acpi0 bios0: ROM list: 0xc/0xb000! 0xcb000/0x1000 0xcc000/0x800 0xcc800/0x1000 0xcd800/0x2200 0xd/0x600 0xec000/0x4000! ipmi at mainbus0 not configured pci0 at mainbus0 bus 0: configuration mode 1 (bios) pchb0 at pci0 dev 0 function 0 Intel E7520 Host rev 0x09 ppb0 at pci0 dev 2 function 0 Intel E7520 PCIE rev 0x09 pci1 at ppb0 bus 1 ppb1 at pci1 dev 0 function 0 Intel IOP332 PCIE-PCIX rev 0x06 pci2 at ppb1 bus 2 ami0 at pci2 dev 14 function 0 Dell PERC 4e/Di rev 0x06: apic 3 int 14 (irq 7) ami0: Dell 16c, 32b, FW 521X, BIOS vH430, 256MB RAM ami0: 1 channels, 0 FC loops, 1 logical drives scsibus0 at ami0: 40 targets sd0 at scsibus0 targ 0 lun 0: AMI, Host drive #00, SCSI2 0/direct fixed sd0: 70010MB, 512 bytes/sec, 143380480 sec total scsibus1 at ami0: 16 targets safte0 at scsibus1 targ 6 lun 0: PE/PV, 1x2 SCSI BP, 1.0 SCSI2 3/processor fixed ppb2 at pci1 dev 0 function 2 Intel IOP332 PCIE-PCIX rev 0x06 pci3 at ppb2 bus 3 skc0 at pci3 dev 11 function 0 3Com 3c940 rev 0x10, Yukon (0x1): apic 3 int 5 (irq 3) sk0 at skc0 port A: address 00:0a:5e:1b:01:6f eephy0 at sk0 phy 0: 88E1011 Gigabit PHY, rev. 3 ppb3 at pci0 dev 4 function 0 Intel E7520 PCIE rev 0x09 pci4 at ppb3 bus 4 ppb4 at pci0 dev 5 function 0
uvm_mapent_alloc: out of static map entries
Last week I setup an 4.6 i386 OpenBSD server. The hardware is Dell Poweredge 1850 with 2GB RAM. Its fully updated 4.6 stable. In only a few days it has twice hung with the error on screen: uvm_mapent_alloc: out of static map entries I have looked on the web and found various OpenBSD maillist archive references to this but the impression is that this was fixed ages ago. The server should be a very lightly used mail relay running exim, ssh and nothing much else. I certainly dont want to randomly push buttons: http://kerneltrap.org/mailarchive/openbsd-misc/2008/5/16/1842014 but 2 years ago this seemed under control: http://kerneltrap.org/mailarchive/openbsd-misc/2008/5/16/1841134 But it's not really alarming, unless it continues to print that continuously. Its not doing it continuously - just once and then hang. Below is dmesg output. Please let me know if i can provide any more useful information. Many thanks, Alastair Johnson [r...@relayb..com /etc]# dmesg OpenBSD 4.6-stable (GENERIC) #0: Thu Apr 22 22:41:04 BST 2010 r...@relayb:/usr/src/sys/arch/i386/compile/GENERIC cpu0: Intel(R) Xeon(TM) CPU 3.20GHz (GenuineIntel 686-class) 3.20 GHz cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,CNXT-ID,CX16,xTPR real mem = 2146795520 (2047MB) avail mem = 2067058688 (1971MB) mainbus0 at root bios0 at mainbus0: AT/286+ BIOS, date 01/09/06, BIOS32 rev. 0 @ 0xffe90, SMBIOS rev. 2.3 @ 0xf9920 (87 entries) bios0: vendor Dell Computer Corporation version A05 date 01/09/2006 bios0: Dell Computer Corporation PowerEdge 1850 acpi0 at bios0: rev 0 acpi0: tables DSDT FACP APIC SPCR HPET MCFG acpi0: wakeup devices PCI0(S5) PALO(S5) PBLO(S5) VPR0(S5) PBHI(S5) VPR1(S5) PICH(S5) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: apic clock running at 199MHz cpu at mainbus0: not configured ioapic0 at mainbus0: apid 2 pa 0xfec0, version 20, 24 pins ioapic0: misconfigured as apic 0, remapped to apid 2 ioapic1 at mainbus0: apid 3 pa 0xfec8, version 20, 24 pins ioapic1: misconfigured as apic 0, remapped to apid 3 ioapic2 at mainbus0: apid 4 pa 0xfec83000, version 20, 24 pins ioapic2: misconfigured as apic 0, remapped to apid 4 acpihpet0 at acpi0: 14318179 Hz acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 1 (PALO) acpiprt2 at acpi0: bus 2 (DOBA) acpiprt3 at acpi0: bus 3 (DOBB) acpiprt4 at acpi0: bus 4 (PBLO) acpiprt5 at acpi0: bus 8 (VPR0) acpiprt6 at acpi0: bus 5 (PBHI) acpiprt7 at acpi0: bus 6 (PXB1) acpiprt8 at acpi0: bus 7 (PXB2) acpiprt9 at acpi0: bus 9 (PICH) acpicpu0 at acpi0 bios0: ROM list: 0xc/0xb000! 0xcb000/0x1000 0xcc000/0x800 0xcc800/0x1000 0xcd800/0x2200 0xd/0x600 0xec000/0x4000! ipmi at mainbus0 not configured pci0 at mainbus0 bus 0: configuration mode 1 (bios) pchb0 at pci0 dev 0 function 0 Intel E7520 Host rev 0x09 ppb0 at pci0 dev 2 function 0 Intel E7520 PCIE rev 0x09 pci1 at ppb0 bus 1 ppb1 at pci1 dev 0 function 0 Intel IOP332 PCIE-PCIX rev 0x06 pci2 at ppb1 bus 2 ami0 at pci2 dev 14 function 0 Dell PERC 4e/Di rev 0x06: apic 3 int 14 (irq 7) ami0: Dell 16c, 32b, FW 521X, BIOS vH430, 256MB RAM ami0: 1 channels, 0 FC loops, 1 logical drives scsibus0 at ami0: 40 targets sd0 at scsibus0 targ 0 lun 0: AMI, Host drive #00, SCSI2 0/direct fixed sd0: 70010MB, 512 bytes/sec, 143380480 sec total scsibus1 at ami0: 16 targets safte0 at scsibus1 targ 6 lun 0: PE/PV, 1x2 SCSI BP, 1.0 SCSI2 3/processor fixed ppb2 at pci1 dev 0 function 2 Intel IOP332 PCIE-PCIX rev 0x06 pci3 at ppb2 bus 3 skc0 at pci3 dev 11 function 0 3Com 3c940 rev 0x10, Yukon (0x1): apic 3 int 5 (irq 3) sk0 at skc0 port A: address 00:0a:5e:1b:01:6f eephy0 at sk0 phy 0: 88E1011 Gigabit PHY, rev. 3 ppb3 at pci0 dev 4 function 0 Intel E7520 PCIE rev 0x09 pci4 at ppb3 bus 4 ppb4 at pci0 dev 5 function 0 Intel E7520 PCIE rev 0x09 pci5 at ppb4 bus 5 ppb5 at pci5 dev 0 function 0 Intel PCIE-PCIE rev 0x09 pci6 at ppb5 bus 6 em0 at pci6 dev 7 function 0 Intel PRO/1000MT (82541GI) rev 0x05: apic 4 int 0 (irq 11), address 00:13:72:52:09:16 ppb6 at pci5 dev 0 function 2 Intel PCIE-PCIE rev 0x09 pci7 at ppb6 bus 7 em1 at pci7 dev 8 function 0 Intel PRO/1000MT (82541GI) rev 0x05: apic 4 int 1 (irq 3), address 00:13:72:52:09:17 ppb7 at pci0 dev 6 function 0 Intel E7520 PCIE rev 0x09 pci8 at ppb7 bus 8 uhci0 at pci0 dev 29 function 0 Intel 82801EB/ER USB rev 0x02: apic 2 int 16 (irq 11) uhci1 at pci0 dev 29 function 1 Intel 82801EB/ER USB rev 0x02: apic 2 int 19 (irq 10) uhci2 at pci0 dev 29 function 2 Intel 82801EB/ER USB rev 0x02: apic 2 int 18 (irq 7) ehci0 at pci0 dev 29 function 7 Intel 82801EB/ER USB2 rev 0x02: apic 2 int 23 (irq 5) usb0 at ehci0: USB revision 2.0 uhub0 at usb0 Intel EHCI root hub rev 2.00/1.00 addr 1 ppb8 at pci0 dev 30 function 0 Intel 82801BA Hub-to-PCI rev 0xc2 pci9 at ppb8 bus 9 Dell DRAC 4 rev 0x00 at pci9
uvm_mapent_alloc: out of static map entries on 4.4 i386
the server halt after running one day show uvm_mapent_alloc: out of static map entries when i shutdown it how can i fix it? dmesg: OpenBSD 4.4-stable (GENERIC.MP) #2: Mon Nov 3 10:06:38 CST 2008 [EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC.MP cpu0: AMD Athlon(tm) 64 X2 Dual Core Processor 4200+ (AuthenticAMD 686-class, 512KB L2 cache) 2.21 GHz cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,CX16 real mem = 2112319488 (2014MB) avail mem = 2033975296 (1939MB) mainbus0 at root bios0 at mainbus0: AT/286+ BIOS, date 11/30/07, BIOS32 rev. 0 @ 0xfb570, SMBIOS rev. 2.4 @ 0xf0100 (53 entries) bios0: vendor Award Software International, Inc. version F5 date 11/30/2007 bios0: Gigabyte Technology Co., Ltd. GA-MA69G-S3H acpi0 at bios0: rev 0 acpi0: tables DSDT FACP SSDT HPET MCFG APIC acpi0: wakeup devices USB0(S3) USB1(S3) USB2(S3) USB3(S3) USB4(S3) USB5(S3) SBAZ(S4) P2P_(S5) PCE2(S4) PCE3(S4) PCE4(S4) PCE5 (S4) PCE6(S4) PCE7(S4) PCE8(S4) PS2M(S5) PS2K(S5) PCI0(S5) acpitimer0 at acpi0: 3579545 Hz, 32 bits acpihpet0 at acpi0: 14318180 Hz acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: apic clock running at 200MHz cpu1 at mainbus0: apid 1 (application processor) cpu1: AMD Athlon(tm) 64 X2 Dual Core Processor 4200+ (AuthenticAMD 686-class, 512KB L2 cache) 2.21 GHz cpu1: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,CX16 ioapic0 at mainbus0: apid 2 pa 0xfec0, version 21, 24 pins ioapic0: misconfigured as apic 0, remapped to apid 2 acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 2 (P2P_) acpiprt2 at acpi0: bus -1 (PCE2) acpiprt3 at acpi0: bus -1 (PCE3) acpiprt4 at acpi0: bus -1 (PCE4) acpiprt5 at acpi0: bus -1 (PCE5) acpiprt6 at acpi0: bus -1 (PCE6) acpiprt7 at acpi0: bus -1 (PCE7) acpiprt8 at acpi0: bus -1 (PCE8) acpiprt9 at acpi0: bus 1 (AGP_) acpicpu0 at acpi0: PSS acpicpu1 at acpi0: PSS acpibtn0 at acpi0: PWRB bios0: ROM list: 0xc/0xd600 cpu0: PowerNow! K8 2206 MHz: speeds: 2200 2000 1800 1000 MHz pci0 at mainbus0 bus 0: configuration mode 1 (no bios) pchb0 at pci0 dev 0 function 0 ATI RS690 Host rev 0x00 ppb0 at pci0 dev 1 function 0 ATI RS690 PCIE rev 0x00 pci1 at ppb0 bus 1 vga1 at pci1 dev 5 function 0 ATI Radeon X1250 rev 0x00 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) drm at vga1 unsupported ahci0 at pci0 dev 18 function 0 ATI IXP600 SATA rev 0x00: apic 2 int 22 (irq 255), AHCI 1.1 scsibus0 at ahci0: 32 targets, initiator 32 sd0 at scsibus0 targ 2 lun 0: ATA, ST3320620SV, 3.AC SCSI3 0/direct fixed sd0: 305244MB, 38913 cyl, 255 head, 63 sec, 512 bytes/sec, 625140335 sec total piixpm0 at pci0 dev 20 function 0 ATI IXPx00 SMBus rev 0x14: SMI iic0 at piixpm0 iic0: addr 0x2e 00=00 01=00 02=82 03=00 04=a1 05=07 06=00 07=00 words 00=00ff 01=00ff 02=82ff 03=00ff 04=a1ff 05=07ff 06=00ff 07=00ff spdmem0 at iic0 addr 0x50: 1GB DDR2 SDRAM non-parity PC2-6400CL5 spdmem1 at iic0 addr 0x51: 1GB DDR2 SDRAM non-parity PC2-6400CL5 ATI IXP600 IDE rev 0x00 at pci0 dev 20 function 1 not configured pcib0 at pci0 dev 20 function 3 ATI IXP600 ISA rev 0x00 ppb1 at pci0 dev 20 function 4 ATI IXP600 PCI rev 0x00 pci2 at ppb1 bus 2 re0 at pci2 dev 15 function 0 Realtek 8169SC rev 0x10: RTL8169/8110SCd (0x1800), apic 2 int 23 (irq 10), address 00:1d:7d:d 6:7b:9f rgephy0 at re0 phy 7: RTL8169S/8110S PHY, rev. 2 pchb1 at pci0 dev 24 function 0 AMD AMD64 0Fh HyperTransport rev 0x00 pchb2 at pci0 dev 24 function 1 AMD AMD64 0Fh Address Map rev 0x00 pchb3 at pci0 dev 24 function 2 AMD AMD64 0Fh DRAM Cfg rev 0x00 kate0 at pci0 dev 24 function 3 AMD AMD64 0Fh Misc Cfg rev 0x00: core rev BH-G2 isa0 at pcib0 isadma0 at isa0 pckbc0 at isa0 port 0x60/5 pckbd0 at pckbc0 (kbd slot) pckbc0: using irq 1 for kbd slot wskbd0 at pckbd0: console keyboard, using wsdisplay0 it0 at isa0 port 0x2e/2: IT8716F rev 0x03, EC port 0x228 npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16 lms1 at isa0 port 0x238/4 irq 5 wsmouse0 at lms1 mux 0 mtrr: Pentium Pro MTRR support softraid0 at root root on sd0a swap on sd0b dump on sd0b
Re: uvm_mapent_alloc
http://www.strangecosmos.com/images/content/110397.jpg
//art
Vladimir Kirillov [EMAIL PROTECTED] writes:
On 14:44 Fri 10 Oct, Beavis wrote:
thanks for the reply vladimir.
is it needed to upgrade my 4.3 stable to -current? isn't there a patch
available for this?
The 4.3 uvm_map.c is 5 diffs far from this patch
http://www.openbsd.org/cgi-bin/cvsweb/src/sys/uvm/uvm_map.c?r1=1.104#rev1.104
you can generate the diff yourself,
cvs diff -r1.99 -r1.104 uvm_map.c
or here:
Index: uvm_map.c
===
RCS file: /cvs/src/sys/uvm/uvm_map.c,v
retrieving revision 1.99
retrieving revision 1.104
diff -u -p -r1.99 -r1.104
--- uvm_map.c 15 Sep 2007 10:10:37 - 1.99
+++ uvm_map.c 23 Sep 2008 13:25:46 - 1.104
@@ -1,4 +1,4 @@
-/* $OpenBSD: uvm_map.c,v 1.99 2007/09/15 10:10:37 martin Exp $ */
+/* $OpenBSD: uvm_map.c,v 1.104 2008/09/23 13:25:46 art Exp $ */
/* $NetBSD: uvm_map.c,v 1.86 2000/11/27 08:40:03 chs Exp $ */
/*
@@ -98,6 +98,7 @@ static struct timeval uvm_kmapent_last_w
static struct timeval uvm_kmapent_warn_rate = { 10, 0 };
struct uvm_cnt uvm_map_call, map_backmerge, map_forwmerge;
+struct uvm_cnt map_nousermerge;
struct uvm_cnt uvm_mlk_call, uvm_mlk_hint;
const char vmmapbsy[] = vmmapbsy;
@@ -538,6 +539,7 @@ uvm_map_init(void)
UVMCNT_INIT(map_backmerge, UVMCNT_CNT, 0, # uvm_map() back merges,
0);
UVMCNT_INIT(map_forwmerge, UVMCNT_CNT, 0, # uvm_map() missed
forward,
0);
+ UVMCNT_INIT(map_nousermerge, UVMCNT_CNT, 0, # back merges skipped,
0);
UVMCNT_INIT(uvm_mlk_call, UVMCNT_CNT, 0, # map lookup calls, 0);
UVMCNT_INIT(uvm_mlk_hint, UVMCNT_CNT, 0, # map lookup hint hits,
0);
@@ -726,6 +728,8 @@ uvm_map_p(struct vm_map *map, vaddr_t *s
if ((map-flags VM_MAP_INTRSAFE) == 0)
splassert(IPL_NONE);
+ else
+ splassert(IPL_VM);
/*
* step 0: sanity check of protection code
@@ -832,6 +836,15 @@ uvm_map_p(struct vm_map *map, vaddr_t *s
goto step3;
}
+ /*
+* Only merge kernel mappings, but keep track
+* of how much we skipped.
+*/
+ if (map != kernel_map map != kmem_map) {
+ UVMCNT_INCR(map_nousermerge);
+ goto step3;
+ }
+
if (prev_entry-aref.ar_amap) {
error = amap_extend(prev_entry, size);
if (error) {
@@ -897,6 +910,8 @@ step3:
if ((flags UVM_FLAG_OVERLAY) == 0)
new_entry-etype |= UVM_ET_NEEDSCOPY;
}
+ if (flags UVM_FLAG_HOLE)
+ new_entry-etype |= UVM_ET_HOLE;
new_entry-protection = prot;
new_entry-max_protection = maxprot;
@@ -1098,6 +1113,45 @@ uvm_map_spacefits(struct vm_map *map, va
}
/*
+ * uvm_map_pie: return a random load address for a PIE executable
+ * properly aligned.
+ */
+
+#ifndef VM_PIE_MAX_ADDR
+#define VM_PIE_MAX_ADDR (VM_MAXUSER_ADDRESS / 4)
+#endif
+
+#ifndef VM_PIE_MIN_ADDR
+#define VM_PIE_MIN_ADDR VM_MIN_ADDRESS
+#endif
+
+#ifndef VM_PIE_MIN_ALIGN
+#define VM_PIE_MIN_ALIGN PAGE_SIZE
+#endif
+
+vaddr_t
+uvm_map_pie(vaddr_t align)
+{
+ vaddr_t addr, space, min;
+
+ align = MAX(align, VM_PIE_MIN_ALIGN);
+
+ /* round up to next alignment */
+ min = (VM_PIE_MIN_ADDR + align - 1) ~(align - 1);
+
+ if (align = VM_PIE_MAX_ADDR || min = VM_PIE_MAX_ADDR)
+ return (align);
+
+ space = (VM_PIE_MAX_ADDR - min) / align;
+ space = MIN(space, (u_int32_t)-1);
+
+ addr = (vaddr_t)arc4random_uniform((u_int32_t)space) * align;
+ addr += min;
+
+ return (addr);
+}
+
+/*
* uvm_map_hint: return the beginning of the best area suitable for
* creating a new mapping with prot protection.
*/
@@ -1385,6 +1439,8 @@ uvm_unmap_remove(struct vm_map *map, vad
if ((map-flags VM_MAP_INTRSAFE) == 0)
splassert(IPL_NONE);
+ else
+ splassert(IPL_VM);
/*
* find first entry
@@ -1451,7 +1507,9 @@ uvm_unmap_remove(struct vm_map *map, vad
* special case: handle mappings to anonymous kernel objects.
* we want to free these pages right away...
*/
- if (map-flags VM_MAP_INTRSAFE) {
+ if (UVM_ET_ISHOLE(entry)) {
+ /* nothing to do! */
+ } else if (map-flags VM_MAP_INTRSAFE) {
uvm_km_pgremove_intrsafe(entry-start, entry-end);
pmap_kremove(entry-start, len);
} else if (UVM_ET_ISOBJ(entry)
@@ -3697,9 +3755,8 @@
Re: uvm_mapent_alloc
Wow. I'm impressed. So if I mailed you a random diff that you don't
understand you'd happily apply it without having a single clue about
what the diff does and who sent it?
Cool. Can I have your money and business without going through that
hassle? Can't be bothered to make a malicious diff right now,
haven't had coffee yet.
//art
Beavis [EMAIL PROTECTED] writes:
Vladimir,
Sorry to bother you but I tried to apply the patch on uvm_map.c
i copied the patch you gave me here and run
patch -p0 uvm_map.patch
I get some rej. files. any pointers or help will be greatly
appreciated from anyone.
-b
On Fri, Oct 10, 2008 at 3:01 PM, Vladimir Kirillov [EMAIL PROTECTED] wrote:
On 14:44 Fri 10 Oct, Beavis wrote:
thanks for the reply vladimir.
is it needed to upgrade my 4.3 stable to -current? isn't there a patch
available for this?
The 4.3 uvm_map.c is 5 diffs far from this patch
http://www.openbsd.org/cgi-bin/cvsweb/src/sys/uvm/uvm_map.c?r1=1.104#rev1.104
you can generate the diff yourself,
cvs diff -r1.99 -r1.104 uvm_map.c
or here:
Index: uvm_map.c
===
RCS file: /cvs/src/sys/uvm/uvm_map.c,v
retrieving revision 1.99
retrieving revision 1.104
diff -u -p -r1.99 -r1.104
--- uvm_map.c 15 Sep 2007 10:10:37 - 1.99
+++ uvm_map.c 23 Sep 2008 13:25:46 - 1.104
@@ -1,4 +1,4 @@
-/* $OpenBSD: uvm_map.c,v 1.99 2007/09/15 10:10:37 martin Exp $ */
+/* $OpenBSD: uvm_map.c,v 1.104 2008/09/23 13:25:46 art Exp $ */
/* $NetBSD: uvm_map.c,v 1.86 2000/11/27 08:40:03 chs Exp $ */
/*
@@ -98,6 +98,7 @@ static struct timeval uvm_kmapent_last_w
static struct timeval uvm_kmapent_warn_rate = { 10, 0 };
struct uvm_cnt uvm_map_call, map_backmerge, map_forwmerge;
+struct uvm_cnt map_nousermerge;
struct uvm_cnt uvm_mlk_call, uvm_mlk_hint;
const char vmmapbsy[] = vmmapbsy;
@@ -538,6 +539,7 @@ uvm_map_init(void)
UVMCNT_INIT(map_backmerge, UVMCNT_CNT, 0, # uvm_map() back merges,
0);
UVMCNT_INIT(map_forwmerge, UVMCNT_CNT, 0, # uvm_map() missed
forward,
0);
+ UVMCNT_INIT(map_nousermerge, UVMCNT_CNT, 0, # back merges skipped,
0);
UVMCNT_INIT(uvm_mlk_call, UVMCNT_CNT, 0, # map lookup calls, 0);
UVMCNT_INIT(uvm_mlk_hint, UVMCNT_CNT, 0, # map lookup hint hits,
0);
@@ -726,6 +728,8 @@ uvm_map_p(struct vm_map *map, vaddr_t *s
if ((map-flags VM_MAP_INTRSAFE) == 0)
splassert(IPL_NONE);
+ else
+ splassert(IPL_VM);
/*
* step 0: sanity check of protection code
@@ -832,6 +836,15 @@ uvm_map_p(struct vm_map *map, vaddr_t *s
goto step3;
}
+ /*
+* Only merge kernel mappings, but keep track
+* of how much we skipped.
+*/
+ if (map != kernel_map map != kmem_map) {
+ UVMCNT_INCR(map_nousermerge);
+ goto step3;
+ }
+
if (prev_entry-aref.ar_amap) {
error = amap_extend(prev_entry, size);
if (error) {
@@ -897,6 +910,8 @@ step3:
if ((flags UVM_FLAG_OVERLAY) == 0)
new_entry-etype |= UVM_ET_NEEDSCOPY;
}
+ if (flags UVM_FLAG_HOLE)
+ new_entry-etype |= UVM_ET_HOLE;
new_entry-protection = prot;
new_entry-max_protection = maxprot;
@@ -1098,6 +1113,45 @@ uvm_map_spacefits(struct vm_map *map, va
}
/*
+ * uvm_map_pie: return a random load address for a PIE executable
+ * properly aligned.
+ */
+
+#ifndef VM_PIE_MAX_ADDR
+#define VM_PIE_MAX_ADDR (VM_MAXUSER_ADDRESS / 4)
+#endif
+
+#ifndef VM_PIE_MIN_ADDR
+#define VM_PIE_MIN_ADDR VM_MIN_ADDRESS
+#endif
+
+#ifndef VM_PIE_MIN_ALIGN
+#define VM_PIE_MIN_ALIGN PAGE_SIZE
+#endif
+
+vaddr_t
+uvm_map_pie(vaddr_t align)
+{
+ vaddr_t addr, space, min;
+
+ align = MAX(align, VM_PIE_MIN_ALIGN);
+
+ /* round up to next alignment */
+ min = (VM_PIE_MIN_ADDR + align - 1) ~(align - 1);
+
+ if (align = VM_PIE_MAX_ADDR || min = VM_PIE_MAX_ADDR)
+ return (align);
+
+ space = (VM_PIE_MAX_ADDR - min) / align;
+ space = MIN(space, (u_int32_t)-1);
+
+ addr = (vaddr_t)arc4random_uniform((u_int32_t)space) * align;
+ addr += min;
+
+ return (addr);
+}
+
+/*
* uvm_map_hint: return the beginning of the best area suitable for
* creating a new mapping with prot protection.
*/
@@ -1385,6 +1439,8 @@ uvm_unmap_remove(struct vm_map *map, vad
if ((map-flags VM_MAP_INTRSAFE) == 0)
splassert(IPL_NONE);
+ else
+ splassert(IPL_VM);
/*
* find first entry
@@ -1451,7 +1507,9 @@ uvm_unmap_remove(struct vm_map *map, vad
* special case: handle
Re: uvm_mapent_alloc
On a more serious note.
No, there is no diff to 4.3
No, there won't be.
No, the random changes this guy mailed do not solve the problem (at
least one thing here can make things worse and one is pulled out of
its context and will cause problems).
No, I'm not going to tell you which changes there are since there
were lots of things since 4.3 that lowered the pressure on static
map entries.
//art
Artur Grabowski [EMAIL PROTECTED] writes:
Wow. I'm impressed. So if I mailed you a random diff that you don't
understand you'd happily apply it without having a single clue about
what the diff does and who sent it?
Cool. Can I have your money and business without going through that
hassle? Can't be bothered to make a malicious diff right now,
haven't had coffee yet.
//art
Beavis [EMAIL PROTECTED] writes:
Vladimir,
Sorry to bother you but I tried to apply the patch on uvm_map.c
i copied the patch you gave me here and run
patch -p0 uvm_map.patch
I get some rej. files. any pointers or help will be greatly
appreciated from anyone.
-b
On Fri, Oct 10, 2008 at 3:01 PM, Vladimir Kirillov [EMAIL PROTECTED] wrote:
On 14:44 Fri 10 Oct, Beavis wrote:
thanks for the reply vladimir.
is it needed to upgrade my 4.3 stable to -current? isn't there a patch
available for this?
The 4.3 uvm_map.c is 5 diffs far from this patch
http://www.openbsd.org/cgi-bin/cvsweb/src/sys/uvm/uvm_map.c?r1=1.104#rev1.104
you can generate the diff yourself,
cvs diff -r1.99 -r1.104 uvm_map.c
or here:
Index: uvm_map.c
===
RCS file: /cvs/src/sys/uvm/uvm_map.c,v
retrieving revision 1.99
retrieving revision 1.104
diff -u -p -r1.99 -r1.104
--- uvm_map.c 15 Sep 2007 10:10:37 - 1.99
+++ uvm_map.c 23 Sep 2008 13:25:46 - 1.104
@@ -1,4 +1,4 @@
-/* $OpenBSD: uvm_map.c,v 1.99 2007/09/15 10:10:37 martin Exp $ */
+/* $OpenBSD: uvm_map.c,v 1.104 2008/09/23 13:25:46 art Exp $ */
/* $NetBSD: uvm_map.c,v 1.86 2000/11/27 08:40:03 chs Exp $ */
/*
@@ -98,6 +98,7 @@ static struct timeval uvm_kmapent_last_w
static struct timeval uvm_kmapent_warn_rate = { 10, 0 };
struct uvm_cnt uvm_map_call, map_backmerge, map_forwmerge;
+struct uvm_cnt map_nousermerge;
struct uvm_cnt uvm_mlk_call, uvm_mlk_hint;
const char vmmapbsy[] = vmmapbsy;
@@ -538,6 +539,7 @@ uvm_map_init(void)
UVMCNT_INIT(map_backmerge, UVMCNT_CNT, 0, # uvm_map() back merges,
0);
UVMCNT_INIT(map_forwmerge, UVMCNT_CNT, 0, # uvm_map() missed
forward,
0);
+ UVMCNT_INIT(map_nousermerge, UVMCNT_CNT, 0, # back merges
skipped, 0);
UVMCNT_INIT(uvm_mlk_call, UVMCNT_CNT, 0, # map lookup calls, 0);
UVMCNT_INIT(uvm_mlk_hint, UVMCNT_CNT, 0, # map lookup hint hits,
0);
@@ -726,6 +728,8 @@ uvm_map_p(struct vm_map *map, vaddr_t *s
if ((map-flags VM_MAP_INTRSAFE) == 0)
splassert(IPL_NONE);
+ else
+ splassert(IPL_VM);
/*
* step 0: sanity check of protection code
@@ -832,6 +836,15 @@ uvm_map_p(struct vm_map *map, vaddr_t *s
goto step3;
}
+ /*
+* Only merge kernel mappings, but keep track
+* of how much we skipped.
+*/
+ if (map != kernel_map map != kmem_map) {
+ UVMCNT_INCR(map_nousermerge);
+ goto step3;
+ }
+
if (prev_entry-aref.ar_amap) {
error = amap_extend(prev_entry, size);
if (error) {
@@ -897,6 +910,8 @@ step3:
if ((flags UVM_FLAG_OVERLAY) == 0)
new_entry-etype |= UVM_ET_NEEDSCOPY;
}
+ if (flags UVM_FLAG_HOLE)
+ new_entry-etype |= UVM_ET_HOLE;
new_entry-protection = prot;
new_entry-max_protection = maxprot;
@@ -1098,6 +1113,45 @@ uvm_map_spacefits(struct vm_map *map, va
}
/*
+ * uvm_map_pie: return a random load address for a PIE executable
+ * properly aligned.
+ */
+
+#ifndef VM_PIE_MAX_ADDR
+#define VM_PIE_MAX_ADDR (VM_MAXUSER_ADDRESS / 4)
+#endif
+
+#ifndef VM_PIE_MIN_ADDR
+#define VM_PIE_MIN_ADDR VM_MIN_ADDRESS
+#endif
+
+#ifndef VM_PIE_MIN_ALIGN
+#define VM_PIE_MIN_ALIGN PAGE_SIZE
+#endif
+
+vaddr_t
+uvm_map_pie(vaddr_t align)
+{
+ vaddr_t addr, space, min;
+
+ align = MAX(align, VM_PIE_MIN_ALIGN);
+
+ /* round up to next alignment */
+ min = (VM_PIE_MIN_ADDR + align - 1) ~(align - 1);
+
+ if (align = VM_PIE_MAX_ADDR || min = VM_PIE_MAX_ADDR)
+ return (align);
+
+ space = (VM_PIE_MAX_ADDR - min) / align;
+ space = MIN(space, (u_int32_t)-1);
+
+ addr = (vaddr_t)arc4random_uniform((u_int32_t)space) * align;
+ addr += min;
+
+ return (addr);
+}
+
+/*
* uvm_map_hint:
Re: uvm_mapent_alloc
On 10:18 Mon 13 Oct, Artur Grabowski wrote: Wow. I'm impressed. So if I mailed you a random diff that you don't understand you'd happily apply it without having a single clue about what the diff does and who sent it? Cool. Can I have your money and business without going through that hassle? Can't be bothered to make a malicious diff right now, haven't had coffee yet. Whoops, sorry for just mailing the random diff, did it because one familiar guy had the same problem on 4.3 and updating to current after this uvm-related commit got into tree did the trick (probably getting to -current could be the only right answer, not just playing with diffs) My bad. -- Vladimir Kirillov http://darkproger.net
Re: uvm_mapent_alloc
fwiw, i had uvm_mapent_alloc terrors a while back, which have been nonpresent since the july 14th 2008 snapshots -- jared
Re: uvm_mapent_alloc
taking all the heat that was posted on the email i sent.
I thank you all for trying to to give me as much pointers Things i learn.
I run a small hosting site. i use openbsd, this is the first time that
it showed me that uvm_alloc issue. 2nd im not a coder just your
regular guy that likes to use this OS. Regardless, it's true that
getting hand-outs of some diff is bad. specially if I myself don't
know what the heck it does. Given that response I will.
- RTFM my way through diff and patch and all.
- Stop asking stupid questions.
but all things aside. I am not going to back down on supporting this
OS. regardless of people chewing my head on the list or ... some funny
comments like Can I have your money and business (nice one art)
I've moved away from using a free disorganized os (the L in the *nix),
to OpenBSD, clear, simple and precise. (not offending anyone this is
MY POINT OF VIEW)
so thanks again List! :)
regards,
Beavis A. ButtHead
On Mon, Oct 13, 2008 at 2:28 AM, Artur Grabowski [EMAIL PROTECTED] wrote:
On a more serious note.
No, there is no diff to 4.3
No, there won't be.
No, the random changes this guy mailed do not solve the problem (at
least one thing here can make things worse and one is pulled out of
its context and will cause problems).
No, I'm not going to tell you which changes there are since there
were lots of things since 4.3 that lowered the pressure on static
map entries.
//art
Artur Grabowski [EMAIL PROTECTED] writes:
Wow. I'm impressed. So if I mailed you a random diff that you don't
understand you'd happily apply it without having a single clue about
what the diff does and who sent it?
Cool. Can I have your money and business without going through that
hassle? Can't be bothered to make a malicious diff right now,
haven't had coffee yet.
//art
Beavis [EMAIL PROTECTED] writes:
Vladimir,
Sorry to bother you but I tried to apply the patch on uvm_map.c
i copied the patch you gave me here and run
patch -p0 uvm_map.patch
I get some rej. files. any pointers or help will be greatly
appreciated from anyone.
-b
On Fri, Oct 10, 2008 at 3:01 PM, Vladimir Kirillov [EMAIL PROTECTED]
wrote:
On 14:44 Fri 10 Oct, Beavis wrote:
thanks for the reply vladimir.
is it needed to upgrade my 4.3 stable to -current? isn't there a patch
available for this?
The 4.3 uvm_map.c is 5 diffs far from this patch
http://www.openbsd.org/cgi-bin/cvsweb/src/sys/uvm/uvm_map.c?r1=1.104#rev1.104
you can generate the diff yourself,
cvs diff -r1.99 -r1.104 uvm_map.c
or here:
Index: uvm_map.c
===
RCS file: /cvs/src/sys/uvm/uvm_map.c,v
retrieving revision 1.99
retrieving revision 1.104
diff -u -p -r1.99 -r1.104
--- uvm_map.c 15 Sep 2007 10:10:37 - 1.99
+++ uvm_map.c 23 Sep 2008 13:25:46 - 1.104
@@ -1,4 +1,4 @@
-/* $OpenBSD: uvm_map.c,v 1.99 2007/09/15 10:10:37 martin Exp $ */
+/* $OpenBSD: uvm_map.c,v 1.104 2008/09/23 13:25:46 art Exp $ */
/* $NetBSD: uvm_map.c,v 1.86 2000/11/27 08:40:03 chs Exp $ */
/*
@@ -98,6 +98,7 @@ static struct timeval uvm_kmapent_last_w
static struct timeval uvm_kmapent_warn_rate = { 10, 0 };
struct uvm_cnt uvm_map_call, map_backmerge, map_forwmerge;
+struct uvm_cnt map_nousermerge;
struct uvm_cnt uvm_mlk_call, uvm_mlk_hint;
const char vmmapbsy[] = vmmapbsy;
@@ -538,6 +539,7 @@ uvm_map_init(void)
UVMCNT_INIT(map_backmerge, UVMCNT_CNT, 0, # uvm_map() back
merges, 0);
UVMCNT_INIT(map_forwmerge, UVMCNT_CNT, 0, # uvm_map() missed
forward,
0);
+ UVMCNT_INIT(map_nousermerge, UVMCNT_CNT, 0, # back merges
skipped, 0);
UVMCNT_INIT(uvm_mlk_call, UVMCNT_CNT, 0, # map lookup calls, 0);
UVMCNT_INIT(uvm_mlk_hint, UVMCNT_CNT, 0, # map lookup hint hits,
0);
@@ -726,6 +728,8 @@ uvm_map_p(struct vm_map *map, vaddr_t *s
if ((map-flags VM_MAP_INTRSAFE) == 0)
splassert(IPL_NONE);
+ else
+ splassert(IPL_VM);
/*
* step 0: sanity check of protection code
@@ -832,6 +836,15 @@ uvm_map_p(struct vm_map *map, vaddr_t *s
goto step3;
}
+ /*
+* Only merge kernel mappings, but keep track
+* of how much we skipped.
+*/
+ if (map != kernel_map map != kmem_map) {
+ UVMCNT_INCR(map_nousermerge);
+ goto step3;
+ }
+
if (prev_entry-aref.ar_amap) {
error = amap_extend(prev_entry, size);
if (error) {
@@ -897,6 +910,8 @@ step3:
if ((flags UVM_FLAG_OVERLAY) == 0)
new_entry-etype |= UVM_ET_NEEDSCOPY;
}
+ if (flags UVM_FLAG_HOLE)
+ new_entry-etype |= UVM_ET_HOLE;
new_entry-protection =
Re: uvm_mapent_alloc
Vladimir Kirillov wrote: On 19:25 Fri 10 Oct, Beavis wrote: Vladimir, Sorry to bother you but I tried to apply the patch on uvm_map.c i copied the patch you gave me here and run patch -p0 uvm_map.patch I get some rej. files. any pointers or help will be greatly appreciated from anyone. or probably copy the diff to sys/uvm/ and apply it there... This diff is corrupted by your MUA. Alexey
uvm_mapent_alloc
Greetings, I currently have a 4.3 running a modified kernel (disabled ACPI and APM because they hang on my HS20 Blade) I'm receiving the following Error: uvm_mapent_alloc: out of static map entries Is there a way for me to adjust this through sysctl? any help will be greatly appreciated. I run webservers on this box. thanks, -b
Re: uvm_mapent_alloc
On 13:42 Fri 10 Oct, Beavis wrote: Greetings, I currently have a 4.3 running a modified kernel (disabled ACPI and APM because they hang on my HS20 Blade) I'm receiving the following Error: uvm_mapent_alloc: out of static map entries Is there a way for me to adjust this through sysctl? any help will be greatly appreciated. I run webservers on this box. looks like this is fixed in -current, see http://article.gmane.org/gmane.os.openbsd.cvs/79457 -- Vladimir Kirillov http://darkproger.net
Re: uvm_mapent_alloc
thanks for the reply vladimir. is it needed to upgrade my 4.3 stable to -current? isn't there a patch available for this? thank you, -b On Fri, Oct 10, 2008 at 2:40 PM, Vladimir Kirillov [EMAIL PROTECTED] wrote: On 13:42 Fri 10 Oct, Beavis wrote: Greetings, I currently have a 4.3 running a modified kernel (disabled ACPI and APM because they hang on my HS20 Blade) I'm receiving the following Error: uvm_mapent_alloc: out of static map entries Is there a way for me to adjust this through sysctl? any help will be greatly appreciated. I run webservers on this box. looks like this is fixed in -current, see http://article.gmane.org/gmane.os.openbsd.cvs/79457 -- Vladimir Kirillov http://darkproger.net
Re: uvm_mapent_alloc
On 14:44 Fri 10 Oct, Beavis wrote:
thanks for the reply vladimir.
is it needed to upgrade my 4.3 stable to -current? isn't there a patch
available for this?
The 4.3 uvm_map.c is 5 diffs far from this patch
http://www.openbsd.org/cgi-bin/cvsweb/src/sys/uvm/uvm_map.c?r1=1.104#rev1.104
you can generate the diff yourself,
cvs diff -r1.99 -r1.104 uvm_map.c
or here:
Index: uvm_map.c
===
RCS file: /cvs/src/sys/uvm/uvm_map.c,v
retrieving revision 1.99
retrieving revision 1.104
diff -u -p -r1.99 -r1.104
--- uvm_map.c 15 Sep 2007 10:10:37 - 1.99
+++ uvm_map.c 23 Sep 2008 13:25:46 - 1.104
@@ -1,4 +1,4 @@
-/* $OpenBSD: uvm_map.c,v 1.99 2007/09/15 10:10:37 martin Exp $ */
+/* $OpenBSD: uvm_map.c,v 1.104 2008/09/23 13:25:46 art Exp $ */
/* $NetBSD: uvm_map.c,v 1.86 2000/11/27 08:40:03 chs Exp $ */
/*
@@ -98,6 +98,7 @@ static struct timeval uvm_kmapent_last_w
static struct timeval uvm_kmapent_warn_rate = { 10, 0 };
struct uvm_cnt uvm_map_call, map_backmerge, map_forwmerge;
+struct uvm_cnt map_nousermerge;
struct uvm_cnt uvm_mlk_call, uvm_mlk_hint;
const char vmmapbsy[] = vmmapbsy;
@@ -538,6 +539,7 @@ uvm_map_init(void)
UVMCNT_INIT(map_backmerge, UVMCNT_CNT, 0, # uvm_map() back merges, 0);
UVMCNT_INIT(map_forwmerge, UVMCNT_CNT, 0, # uvm_map() missed forward,
0);
+ UVMCNT_INIT(map_nousermerge, UVMCNT_CNT, 0, # back merges skipped, 0);
UVMCNT_INIT(uvm_mlk_call, UVMCNT_CNT, 0, # map lookup calls, 0);
UVMCNT_INIT(uvm_mlk_hint, UVMCNT_CNT, 0, # map lookup hint hits, 0);
@@ -726,6 +728,8 @@ uvm_map_p(struct vm_map *map, vaddr_t *s
if ((map-flags VM_MAP_INTRSAFE) == 0)
splassert(IPL_NONE);
+ else
+ splassert(IPL_VM);
/*
* step 0: sanity check of protection code
@@ -832,6 +836,15 @@ uvm_map_p(struct vm_map *map, vaddr_t *s
goto step3;
}
+ /*
+* Only merge kernel mappings, but keep track
+* of how much we skipped.
+*/
+ if (map != kernel_map map != kmem_map) {
+ UVMCNT_INCR(map_nousermerge);
+ goto step3;
+ }
+
if (prev_entry-aref.ar_amap) {
error = amap_extend(prev_entry, size);
if (error) {
@@ -897,6 +910,8 @@ step3:
if ((flags UVM_FLAG_OVERLAY) == 0)
new_entry-etype |= UVM_ET_NEEDSCOPY;
}
+ if (flags UVM_FLAG_HOLE)
+ new_entry-etype |= UVM_ET_HOLE;
new_entry-protection = prot;
new_entry-max_protection = maxprot;
@@ -1098,6 +1113,45 @@ uvm_map_spacefits(struct vm_map *map, va
}
/*
+ * uvm_map_pie: return a random load address for a PIE executable
+ * properly aligned.
+ */
+
+#ifndef VM_PIE_MAX_ADDR
+#define VM_PIE_MAX_ADDR (VM_MAXUSER_ADDRESS / 4)
+#endif
+
+#ifndef VM_PIE_MIN_ADDR
+#define VM_PIE_MIN_ADDR VM_MIN_ADDRESS
+#endif
+
+#ifndef VM_PIE_MIN_ALIGN
+#define VM_PIE_MIN_ALIGN PAGE_SIZE
+#endif
+
+vaddr_t
+uvm_map_pie(vaddr_t align)
+{
+ vaddr_t addr, space, min;
+
+ align = MAX(align, VM_PIE_MIN_ALIGN);
+
+ /* round up to next alignment */
+ min = (VM_PIE_MIN_ADDR + align - 1) ~(align - 1);
+
+ if (align = VM_PIE_MAX_ADDR || min = VM_PIE_MAX_ADDR)
+ return (align);
+
+ space = (VM_PIE_MAX_ADDR - min) / align;
+ space = MIN(space, (u_int32_t)-1);
+
+ addr = (vaddr_t)arc4random_uniform((u_int32_t)space) * align;
+ addr += min;
+
+ return (addr);
+}
+
+/*
* uvm_map_hint: return the beginning of the best area suitable for
* creating a new mapping with prot protection.
*/
@@ -1385,6 +1439,8 @@ uvm_unmap_remove(struct vm_map *map, vad
if ((map-flags VM_MAP_INTRSAFE) == 0)
splassert(IPL_NONE);
+ else
+ splassert(IPL_VM);
/*
* find first entry
@@ -1451,7 +1507,9 @@ uvm_unmap_remove(struct vm_map *map, vad
* special case: handle mappings to anonymous kernel objects.
* we want to free these pages right away...
*/
- if (map-flags VM_MAP_INTRSAFE) {
+ if (UVM_ET_ISHOLE(entry)) {
+ /* nothing to do! */
+ } else if (map-flags VM_MAP_INTRSAFE) {
uvm_km_pgremove_intrsafe(entry-start, entry-end);
pmap_kremove(entry-start, len);
} else if (UVM_ET_ISOBJ(entry)
@@ -3697,9 +3755,8 @@ uvm_object_printit(uobj, full, pr)
static const char page_flagbits[] =
\20\1BUSY\2WANTED\3TABLED\4CLEAN\5CLEANCHK\6RELEASED\7FAKE\10RDONLY
- \11ZERO\15PAGER1;
-static const char page_pqflagbits[] =
-
Re: uvm_mapent_alloc
On 2008-10-10, Vladimir Kirillov [EMAIL PROTECTED] wrote: On 14:44 Fri 10 Oct, Beavis wrote: thanks for the reply vladimir. is it needed to upgrade my 4.3 stable to -current? isn't there a patch available for this? no there isn't. The 4.3 uvm_map.c is 5 diffs far from this patch http://www.openbsd.org/cgi-bin/cvsweb/src/sys/uvm/uvm_map.c?r1=1.104#rev1.104 you can generate the diff yourself, cvs diff -r1.99 -r1.104 uvm_map.c running -current would be a much smarter move than grafting in this sort of diff.
Re: uvm_mapent_alloc
Vladimir,
Sorry to bother you but I tried to apply the patch on uvm_map.c
i copied the patch you gave me here and run
patch -p0 uvm_map.patch
I get some rej. files. any pointers or help will be greatly
appreciated from anyone.
-b
On Fri, Oct 10, 2008 at 3:01 PM, Vladimir Kirillov [EMAIL PROTECTED] wrote:
On 14:44 Fri 10 Oct, Beavis wrote:
thanks for the reply vladimir.
is it needed to upgrade my 4.3 stable to -current? isn't there a patch
available for this?
The 4.3 uvm_map.c is 5 diffs far from this patch
http://www.openbsd.org/cgi-bin/cvsweb/src/sys/uvm/uvm_map.c?r1=1.104#rev1.104
you can generate the diff yourself,
cvs diff -r1.99 -r1.104 uvm_map.c
or here:
Index: uvm_map.c
===
RCS file: /cvs/src/sys/uvm/uvm_map.c,v
retrieving revision 1.99
retrieving revision 1.104
diff -u -p -r1.99 -r1.104
--- uvm_map.c 15 Sep 2007 10:10:37 - 1.99
+++ uvm_map.c 23 Sep 2008 13:25:46 - 1.104
@@ -1,4 +1,4 @@
-/* $OpenBSD: uvm_map.c,v 1.99 2007/09/15 10:10:37 martin Exp $ */
+/* $OpenBSD: uvm_map.c,v 1.104 2008/09/23 13:25:46 art Exp $ */
/* $NetBSD: uvm_map.c,v 1.86 2000/11/27 08:40:03 chs Exp $ */
/*
@@ -98,6 +98,7 @@ static struct timeval uvm_kmapent_last_w
static struct timeval uvm_kmapent_warn_rate = { 10, 0 };
struct uvm_cnt uvm_map_call, map_backmerge, map_forwmerge;
+struct uvm_cnt map_nousermerge;
struct uvm_cnt uvm_mlk_call, uvm_mlk_hint;
const char vmmapbsy[] = vmmapbsy;
@@ -538,6 +539,7 @@ uvm_map_init(void)
UVMCNT_INIT(map_backmerge, UVMCNT_CNT, 0, # uvm_map() back merges,
0);
UVMCNT_INIT(map_forwmerge, UVMCNT_CNT, 0, # uvm_map() missed forward,
0);
+ UVMCNT_INIT(map_nousermerge, UVMCNT_CNT, 0, # back merges skipped,
0);
UVMCNT_INIT(uvm_mlk_call, UVMCNT_CNT, 0, # map lookup calls, 0);
UVMCNT_INIT(uvm_mlk_hint, UVMCNT_CNT, 0, # map lookup hint hits, 0);
@@ -726,6 +728,8 @@ uvm_map_p(struct vm_map *map, vaddr_t *s
if ((map-flags VM_MAP_INTRSAFE) == 0)
splassert(IPL_NONE);
+ else
+ splassert(IPL_VM);
/*
* step 0: sanity check of protection code
@@ -832,6 +836,15 @@ uvm_map_p(struct vm_map *map, vaddr_t *s
goto step3;
}
+ /*
+* Only merge kernel mappings, but keep track
+* of how much we skipped.
+*/
+ if (map != kernel_map map != kmem_map) {
+ UVMCNT_INCR(map_nousermerge);
+ goto step3;
+ }
+
if (prev_entry-aref.ar_amap) {
error = amap_extend(prev_entry, size);
if (error) {
@@ -897,6 +910,8 @@ step3:
if ((flags UVM_FLAG_OVERLAY) == 0)
new_entry-etype |= UVM_ET_NEEDSCOPY;
}
+ if (flags UVM_FLAG_HOLE)
+ new_entry-etype |= UVM_ET_HOLE;
new_entry-protection = prot;
new_entry-max_protection = maxprot;
@@ -1098,6 +1113,45 @@ uvm_map_spacefits(struct vm_map *map, va
}
/*
+ * uvm_map_pie: return a random load address for a PIE executable
+ * properly aligned.
+ */
+
+#ifndef VM_PIE_MAX_ADDR
+#define VM_PIE_MAX_ADDR (VM_MAXUSER_ADDRESS / 4)
+#endif
+
+#ifndef VM_PIE_MIN_ADDR
+#define VM_PIE_MIN_ADDR VM_MIN_ADDRESS
+#endif
+
+#ifndef VM_PIE_MIN_ALIGN
+#define VM_PIE_MIN_ALIGN PAGE_SIZE
+#endif
+
+vaddr_t
+uvm_map_pie(vaddr_t align)
+{
+ vaddr_t addr, space, min;
+
+ align = MAX(align, VM_PIE_MIN_ALIGN);
+
+ /* round up to next alignment */
+ min = (VM_PIE_MIN_ADDR + align - 1) ~(align - 1);
+
+ if (align = VM_PIE_MAX_ADDR || min = VM_PIE_MAX_ADDR)
+ return (align);
+
+ space = (VM_PIE_MAX_ADDR - min) / align;
+ space = MIN(space, (u_int32_t)-1);
+
+ addr = (vaddr_t)arc4random_uniform((u_int32_t)space) * align;
+ addr += min;
+
+ return (addr);
+}
+
+/*
* uvm_map_hint: return the beginning of the best area suitable for
* creating a new mapping with prot protection.
*/
@@ -1385,6 +1439,8 @@ uvm_unmap_remove(struct vm_map *map, vad
if ((map-flags VM_MAP_INTRSAFE) == 0)
splassert(IPL_NONE);
+ else
+ splassert(IPL_VM);
/*
* find first entry
@@ -1451,7 +1507,9 @@ uvm_unmap_remove(struct vm_map *map, vad
* special case: handle mappings to anonymous kernel objects.
* we want to free these pages right away...
*/
- if (map-flags VM_MAP_INTRSAFE) {
+ if (UVM_ET_ISHOLE(entry)) {
+ /* nothing to do! */
+ } else if (map-flags VM_MAP_INTRSAFE) {
uvm_km_pgremove_intrsafe(entry-start,
Re: uvm_mapent_alloc
On 19:25 Fri 10 Oct, Beavis wrote: Vladimir, Sorry to bother you but I tried to apply the patch on uvm_map.c i copied the patch you gave me here and run patch -p0 uvm_map.patch I get some rej. files. any pointers or help will be greatly appreciated from anyone. you should do patch -p3 in /usr/src but running -current might be better :) -- Vladimir Kirillov http://darkproger.net
Re: uvm_mapent_alloc
On 19:25 Fri 10 Oct, Beavis wrote: Vladimir, Sorry to bother you but I tried to apply the patch on uvm_map.c i copied the patch you gave me here and run patch -p0 uvm_map.patch I get some rej. files. any pointers or help will be greatly appreciated from anyone. or probably copy the diff to sys/uvm/ and apply it there... -- Vladimir Kirillov http://darkproger.net
Re: uvm_mapent_alloc: out of static map entries
On Wed, Jul 16, 2008 at 11:15:50PM -0401, jared r r spiegel wrote: On Wed, Jul 16, 2008 at 09:13:14PM -0400, jared r r spiegel wrote: on jul 11 snapshots now, have gone thru i think 2 or 3 snapshot iterations since ~early/mid june. cracked out again hardcore a bit ago (when it shits out it seems accurate to call it a deadlock), so now am on: OpenBSD 4.4-beta (GENERIC.MP) #800: Mon Jul 14 20:28:22 MDT 2008 still on that snapshot; seems to be OK so far. two curious spuriousness happened today tho: Aug 1 22:12:47 iorek /bsd: uvm_mapent_alloc: out of static map entries Aug 1 23:12:49 iorek /bsd: uvm_mapent_alloc: out of static map entries those're the only instances of this message since previous email. two of those log warnings showed up; i corroborated those times against the log for the vmstat_watcher script i made up (which gets all its data from sysctl(1)): --- Aug 1 22:10:38 iorek vmstat_watcher: vmstat_watcher: name: UVM_amap memuse: 6337000 maxused: 865 limit: 39322000 Aug 1 22:12:40 iorek vmstat_watcher: vmstat_watcher: name: UVM_amap memuse: 6338000 maxused: 865 limit: 39322000 Aug 1 22:12:50 iorek vmstat_watcher: vmstat_watcher: name: UVM_amap memuse: 7014000 maxused: 865 limit: 39322000 Aug 1 22:13:51 iorek vmstat_watcher: vmstat_watcher: name: UVM_amap memuse: 634 maxused: 865 limit: 39322000 ... Aug 1 23:11:56 iorek vmstat_watcher: vmstat_watcher: name: UVM_amap memuse: 6871000 maxused: 865 limit: 39322000 Aug 1 23:12:47 iorek vmstat_watcher: vmstat_watcher: name: UVM_amap memuse: 7951000 maxused: 865 limit: 39322000 Aug 1 23:12:57 iorek vmstat_watcher: vmstat_watcher: name: UVM_amap memuse: 6868000 maxused: 8899000 limit: 39322000 Aug 1 23:13:07 iorek vmstat_watcher: vmstat_watcher: name: UVM_amap memuse: 6869000 maxused: 8899000 limit: 39322000 --- no other warnings in dmesg. so... something made the yelp to the ringbuffer, but i've got no guess as to what it was... no instance of unrecoverable deadlock for sure, no instance of recoverable deadlock (comalock?) *afaik*. i'm fine with the notion that the alloc/free or whatever happens faster than any granularity than my script can catch, but what i actually find curious is that the 'maxused' i get from sysctl also doesn't reflect even coming anywhere near the roof. http://www.ice-nine.org/jrrs/UVM.png is updated, fwiw... -- jared
uvm_mapent_alloc: out of static map entries
cri
on jul 11 snapshots now, have gone thru i think 2 or 3 snapshot iterations
since ~early/mid june.
first recorded/noticed incident of the
'uvm_mapent_alloc: out of static map entries' jobby was jun.16th while
running a DEBUG.MP kernel i had made in attempt to catch more info on a
panic we had seen (kernel/5799). we never did get that panic again, and
had seen a few weird silent lockups. i didn't have syslog setup to retain
logs forever back then so maybe there were UVM_amap hits and i just didn't
see them.
i've checked our sudo logs (those we do have long back in syslog) and the
only changes that appear significant (around before jun.16) are the addition
of the ices-0.4 pkg and some pf.conf edits.
anyway, the DEBUG.MP kernel wasn't making life any better so we went
back to the GENERIC.MP kernel from the same snapshot we were still riding
on (feb.22); things were cool from then (jun.16) to jun.18 where we got
the UVM_amap hit. a few ppl suggested to me offline to keep chunkin' thru
snapshots so right now we're at jul 11th ones and everything has been
peachy (i made a comment in the recent rtorrent misc@ thread regarding this).
until weird o'clock this morning.
power went out at ~midnight EDT last night and came back on ~4:45 EDT.
UVM_amap is off to the races today!
i've got a script i made who runs sysctl kern.malloc.kmemstat on all
the critters in there and peeps to stdout when something goes up
and what it is at -- i pipe that to logger on startup.
http://www.ice-nine.org/jrrs/UVM.png
is made from those syslog messages
you can see in there the few times it has shot up like skyrockets.
of note is that i seem to get the kprintf before the 'maxused' hits
'limit', per sysctl or vmstat. maybe that's not noteworthy tho..
anyway, where it gets calm on the 12th is us doing the jul.11 snapshots
and popping corks and thinking the dragons were slain; we made it
through monday (whereas the previous monday was nasty), but tuesday
hit and we went bonkers; then after the box came back up this morning, the
UVM_amap was going up very high before anyone logged in.
Jul 16 04:57:20 iorek vmstat_watcher: vmstat_watcher: name: UVM_amap memuse:
395000 maxused: 3026000 limit: 39322000
Jul 16 05:37:03 iorek sshd[16018]: Accepted publickey for jrrs from
192.168.7.18 port 12655 ssh2
Jul 16 05:37:04 iorek vmstat_watcher: vmstat_watcher: name: UVM_amap memuse:
803000 maxused: 3026000 limit: 39322000
# i fix some crap i broke in pf.conf and logout
Jul 16 05:45:55 iorek vmstat_watcher: vmstat_watcher: name: UVM_amap memuse:
953000 maxused: 3026000 limit: 39322000
Jul 16 06:28:50 iorek vmstat_watcher: vmstat_watcher: name: UVM_amap memuse:
1028 maxused: 10325000 limit: 39322000
Jul 16 07:30:54 iorek vmstat_watcher: vmstat_watcher: name: UVM_amap memuse:
15285000 maxused: 18052000 limit: 39322000
Jul 16 07:46:01 iorek vmstat_watcher: vmstat_watcher: name: UVM_amap memuse:
14776000 maxused: 18052000 limit: 39322000
Jul 16 07:46:07 iorek sshd[925]: Accepted password for from X port
4484 ssh2
and keeps going up from there until it is bouncing offa ~34073000.
those #s correspond to the most recent spike in the graph linked above.
so barring anything else, i'll try to go to the jul.15th snapshots
tonight.
i have NO IDEA how to discern what it is that is tying up so much memory.
i can certainly see the usage go up if i do things like make new
screen(1) sessions or terminals or run stuff, but if there is any way
to see who is pissing things off so much, i could maybe be more useful.
i took a quick peek at systat(1) and on the iostat page i see:
STATS
23770 numbufs
23769 freebufs
95057 numbufpages
0 numfreepages
24 numdirtypages
95029 numcleanpages
18014398509481983K pendingwrites
0 pendingreads
11 numwrites
0 numreads
159 cachehits
where that number is very stupid high, but i checked a few other machines
who aren't giving me trouble and they're showing '*', so could be that
means nothing.
we used to have an mfs /tmp; i did away with that when we went to the
jul.11 snapshots, so that doesn't seem to be directly related.
other than that about the only non-standard thing that leaps to mind
about the system is having done:
# sh ./MAKEDEV pty{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}
because we have run out of them a few times prior and /etc/ttys has
all those bastards.
here's a vmstat -m:
---
Memory statistics by bucket size
Size In Use Free Requests HighWater Couldfree
1676971 29781 1395214061280 524682
32 2434 3326 12302459 640 53921
64 4510 14754 21418581 3201041099
128 4277619
Re: uvm_mapent_alloc: out of static map entries
On Wed, Jul 16, 2008 at 09:13:14PM -0400, jared r r spiegel wrote: on jul 11 snapshots now, have gone thru i think 2 or 3 snapshot iterations since ~early/mid june. cracked out again hardcore a bit ago (when it shits out it seems accurate to call it a deadlock), so now am on: OpenBSD 4.4-beta (GENERIC.MP) #800: Mon Jul 14 20:28:22 MDT 2008 -- jared
Re: uvm_mapent_alloc: out of static map entries on 4.3 i386
On Fri, May 16, 2008 at 08:21:25AM -0700, Darrian Hale wrote:
Can you please point me to where the diffs you refer to reside?
I'd definitely like to try them out.
most of these are filed in sendbug (some for months) already...
here is a cumulative diff also w/ a bonus himem high-quality
software (in caase you managed to squeeze more than 4g of memory
in your box ;).
cu
--
paranoic mickey (my employers have changed but, the name has remained)
Index: arch/i386/conf/GENERIC
===
RCS file: /cvs/src/sys/arch/i386/conf/GENERIC,v
retrieving revision 1.603
diff -u -r1.603 GENERIC
--- arch/i386/conf/GENERIC 25 Feb 2008 23:16:47 - 1.603
+++ arch/i386/conf/GENERIC 7 May 2008 12:55:43 -
@@ -37,6 +37,8 @@
config bsd swap generic
mainbus0 at root
+himem0 at root # himem.sys
+scsibus* at himem?
cpu0 at mainbus?
bios0 at mainbus0
Index: arch/i386/conf/files.i386
===
RCS file: /cvs/src/sys/arch/i386/conf/files.i386,v
retrieving revision 1.172
diff -u -r1.172 files.i386
--- arch/i386/conf/files.i386 4 Mar 2008 21:14:29 - 1.172
+++ arch/i386/conf/files.i386 7 May 2008 12:55:43 -
@@ -440,6 +440,10 @@
attach esm at mainbus
file arch/i386/i386/esm.cesm needs-flag
+device himem: scsi
+attach himem at root
+filearch/i386/i386/himem.c himem needs-flag
+
#
# VESA
#
Index: arch/i386/i386/autoconf.c
===
RCS file: /cvs/src/sys/arch/i386/i386/autoconf.c,v
retrieving revision 1.78
diff -u -r1.78 autoconf.c
--- arch/i386/i386/autoconf.c 27 Dec 2007 18:04:27 - 1.78
+++ arch/i386/i386/autoconf.c 7 May 2008 12:55:43 -
@@ -71,6 +71,7 @@
#include dev/cons.h
#include ioapic.h
+#include himem.h
#if NIOAPIC 0
#include machine/i82093var.h
@@ -117,6 +118,10 @@
if (config_rootfound(mainbus, NULL) == NULL)
panic(cpu_configure: mainbus not configured);
+
+#if NHIMEM 0
+ config_rootfound(himem, NULL);
+#endif
#if NIOAPIC 0
if (nioapics 0)
Index: arch/i386/i386/himem.c
===
RCS file: arch/i386/i386/himem.c
diff -N arch/i386/i386/himem.c
--- /dev/null 1 Jan 1970 00:00:00 -
+++ arch/i386/i386/himem.c 9 May 2008 09:23:37 -
@@ -0,0 +1,476 @@
+/* $OpenBSD$ */
+
+/*
+ * Copyright (c) 2008 Michael Shalayeff
+ * All rights reserved.
+ *
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED AS IS AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF MIND, USE, DATA OR PROFITS, WHETHER IN
+ * AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
+ * OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+
+#include sys/param.h
+#include sys/systm.h
+#include sys/proc.h
+#include sys/kthread.h
+#include sys/queue.h
+#include sys/mutex.h
+#include sys/buf.h
+#include sys/disk.h
+#include sys/disklabel.h
+
+#include scsi/scsi_all.h
+#include scsi/scsi_disk.h
+#include scsi/scsiconf.h
+#include scsi/sdvar.h
+
+#include uvm/uvm.h
+
+/* arbitrary numbers */
+#defineHIMEM_MAXCMDS 256 /* each one is a page */
+
+/* derived from page table structure */
+#defineHIMEM_OFFSET((sizeof(struct hibuf) + 7) / 8)
+#defineHIMEM_MAXSEGS (512 - HIMEM_OFFSET - 2)
+#defineHIMEM_MAXPHYS (HIMEM_MAXSEGS * PAGE_SIZE)
+
+#defineHIMEM_PDE (8)
+#defineHIMEM_VA(HIMEM_PDE 21)
+#defineHIMEM_LOW (HIMEM_VA + (PAGE_SIZE * HIMEM_OFFSET))
+#defineHIMEM_HIGH (HIMEM_VA + (PAGE_SIZE * 512))
+#definePDE_MASK((512 * (PAGE_SIZE / DEV_BSIZE)) - 1)
+
+void himem_zefix(u_int64_t *, void *, void *, u_int); /* locore.s */
+
+struct hibuf {
+ TAILQ_ENTRY(hibuf) hb_list;
+ paddr_t hb_pa;
+ struct scsi_xfer *hb_xs;
+ void *hb_src, *hb_dst;
+ u_int hb_bno, hb_len;
+ int hb_flags;
+#defineHIMEM_WAKE 0x0001
+};
+
+struct himem_softc {
+ struct device sc_dev;
+ struct scsi_link sc_link;
+
+ int sc_flags;
+#defineHIMEM_RDONLY0x0001
+#defineHIMEM_DISKLABEL 0x0002
+ int sc_size;/* blocks */
+
+ struct proc *sc_kthread;
+ u_int64_t *sc_pdir;
+ paddr_t sc_paddr;
+ struct mutex sc_inmtx;
+ struct mutex sc_freemtx;
+
Re: uvm_mapent_alloc: out of static map entries on 4.3 i386
On Thu, May 15, 2008 at 7:07 PM, Ted Unangst [EMAIL PROTECTED] wrote: On 5/15/08, Kevin [EMAIL PROTECTED] wrote: All, I'm getting quite a lot of these errors in /var/log/messages and can't seem to find an appropriate fix in the archives: May 14 21:05:54 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 14 21:57:47 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 14 23:00:05 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 15 07:27:53 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 15 07:39:59 svr02 /bsd: uvm_mapent_alloc: out of static map entries N.B. This machine serves mirror content for various F/OSS projects in addition to standard www content, so it quite often has 350 users concurrently connected downloading mirrored content (in addition to visitors who're actually visiting the site). Are you using squid as well? You may try doing something like restarting apache. Funny you should ask. Yes and no. We are proxying some of the site's content, but it's with apache's mod_proxy. (No way around this from what we can see as it solves some business needs in terms of content delivery and is an easy fix to an otherwise vexing problem.) Restarting apache always solves the problem, but that's hardly a fix. Sure, I could crontab it to do so automatically and just periodically kick everyone off, but that's super yucky and still doesn't really *solve* the problem I'd feel good about that being the only answer if this were 1998, and we were running IIS on Windows NT 4.0. :-) The problem seems related to certain long running processes with fragmented address spaces. I'll go out on a limb and assume this is the case since some of the files being downloaded are certainly ~100mb or more... some are entire DVD ISOs. I'd say these downloads qualify as long running processes, no? Basically, in order to manage address spaces, the kernel keeps track of a bunch of maps. Entries in these maps are stored in... map entries. In certain situations, the kernel can't wait to allocate a map entry, so it grabs one from a static list. Previously, when they ran out, the kernel paniced. Now it just says uh oh. The kernel will merrily go on making more static entries as needed. I'd keep track of how often the message appears. At some point, it should stop. But it's not really alarming, unless it continues to print that continuously. It isn't alarming per se, but the sites on the server *definitely* stop accepting new visitors at some point. This seems to correlate directly to the uvm_mapent_alloc log events. If it were only the mirror visitors who were getting turned away that would be one thing, but it's actually interfering with regular traffic, too. :-( In short, I'm trying to find a way to: 1.) serve the oodles of mirror content (since we ourselves rely so heavily on F/OSS we want to make sure the mirrors are running both for ourselves and others) and 2.) also keep our normal site traffic humming along, too. I'm hoping to get to the point where It Just Works, and it sure seems like the server itself has the horsepower to do it. If the CPUs were sweating hard or we were swapping heavily, it would make sense, but for it to be knuckling under what seems to me to be relatively light load seems like there's something else I can do to make it happy. Knobs, dials, levers, custom kernels, and custom apache builds they may be, but at this point I'm open to just about anything and everything including witch doctors, Chinese herbalists, and/or exorcists to get the problem solved. :-) Thanks much, Kevin
Re: uvm_mapent_alloc: out of static map entries on 4.3 i386
* Darrian Hale [EMAIL PROTECTED] [2008-05-15 23:06]: What output to you get from 'netstat -m'? I might get yelled at for this as you mentioned people seem to hate custom kernels. But i've had good luck with the following options, I'm not sure which are still relevant, but they help. option NKMEMPAGES_MAX=81920 option NKMEMPAGES=81920 option MAX_KMAPENT=8192 sure, they help. at least if you want to believe they do. randomly pushing buttons you don't understand until it feels better is going to help how? btw, two of the three up there are completely unrelated to the problem at hand and useless these days. -- Henning Brauer, [EMAIL PROTECTED], [EMAIL PROTECTED] BS Web Services, http://bsws.de Full-Service ISP - Secure Hosting, Mail and DNS Services Dedicated Servers, Rootservers, Application Hosting - Hamburg Amsterdam
Re: uvm_mapent_alloc: out of static map entries on 4.3 i386
* Kevin [EMAIL PROTECTED] [2008-05-16 08:34]: Knobs, dials, levers, custom kernels, and custom apache builds they may be, but at this point I'm open to just about anything and everything including witch doctors, Chinese herbalists, and/or exorcists to get the problem solved. :-) well, use a httpd that is better designed than apache. at least for the static content that should be kinda easy with a couple of redirects and a second IP. lighttpd is a good pick. -- Henning Brauer, [EMAIL PROTECTED], [EMAIL PROTECTED] BS Web Services, http://bsws.de Full-Service ISP - Secure Hosting, Mail and DNS Services Dedicated Servers, Rootservers, Application Hosting - Hamburg Amsterdam
Re: uvm_mapent_alloc: out of static map entries on 4.3 i386
On Thu, May 15, 2008 at 10:07:14PM -0400, Ted Unangst wrote: On 5/15/08, Kevin [EMAIL PROTECTED] wrote: All, I'm getting quite a lot of these errors in /var/log/messages and can't seem to find an appropriate fix in the archives: May 14 21:05:54 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 14 21:57:47 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 14 23:00:05 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 15 07:27:53 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 15 07:39:59 svr02 /bsd: uvm_mapent_alloc: out of static map entries N.B. This machine serves mirror content for various F/OSS projects in addition to standard www content, so it quite often has 350 users concurrently connected downloading mirrored content (in addition to visitors who're actually visiting the site). Are you using squid as well? You may try doing something like restarting apache. The problem seems related to certain long running processes with fragmented address spaces. Basically, in order to manage address spaces, the kernel keeps track of a bunch of maps. Entries in these maps are stored in... map entries. In certain situations, the kernel can't wait to allocate a map entry, so it grabs one from a static list. Previously, when they ran out, the kernel paniced. Now it just says uh oh. The kernel will merrily go on making more static entries as needed. the problem is not in the user land. the problem is in i386 pmap which abuses kmem_map that is there for malloc(9)s use and allocates pv_entries from it. this leads to enormous kmem_map fragmentation and unaccounted allocations that does not show up in the vmstat and as well leads to livelocks (sleeping on kmem_map) and out of space in kmem_map panics as well. there is a number of measures to remediate the situation proper - convert pv_entries allocations to pool (i have a diff if you wanna) - backend malloc w/ pool (filed in sendbug) - a number of uvm fixes (such as amap ops) that reduce fragmentation. cu -- paranoic mickey (my employers have changed but, the name has remained)
Re: uvm_mapent_alloc: out of static map entries on 4.3 i386
2008/5/16 Henning Brauer [EMAIL PROTECTED]: well, use a httpd that is better designed than apache. at least for the static content that should be kinda easy with a couple of redirects and a second IP. lighttpd is a good pick. If talking about serving static content: mathopd is doing really good job here. -- Janusz Gumkowski http://www.am.torun.pl/~ja
Re: uvm_mapent_alloc: out of static map entries on 4.3 i386
Can you please point me to where the diffs you refer to reside? I'd definitely like to try them out. Thank you, Darrian On Fri, May 16, 2008 at 3:09 AM, mickey [EMAIL PROTECTED] wrote: the problem is not in the user land. the problem is in i386 pmap which abuses kmem_map that is there for malloc(9)s use and allocates pv_entries from it. this leads to enormous kmem_map fragmentation and unaccounted allocations that does not show up in the vmstat and as well leads to livelocks (sleeping on kmem_map) and out of space in kmem_map panics as well. there is a number of measures to remediate the situation proper - convert pv_entries allocations to pool (i have a diff if you wanna) - backend malloc w/ pool (filed in sendbug) - a number of uvm fixes (such as amap ops) that reduce fragmentation. cu -- paranoic mickey (my employers have changed but, the name has remained)
Re: uvm_mapent_alloc: out of static map entries on 4.3 i386
The problem seems related to certain long running processes with fragmented address spaces. Basically, in order to manage address spaces, the kernel keeps track of a bunch of maps. Entries in these maps are stored in... map entries. In certain situations, the kernel can't wait to allocate a map entry, so it grabs one from a static list. Previously, when they ran out, the kernel paniced. Now it just says uh oh. The kernel will merrily go on making more static entries as needed. the problem is not in the user land. the problem is in i386 pmap which abuses kmem_map that is there for malloc(9)s use and allocates pv_entries from it. this leads to enormous kmem_map fragmentation and unaccounted allocations that does not show up in the vmstat and as well leads to livelocks (sleeping on kmem_map) and out of space in kmem_map panics as well. there is a number of measures to remediate the situation proper - convert pv_entries allocations to pool (i have a diff if you wanna) Yes, please. Definitely... and thanks. FWIW I can bring a spare server online this weekend to keep in the wings in case something goes completely nutty with the diff, so no worries about this affecting production per se. :-) Is the diff against 4.3 or 4.3-current? - backend malloc w/ pool (filed in sendbug) - a number of uvm fixes (such as amap ops) that reduce fragmentation.
uvm_mapent_alloc: out of static map entries on 4.3 i386
All, I'm getting quite a lot of these errors in /var/log/messages and can't seem to find an appropriate fix in the archives: May 14 21:05:54 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 14 21:57:47 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 14 23:00:05 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 15 07:27:53 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 15 07:39:59 svr02 /bsd: uvm_mapent_alloc: out of static map entries N.B. This machine serves mirror content for various F/OSS projects in addition to standard www content, so it quite often has 350 users concurrently connected downloading mirrored content (in addition to visitors who're actually visiting the site). These messages correspond almost exactly with two things: 1.) the sites having quite a few visitors 2.) the sites becoming unavailable. In most cases, it fixes itself when the freeloaders (errr downloaders ;-) complete their file transfers. Possibly worth noting: 1.) We've had to crank various settings in Apache to keep serving traffic, as the stock settings were too low: we were reaching the max daemons for Apache so new visitors were just out-of-luck. 2.) When the system begins to knuckle under load, I'm taking a snapshot of various bits like the following. Here's one example: load averages: 0.45, 0.47, 0.4007:40:00 247 processes: 245 idle, 2 on processor CPU0 states: 7.2% user, 0.0% nice, 2.6% system, 2.2% interrupt, 88.0% idle CPU1 states: 3.6% user, 0.0% nice, 0.3% system, 1.9% interrupt, 94.3% idle Memory: Real: 339M/737M act/tot Free: 1272M Swap: 0K/518M used/tot From the archives this seems to be something for which a fix *used* to be cranking up the following: maxusers 64 option BUFCACHEPERCENT=25 option MULTIPROCESSOR option MAX_KMAPENT=4000 This hardly seems a real fix though--especially given everyone's hatred of knobs, custom kernels, and such though I'm certainly open to it if we can continue to keep the sites--and the mirrors--up. I think I've mentioned everything noteworthy though cluestick applications are welcome. Thanks, Kevin Here's the dmesg for any interested parties: OpenBSD 4.3 (GENERIC.MP) #2: Fri Apr 11 09:00:02 PDT 2008 [EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC.MP cpu0: Intel(R) Pentium(R) III CPU family 1266MHz (GenuineIntel 686-class) 1.27 GHz cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE real mem = 2146992128 (2047MB) avail mem = 2067959808 (1972MB) mainbus0 at root bios0 at mainbus0: AT/286+ BIOS, date 01/25/02, BIOS32 rev. 0 @ 0xffe90, SMBIOS rev. 2.3 @ 0xfae20 (49 entries) bios0: vendor Dell Computer Corporation version A06 date 01/25/2002 bios0: Dell Computer Corporation PowerEdge 2550 acpi0 at bios0: rev 0 acpi0: tables DSDT FACP APIC SPCR acpi0: wakeup devices PCI1(S5) PCI2(S5) PCI0(S5) acpitimer0 at acpi0: 3579545 Hz, 32 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 1 (boot processor) cpu0: apic clock running at 132MHz cpu1 at mainbus0: apid 0 (application processor) cpu1: Intel(R) Pentium(R) III CPU family 1266MHz (GenuineIntel 686-class) 1.27 GHz cpu1: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE ioapic0 at mainbus0: apid 2 pa 0xfec0, version 11, 16 pins ioapic0: misconfigured as apic 0, remapped to apid 2 ioapic1 at mainbus0: apid 3 pa 0xfec01000, version 11, 16 pins ioapic1: misconfigured as apic 0, remapped to apid 3 acpiprt0 at acpi0: bus 0 (PCI1) acpiprt1 at acpi0: bus 1 (PCI2) acpiprt2 at acpi0: bus 2 (PCI0) acpiprt3 at acpi0: bus 3 (I960) acpicpu0 at acpi0 acpicpu1 at acpi0 bios0: ROM list: 0xc/0x8000 0xcc000/0x8000 0xec000/0x4000! esm0 at mainbus0 esm0: PowerEdge 2550 Embedded Server Management 5.50 esm0: Primary System Backplane 1.30 pci0 at mainbus0 bus 0: configuration mode 1 (no bios) pchb0 at pci0 dev 0 function 0 ServerWorks CNB20HE Host rev 0x23 pchb1 at pci0 dev 0 function 1 ServerWorks CNB20HE Host rev 0x01 pci1 at pchb1 bus 2 ppb0 at pci1 dev 2 function 0 Intel i960 RM PCI-PCI rev 0x02 pci2 at ppb0 bus 3 ahc0 at pci2 dev 4 function 0 Adaptec AIC-7899 U160 rev 0x01: apic 3 int 15 (irq 11) scsibus0 at ahc0: 16 targets ahc1 at pci2 dev 4 function 1 Adaptec AIC-7899 U160 rev 0x01: apic 3 int 14 (irq 10) scsibus1 at ahc1: 16 targets fxp0 at pci1 dev 4 function 0 Intel 8255x rev 0x08, i82559: apic 3 int 0 (irq 5), address 00:06:5b:3b:61:27 inphy0 at fxp0 phy 1: i82555 10/100 PHY, rev. 4 pchb2 at pci0 dev 0 function 2 ServerWorks CNB20HE Host rev 0x01 pchb3 at pci0 dev 0 function 3 ServerWorks CNB20HE Host rev 0x01 pci3 at pchb3 bus 1 bge0 at pci3 dev 8 function 0 Broadcom BCM5700 rev 0x12, BCM5700 B2 (0x7102): apic 3 int 1 (irq 10), address 00:06:5b:3b:61:28 brgphy0 at bge0 phy 1: BCM5401 10/100/1000baseT PHY, rev. 3 gdt0 at pci0 dev 4 function 0 Vortex GDT6x18RD rev 0x00: apic 3 int 6 (irq 11) dpmem c8000 2
Re: uvm_mapent_alloc: out of static map entries on 4.3 i386
What output to you get from 'netstat -m'? I might get yelled at for this as you mentioned people seem to hate custom kernels. But i've had good luck with the following options, I'm not sure which are still relevant, but they help. option NKMEMPAGES_MAX=81920 option NKMEMPAGES=81920 option MAX_KMAPENT=8192 I've always received that error you described on any high load openbsd box. Even with the above changes, you will eventually get the same error as your new limits are reached. If you come up with any better solutions, please let me know, i'd be very interested to hear them. -Darrian On Thu, May 15, 2008 at 10:29 AM, Kevin [EMAIL PROTECTED] wrote: All, I'm getting quite a lot of these errors in /var/log/messages and can't seem to find an appropriate fix in the archives: May 14 21:05:54 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 14 21:57:47 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 14 23:00:05 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 15 07:27:53 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 15 07:39:59 svr02 /bsd: uvm_mapent_alloc: out of static map entries N.B. This machine serves mirror content for various F/OSS projects in addition to standard www content, so it quite often has 350 users concurrently connected downloading mirrored content (in addition to visitors who're actually visiting the site). These messages correspond almost exactly with two things: 1.) the sites having quite a few visitors 2.) the sites becoming unavailable. In most cases, it fixes itself when the freeloaders (errr downloaders ;-) complete their file transfers. Possibly worth noting: 1.) We've had to crank various settings in Apache to keep serving traffic, as the stock settings were too low: we were reaching the max daemons for Apache so new visitors were just out-of-luck. 2.) When the system begins to knuckle under load, I'm taking a snapshot of various bits like the following. Here's one example: load averages: 0.45, 0.47, 0.4007:40:00 247 processes: 245 idle, 2 on processor CPU0 states: 7.2% user, 0.0% nice, 2.6% system, 2.2% interrupt, 88.0% idle CPU1 states: 3.6% user, 0.0% nice, 0.3% system, 1.9% interrupt, 94.3% idle Memory: Real: 339M/737M act/tot Free: 1272M Swap: 0K/518M used/tot From the archives this seems to be something for which a fix *used* to be cranking up the following: maxusers 64 option BUFCACHEPERCENT=25 option MULTIPROCESSOR option MAX_KMAPENT=4000 This hardly seems a real fix though--especially given everyone's hatred of knobs, custom kernels, and such though I'm certainly open to it if we can continue to keep the sites--and the mirrors--up. I think I've mentioned everything noteworthy though cluestick applications are welcome. Thanks, Kevin Here's the dmesg for any interested parties: OpenBSD 4.3 (GENERIC.MP) #2: Fri Apr 11 09:00:02 PDT 2008 [EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC.MP cpu0: Intel(R) Pentium(R) III CPU family 1266MHz (GenuineIntel 686-class) 1.27 GHz cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE real mem = 2146992128 (2047MB) avail mem = 2067959808 (1972MB) mainbus0 at root bios0 at mainbus0: AT/286+ BIOS, date 01/25/02, BIOS32 rev. 0 @ 0xffe90, SMBIOS rev. 2.3 @ 0xfae20 (49 entries) bios0: vendor Dell Computer Corporation version A06 date 01/25/2002 bios0: Dell Computer Corporation PowerEdge 2550 acpi0 at bios0: rev 0 acpi0: tables DSDT FACP APIC SPCR acpi0: wakeup devices PCI1(S5) PCI2(S5) PCI0(S5) acpitimer0 at acpi0: 3579545 Hz, 32 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 1 (boot processor) cpu0: apic clock running at 132MHz cpu1 at mainbus0: apid 0 (application processor) cpu1: Intel(R) Pentium(R) III CPU family 1266MHz (GenuineIntel 686-class) 1.27 GHz cpu1: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE ioapic0 at mainbus0: apid 2 pa 0xfec0, version 11, 16 pins ioapic0: misconfigured as apic 0, remapped to apid 2 ioapic1 at mainbus0: apid 3 pa 0xfec01000, version 11, 16 pins ioapic1: misconfigured as apic 0, remapped to apid 3 acpiprt0 at acpi0: bus 0 (PCI1) acpiprt1 at acpi0: bus 1 (PCI2) acpiprt2 at acpi0: bus 2 (PCI0) acpiprt3 at acpi0: bus 3 (I960) acpicpu0 at acpi0 acpicpu1 at acpi0 bios0: ROM list: 0xc/0x8000 0xcc000/0x8000 0xec000/0x4000! esm0 at mainbus0 esm0: PowerEdge 2550 Embedded Server Management 5.50 esm0: Primary System Backplane 1.30 pci0 at mainbus0 bus 0: configuration mode 1 (no bios) pchb0 at pci0 dev 0 function 0 ServerWorks CNB20HE Host rev 0x23 pchb1 at pci0 dev 0 function 1 ServerWorks CNB20HE Host rev 0x01 pci1 at pchb1 bus 2 ppb0 at pci1 dev 2 function 0 Intel i960 RM PCI-PCI rev 0x02 pci2 at ppb0 bus 3 ahc0 at pci2 dev 4 function 0 Adaptec AIC-7899 U160 rev 0x01: apic 3
Re: uvm_mapent_alloc: out of static map entries on 4.3 i386
On Thu, May 15, 2008 at 2:00 PM, Darrian Hale [EMAIL PROTECTED] wrote: What output to you get from 'netstat -m'? 2867 mbufs in use: 2566 mbufs allocated to data 274 mbufs allocated to packet headers 27 mbufs allocated to socket names and addresses 1129/5450/6144 mbuf clusters in use (current/peak/max) 13028 Kbytes allocated to network (22% in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines I might get yelled at for this as you mentioned people seem to hate custom kernels. But i've had good luck with the following options, I'm not sure which are still relevant, but they help. option NKMEMPAGES_MAX=81920 option NKMEMPAGES=81920 option MAX_KMAPENT=8192 I've always received that error you described on any high load openbsd box. Even with the above changes, you will eventually get the same error as your new limits are reached. If you come up with any better solutions, please let me know, i'd be very interested to hear them. -Darrian On Thu, May 15, 2008 at 10:29 AM, Kevin [EMAIL PROTECTED] wrote: All, I'm getting quite a lot of these errors in /var/log/messages and can't seem to find an appropriate fix in the archives: May 14 21:05:54 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 14 21:57:47 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 14 23:00:05 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 15 07:27:53 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 15 07:39:59 svr02 /bsd: uvm_mapent_alloc: out of static map entries N.B. This machine serves mirror content for various F/OSS projects in addition to standard www content, so it quite often has 350 users concurrently connected downloading mirrored content (in addition to visitors who're actually visiting the site). These messages correspond almost exactly with two things: 1.) the sites having quite a few visitors 2.) the sites becoming unavailable. In most cases, it fixes itself when the freeloaders (errr downloaders ;-) complete their file transfers. Possibly worth noting: 1.) We've had to crank various settings in Apache to keep serving traffic, as the stock settings were too low: we were reaching the max daemons for Apache so new visitors were just out-of-luck. 2.) When the system begins to knuckle under load, I'm taking a snapshot of various bits like the following. Here's one example: load averages: 0.45, 0.47, 0.4007:40:00 247 processes: 245 idle, 2 on processor CPU0 states: 7.2% user, 0.0% nice, 2.6% system, 2.2% interrupt, 88.0% idle CPU1 states: 3.6% user, 0.0% nice, 0.3% system, 1.9% interrupt, 94.3% idle Memory: Real: 339M/737M act/tot Free: 1272M Swap: 0K/518M used/tot From the archives this seems to be something for which a fix *used* to be cranking up the following: maxusers 64 option BUFCACHEPERCENT=25 option MULTIPROCESSOR option MAX_KMAPENT=4000 This hardly seems a real fix though--especially given everyone's hatred of knobs, custom kernels, and such though I'm certainly open to it if we can continue to keep the sites--and the mirrors--up. I think I've mentioned everything noteworthy though cluestick applications are welcome. Thanks, Kevin Here's the dmesg for any interested parties: OpenBSD 4.3 (GENERIC.MP) #2: Fri Apr 11 09:00:02 PDT 2008 [EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC.MP cpu0: Intel(R) Pentium(R) III CPU family 1266MHz (GenuineIntel 686-class) 1.27 GHz cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE real mem = 2146992128 (2047MB) avail mem = 2067959808 (1972MB) mainbus0 at root bios0 at mainbus0: AT/286+ BIOS, date 01/25/02, BIOS32 rev. 0 @ 0xffe90, SMBIOS rev. 2.3 @ 0xfae20 (49 entries) bios0: vendor Dell Computer Corporation version A06 date 01/25/2002 bios0: Dell Computer Corporation PowerEdge 2550 acpi0 at bios0: rev 0 acpi0: tables DSDT FACP APIC SPCR acpi0: wakeup devices PCI1(S5) PCI2(S5) PCI0(S5) acpitimer0 at acpi0: 3579545 Hz, 32 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 1 (boot processor) cpu0: apic clock running at 132MHz cpu1 at mainbus0: apid 0 (application processor) cpu1: Intel(R) Pentium(R) III CPU family 1266MHz (GenuineIntel 686-class) 1.27 GHz cpu1: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE ioapic0 at mainbus0: apid 2 pa 0xfec0, version 11, 16 pins ioapic0: misconfigured as apic 0, remapped to apid 2 ioapic1 at mainbus0: apid 3 pa 0xfec01000, version 11, 16 pins ioapic1: misconfigured as apic 0, remapped to apid 3 acpiprt0 at acpi0: bus 0 (PCI1) acpiprt1 at acpi0: bus 1 (PCI2) acpiprt2 at acpi0: bus 2 (PCI0) acpiprt3 at acpi0: bus 3 (I960) acpicpu0 at acpi0 acpicpu1 at acpi0 bios0: ROM list: 0xc/0x8000 0xcc000/0x8000 0xec000/0x4000! esm0 at mainbus0 esm0
Re: uvm_mapent_alloc: out of static map entries on 4.3 i386
I see Allen beat me to the reply with the requested netstat data below, but in the mean time, I'm going to do the unthinkable and build a custom kernel with your mods and see where the chips fall. :-) Thanks for the suggestion. Kevin On Thu, May 15, 2008 at 2:45 PM, Allen [EMAIL PROTECTED] wrote: On Thu, May 15, 2008 at 2:00 PM, Darrian Hale [EMAIL PROTECTED] wrote: What output to you get from 'netstat -m'? 2867 mbufs in use: 2566 mbufs allocated to data 274 mbufs allocated to packet headers 27 mbufs allocated to socket names and addresses 1129/5450/6144 mbuf clusters in use (current/peak/max) 13028 Kbytes allocated to network (22% in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines I might get yelled at for this as you mentioned people seem to hate custom kernels. But i've had good luck with the following options, I'm not sure which are still relevant, but they help. option NKMEMPAGES_MAX=81920 option NKMEMPAGES=81920 option MAX_KMAPENT=8192 I've always received that error you described on any high load openbsd box. Even with the above changes, you will eventually get the same error as your new limits are reached. If you come up with any better solutions, please let me know, i'd be very interested to hear them. -Darrian On Thu, May 15, 2008 at 10:29 AM, Kevin [EMAIL PROTECTED] wrote: All, I'm getting quite a lot of these errors in /var/log/messages and can't seem to find an appropriate fix in the archives: May 14 21:05:54 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 14 21:57:47 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 14 23:00:05 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 15 07:27:53 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 15 07:39:59 svr02 /bsd: uvm_mapent_alloc: out of static map entries N.B. This machine serves mirror content for various F/OSS projects in addition to standard www content, so it quite often has 350 users concurrently connected downloading mirrored content (in addition to visitors who're actually visiting the site). These messages correspond almost exactly with two things: 1.) the sites having quite a few visitors 2.) the sites becoming unavailable. In most cases, it fixes itself when the freeloaders (errr downloaders ;-) complete their file transfers. Possibly worth noting: 1.) We've had to crank various settings in Apache to keep serving traffic, as the stock settings were too low: we were reaching the max daemons for Apache so new visitors were just out-of-luck. 2.) When the system begins to knuckle under load, I'm taking a snapshot of various bits like the following. Here's one example: load averages: 0.45, 0.47, 0.4007:40:00 247 processes: 245 idle, 2 on processor CPU0 states: 7.2% user, 0.0% nice, 2.6% system, 2.2% interrupt, 88.0% idle CPU1 states: 3.6% user, 0.0% nice, 0.3% system, 1.9% interrupt, 94.3% idle Memory: Real: 339M/737M act/tot Free: 1272M Swap: 0K/518M used/tot From the archives this seems to be something for which a fix *used* to be cranking up the following: maxusers 64 option BUFCACHEPERCENT=25 option MULTIPROCESSOR option MAX_KMAPENT=4000 This hardly seems a real fix though--especially given everyone's hatred of knobs, custom kernels, and such though I'm certainly open to it if we can continue to keep the sites--and the mirrors--up. I think I've mentioned everything noteworthy though cluestick applications are welcome. Thanks, Kevin Here's the dmesg for any interested parties: OpenBSD 4.3 (GENERIC.MP) #2: Fri Apr 11 09:00:02 PDT 2008 [EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC.MP cpu0: Intel(R) Pentium(R) III CPU family 1266MHz (GenuineIntel 686-class) 1.27 GHz cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE real mem = 2146992128 (2047MB) avail mem = 2067959808 (1972MB) mainbus0 at root bios0 at mainbus0: AT/286+ BIOS, date 01/25/02, BIOS32 rev. 0 @ 0xffe90, SMBIOS rev. 2.3 @ 0xfae20 (49 entries) bios0: vendor Dell Computer Corporation version A06 date 01/25/2002 bios0: Dell Computer Corporation PowerEdge 2550 acpi0 at bios0: rev 0 acpi0: tables DSDT FACP APIC SPCR acpi0: wakeup devices PCI1(S5) PCI2(S5) PCI0(S5) acpitimer0 at acpi0: 3579545 Hz, 32 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 1 (boot processor) cpu0: apic clock running at 132MHz cpu1 at mainbus0: apid 0 (application processor) cpu1: Intel(R) Pentium(R) III CPU family 1266MHz (GenuineIntel 686-class) 1.27 GHz cpu1: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE ioapic0 at mainbus0: apid 2 pa 0xfec0, version 11, 16 pins ioapic0: misconfigured as apic 0, remapped to apid 2 ioapic1 at mainbus0: apid 3 pa 0xfec01000, version 11, 16 pins
Re: uvm_mapent_alloc: out of static map entries on 4.3 i386
Based on that netstat output, things look OK on your system. On some of my heavier loaded systems, I will see the peak mbuf use hit the max. Good luck, and as I said if you come up with something better, please let me know. -Darrian On Thu, May 15, 2008 at 2:59 PM, Kevin [EMAIL PROTECTED] wrote: I see Allen beat me to the reply with the requested netstat data below, but in the mean time, I'm going to do the unthinkable and build a custom kernel with your mods and see where the chips fall. :-) Thanks for the suggestion. Kevin On Thu, May 15, 2008 at 2:45 PM, Allen [EMAIL PROTECTED] wrote: On Thu, May 15, 2008 at 2:00 PM, Darrian Hale [EMAIL PROTECTED] wrote: What output to you get from 'netstat -m'? 2867 mbufs in use: 2566 mbufs allocated to data 274 mbufs allocated to packet headers 27 mbufs allocated to socket names and addresses 1129/5450/6144 mbuf clusters in use (current/peak/max) 13028 Kbytes allocated to network (22% in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines I might get yelled at for this as you mentioned people seem to hate custom kernels. But i've had good luck with the following options, I'm not sure which are still relevant, but they help. option NKMEMPAGES_MAX=81920 option NKMEMPAGES=81920 option MAX_KMAPENT=8192 I've always received that error you described on any high load openbsd box. Even with the above changes, you will eventually get the same error as your new limits are reached. If you come up with any better solutions, please let me know, i'd be very interested to hear them. -Darrian On Thu, May 15, 2008 at 10:29 AM, Kevin [EMAIL PROTECTED] wrote: All, I'm getting quite a lot of these errors in /var/log/messages and can't seem to find an appropriate fix in the archives: May 14 21:05:54 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 14 21:57:47 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 14 23:00:05 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 15 07:27:53 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 15 07:39:59 svr02 /bsd: uvm_mapent_alloc: out of static map entries N.B. This machine serves mirror content for various F/OSS projects in addition to standard www content, so it quite often has 350 users concurrently connected downloading mirrored content (in addition to visitors who're actually visiting the site). These messages correspond almost exactly with two things: 1.) the sites having quite a few visitors 2.) the sites becoming unavailable. In most cases, it fixes itself when the freeloaders (errr downloaders ;-) complete their file transfers. Possibly worth noting: 1.) We've had to crank various settings in Apache to keep serving traffic, as the stock settings were too low: we were reaching the max daemons for Apache so new visitors were just out-of-luck. 2.) When the system begins to knuckle under load, I'm taking a snapshot of various bits like the following. Here's one example: load averages: 0.45, 0.47, 0.4007:40:00 247 processes: 245 idle, 2 on processor CPU0 states: 7.2% user, 0.0% nice, 2.6% system, 2.2% interrupt, 88.0% idle CPU1 states: 3.6% user, 0.0% nice, 0.3% system, 1.9% interrupt, 94.3% idle Memory: Real: 339M/737M act/tot Free: 1272M Swap: 0K/518M used/tot From the archives this seems to be something for which a fix *used* to be cranking up the following: maxusers 64 option BUFCACHEPERCENT=25 option MULTIPROCESSOR option MAX_KMAPENT=4000 This hardly seems a real fix though--especially given everyone's hatred of knobs, custom kernels, and such though I'm certainly open to it if we can continue to keep the sites--and the mirrors--up. I think I've mentioned everything noteworthy though cluestick applications are welcome. Thanks, Kevin Here's the dmesg for any interested parties: OpenBSD 4.3 (GENERIC.MP) #2: Fri Apr 11 09:00:02 PDT 2008 [EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC.MP cpu0: Intel(R) Pentium(R) III CPU family 1266MHz (GenuineIntel 686-class) 1.27 GHz cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE real mem = 2146992128 (2047MB) avail mem = 2067959808 (1972MB) mainbus0 at root bios0 at mainbus0: AT/286+ BIOS, date 01/25/02, BIOS32 rev. 0 @ 0xffe90, SMBIOS rev. 2.3 @ 0xfae20 (49 entries) bios0: vendor Dell Computer Corporation version A06 date 01/25/2002 bios0: Dell Computer Corporation PowerEdge 2550 acpi0 at bios0: rev 0 acpi0: tables DSDT FACP APIC SPCR acpi0: wakeup devices PCI1(S5) PCI2(S5) PCI0(S5) acpitimer0 at acpi0: 3579545 Hz, 32 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 1 (boot processor) cpu0: apic clock running at 132MHz cpu1 at mainbus0: apid 0 (application processor) cpu1: Intel(R) Pentium(R) III CPU family 1266MHz
Re: uvm_mapent_alloc: out of static map entries on 4.3 i386
On 5/15/08, Kevin [EMAIL PROTECTED] wrote: All, I'm getting quite a lot of these errors in /var/log/messages and can't seem to find an appropriate fix in the archives: May 14 21:05:54 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 14 21:57:47 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 14 23:00:05 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 15 07:27:53 svr02 /bsd: uvm_mapent_alloc: out of static map entries May 15 07:39:59 svr02 /bsd: uvm_mapent_alloc: out of static map entries N.B. This machine serves mirror content for various F/OSS projects in addition to standard www content, so it quite often has 350 users concurrently connected downloading mirrored content (in addition to visitors who're actually visiting the site). Are you using squid as well? You may try doing something like restarting apache. The problem seems related to certain long running processes with fragmented address spaces. Basically, in order to manage address spaces, the kernel keeps track of a bunch of maps. Entries in these maps are stored in... map entries. In certain situations, the kernel can't wait to allocate a map entry, so it grabs one from a static list. Previously, when they ran out, the kernel paniced. Now it just says uh oh. The kernel will merrily go on making more static entries as needed. I'd keep track of how often the message appears. At some point, it should stop. But it's not really alarming, unless it continues to print that continuously.
uvm_mapent_alloc: out of static map entries
on my not so busy i386 4.0-current web server I get uvm_mapent_alloc: out of static map entries ~ once every two days. The archives bear a wide range of suggestions, from tweaking kernel feature xy to not touching anything, because that's stupid. However, this message bothers me a bit and so I am seeking for advice how to track the problem down (if any). The machine runs apache with mod_gzip, openssl and php5 plus myqsql. I have set those up in a most default way, for mysql I've followed the guidelines on openbsdsupport.org. below you'll find output of vmstat -m, netstat -n and ps aux thanks, Stephan $ vmstat -m Memory statistics by bucket size Size In Use Free Requests HighWater Couldfree 16 9497 285415 2199511001280 83060 32 3563 108949 102837831 640 903052 64 4463 613294411422 320 338543 128 5238 9162 987939 160 7321 256 5185239 826192 80 18140 512 92443401313642 40 79758 1024 2241 516078151 201445813 2048 163 23 32751 10 11346 4096 30 32 87109 5 59289 8192 72 48770 5474 163845 0 1905 5 0 32768 33 0 1527 5 0 65536 10 0462 5 0 1310720 0864 5 0 2621440 0 1241 5 0 5242880 0 3584 5 0 Memory usage type by bucket size Size Type(s) 16 devbuf, pcb, routetbl, ifaddr, sysctl, UFS mount, sem, dirhash, in_multi, exec, xform_data, VM swap, UVM amap, UVM aobj, USB, USB device, temp 32 devbuf, pcb, routetbl, ifaddr, vnodes, UFS mount, sem, dirhash, proc, VFS cluster, ether_multi, xform_data, VM swap, UVM amap, USB, packet tags, temp 64 devbuf, pcb, routetbl, UFS mount, sem, dirhash, in_multi, pfkey data, UVM amap, UVM aobj, USB, NDP, temp 128 devbuf, routetbl, ifaddr, vnodes, dirhash, ttys, exec, inodedep, UVM amap, USB, USB device, ip6_options, NDP, temp 256 devbuf, routetbl, ifaddr, sysctl, ioctlops, vnodes, shm, VM map, sem, dirhash, file desc, proc, NFS srvsock, NFS daemon, newblk, UVM amap, USB, USB device, temp 512 devbuf, pcb, ifaddr, ioctlops, mount, UFS mount, shm, dirhash, file desc, ttys, exec, UVM amap, USB device, temp 1024 devbuf, ioctlops, namecache, UFS mount, file desc, proc, ttys, exec, UVM amap, UVM aobj, crypto data, temp 2048 devbuf, ifaddr, sysctl, ioctlops, UFS mount, shm, file desc, pagedep, VM swap, UVM amap, temp 4096 devbuf, ioctlops, UFS mount, file desc, MSDOSFS mount, UVM amap, temp 8192 devbuf, NFS node, namecache, UFS quota, UFS mount, ISOFS mount, inodedep, UVM amap 16384 devbuf, indirdep, UVM amap, temp 32768 devbuf, namecache, UFS mount, VM swap, UVM amap 65536 VM swap, UVM amap 131072 UVM amap 262144 UVM amap 524288 UVM amap Memory statistics by type Type Kern Type InUse MemUse HighUse Limit Requests Limit Limit Size(s) devbuf 534 744K809K 39322K 8900 0 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768 pcb32 4K 5K 39322K175720 0 16,32,64,512 routetbl74 6K 7K 39322K 21010 0 16,32,64,128,256 ifaddr39 9K 9K 39322K 390 0 16,32,128,256,512,2048 sysctl 3 3K 3K 39322K30 0 16,256,2048 ioctlops 0 0K 4K 39322K 10720080 0 256,512,1024,2048,4096 mount10 5K 5K 39322K 110 0 512 NFS node 1 8K 8K 39322K10 0 8192 vnodes55 7K 43K 39322K 2161460 0 32,128,256 namecache 341K 41K 39322K30 0 1024,8192,32768 UFS quota 1 8K 8K 39322K10 0 8192 UFS mount4191K 91K 39322K 410 0 16,32,64,512,1024,2048,4096,8192,32768 shm 3 3K 3K 39322K 180 0 256,512,2048 VM map 3 1K 1K 39322K30 0 256 sem11 1K 1K 39322K 740 0 16,32,64,256 dirhash7514K 49K 39322K533310 0 16,32,64,128,256,512 file desc2510K 13K 39322K 29090 0 256,512,1024,2048,4096 proc19 3K 3K 39322K 190 0 32,256,1024 VFS cluster 0 0K 4K 39322K 2955160
Re: uvm_mapent_alloc: out of static map entries, check MAX_KMAPENT
On Fri, Oct 07, 2005 at 12:29:17PM -0400, Brad wrote: Now instead of your system panicing, the kernel will try to allocate more memory for additional map entries. The kernel will print ouf the usual uvm_mapent_alloc: out of static map entries but not panic. Indeed, I upgraded a system that used to panic() without raising MAX_KMAPENT and now if only prints the message without panic()ing. Also, looking at the vmstat display of systat you will see that kmapent has been added to the bottom right corner, this will show you the number of map entries currently in use by the kernel. Unfortunately, that number is hidden in a 80x24 terminal. That host currently has 1583 kmap entries. -- Frank - my stupid blog: http://00f.net L'annuaire des professionnels de la manucure et de la pedicure : http://www.manucure-pro.com
uvm_mapent_alloc: out of static map entries, check MAX_KMAPENT
To all end users experiencing the panic message as mentioned in the topic above. If you are looking for some relief from these panics then I would highly recommend trying out a -current snapshot. As of a week ago pedro@ had commited what should be a permanent fix for this problem. Now instead of your system panicing, the kernel will try to allocate more memory for additional map entries. The kernel will print ouf the usual uvm_mapent_alloc: out of static map entries but not panic. Also, looking at the vmstat display of systat you will see that kmapent has been added to the bottom right corner, this will show you the number of map entries currently in use by the kernel. So, please try this out on any systems which have experienced this panic in the past, and this affects 3.8 too, and post the results back to the list.
