kern.maxswzone causing serious problems
I'm starting to upgrade a set of servers from FreeBSD 11.0-STABLE #0 r308356 (OK, rather old) to FreeBSD 11.1-STABLE #0 r331152 without much success. I'm getting the occasionally discussed kern.maxswzone problems but I really do need to configure that swap space. On an upgraded server I'm getting (line continuation added for readability): warning: total configured swap (5242880 pages) \ exceeds maximum recommended amount (112288 pages). warning: increase kern.maxswzone or reduce amount of swap. warning: total configured swap (10485760 pages) \ exceeds maximum recommended amount (112288 pages). warning: increase kern.maxswzone or reduce amount of swap. The value previously used was not working. I ended up temporarily cutting swap in half as well to get rid of the error. This is only a symptom of a greater problem. This machine has for a long time run multiple VM which total 17GB memory on a server with 8 GB of physical memory. Normally things are fine becuase most of these servers are idle most of the time and all but a small memory footprint can be paged out. I have for years had 40 GB swap, with 20GB each on two spindles (at least going back to FreeBSD 9) on drives identical partitioning with mirrored partitions. \begin{talesofwoe} % ignore if busy When just trying to install base system software using md devices on at a time (no VM running yet) I had been getting "killed: out of swap space" messages. I got rid of this by reducing the size of the VM root partitions (most VM run zfs on another partition with not much on root so this was OK). Next problem was running the VM. I worked around this by reducing some VM memory (-m in bhyve) by half. I can get the whole set of VM to boot but now installing additional software (which involved just moving files with scp and running tar) doesn't work. At this point I have 9 GB of VM running on 8 GB physical memory and still couldn't even install software. Once the maxswzone message went away, so did all of these problems except my VMs now have half as much RAM each. Now each of the VM are reporting swap problems. And this is just upgrading one server to 11.1. \end{talesofwoe} This is a major regression for FreeBSD 11.1. The same value used in FreeBSD 11.0 should just work or there should be some documentation on how to set this (preferably the error message). If nothing else, some advice on converting a kern.maxswzone value from 11.0 to a working value for 11.1 would be nice. The entry in the loader(8) man page is not very helpful. btw- Reporting swap size in MB or KB in the error message would be helpful. In addition to pages would be fine. Mentioning what the highest value kern.maxswzone could be set to would also be helpful. Changing "warning: increase kern.maxswzone" to "warning: increase kern.maxswzone to %d" would be very helpful. \begin{naiveananlysis} The magic (or mess, depending on perspective) is mostly in "void swap_pager_swap_init(void)" in the file vm/swap_pager.c between lines 484 and 563 (in current which is same as stable/11 in this function). The diffs from known working r308356 to current show a diff at "@@ -538,21 +518,25 @@" which has swpctrie_zone and swblk_zone computed based on maxswzone, then runs uma_zone_reserve_kva based on maxswzone and potentially reduces it. In the older code a "Swap zone entries reduced" message would be produced if uma_zone_reserve_kva cut back (which is moved and the message changed a bit). But I didn't see this message so uma_zone_reserve_kva ran fine the first time without reducing "n". In the new code swap_maxpages and swzone are then set. swapon_check_swzone gives the warning but does nothing as far as I can tell other than two printfs. It doesn't appear to do any harm to have too much swap and ignore this warning (you just can't use it). There is a multiplier by SWAP_META_PAGES which is defined to be PCTRIE_COUNT which in sys/pctrie.h is defined as (1 << PCTRIE_WIDTH) and PCTRIE_WIDTH is 4 or 3 depending on __LP64__. swap_pager_swap_init calculates swap_maxpages but swapon_check_swzone doesn't use it, calculating local variable maxpages (the same way) instead. Since it seems that npages / SWAP_META_PAGES is related to what you'd want to set kern.maxswzone to. If so, a better set of printf might at least give better information. Note that VM_SWZONE_SIZE_MAX defaults to (276 * 128 * 1024) which would seem to be 128K * SWAP_META_PAGES * PAGE_SIZE = 8GB or 16GB depending on if __LP64__ is defined, but that is for i386 only. There is no definition of VM_SWZONE_SIZE_MAX for amd64 unless it picks this up from i386 which apparently it does. One problem is the conditional for using maxswzone only allows the swzone size to be reduced and not increased. Those people frobbing kern.maxswzone (including me for a time) were hopelessly wasting their time. Based on my math the old max swap was about 4 * available RAM. Naive or horribly naive? I haven't tried
Re: KBI unexpexted change in stable/11 ?
On Wed, Mar 28, 2018 at 11:16:49AM -0500, Eric van Gyzen wrote: > On 03/28/2018 10:35, Slawa Olhovchenkov wrote: > > On Wed, Mar 28, 2018 at 10:29:10AM -0500, Eric van Gyzen wrote: > > > >> On 03/28/2018 08:09, Slawa Olhovchenkov wrote: > >>> I am upgrade system to latest -STABLE and now see kernel crash: > >>> > >>> - loading virtualbox modules build on 11.1-RELEASE-p6 > >>> - loading nvidia module build on 11.1-RELEASE-p6 and start xdm > >>> > >>> Is this expected? I am mean about loading modules builded on > >>> 11.1-RELEASE on any 11.1-STABLE. > >> > >> This is not expected. Can you bisect to find the stable/11 commit that > >> broke this? > >> > >> If you can roll back to 11.1-RELEASE, you could probably just > >> buildkernel and installkernel from various points along stable/11. That > >> would save a lot of time by avoiding buildworld. > > > > r325665 is previos point and is good. > > r331615 crashed. > > Can I use some script for bisect? > > I'm not aware of a script for this. The only tool I've used is "git > bisect", which is very handy if you're already familiar with git. You may want to try devel/p5-App-SVN-Bisect. Never used it, so no idea if it's functional or helpful, just found it doing a quick search Regards, Gary ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
HEADS-UP: 11.2-RELEASE cycle starting in one month
[Several vendor contacts in BCC.] For those that did not notice the website update adding the 11.2-RELEASE cycle schedule, I apologize for sending this to public lists this late. I thought I had done this, but apparently not. The 11.2-RELEASE release cycle will begin April 20, 2018. The schedule for the release cycle is: stable/11 slush: April 20, 2018 stable/11 freeze:May 4, 2018 BETA1 build starts: May 11, 2018 doc/ tree slush: May 12, 2018 BETA2 build starts: May 18, 2018 BETA3 build starts: May 25, 2018 [*] doc/ tree tag: May 26, 2018 releng/11.2 branch: June 1, 2018 RC1 build starts:June 1, 2018 stable/11 thaw: June 3, 2018 RC2 build starts:June 8, 2018 RC3 build starts:June 15, 2018 [*] RELEASE build starts:June 22, 2018 RELEASE announcement:June 27, 2018 11.1-RELEASE EoL:September 30, 2018 Branch EoL: September 30, 2021 [*] - If needed The schedule is also available on the Project website at: https://www.freebsd.org/releases/11.2R/schedule.html As a reminder, the code "slush" is not a hard freeze on the tree and does not require formal approval from re@, however we ask that you refrain from adding new features, and focus on fixing existing issues. The code "freeze" is the point at which all commits to stable/11 will require explicit approval from re@. If you are aware of any problems in stable/11, please let re@ know as soon as possible. Additionally, if vendors have driver updates that are planned to be included in 11.2-RELEASE, please ensure the drivers are updated in head (preferably as soon as possible, to catch any issues), and merged to stable/11 following the minimum 3-day MFC timeframe. Thank you. Glen On behalf of: re@ signature.asc Description: PGP signature
Re: KBI unexpexted change in stable/11 ?
On 03/28/2018 10:35, Slawa Olhovchenkov wrote: > On Wed, Mar 28, 2018 at 10:29:10AM -0500, Eric van Gyzen wrote: > >> On 03/28/2018 08:09, Slawa Olhovchenkov wrote: >>> I am upgrade system to latest -STABLE and now see kernel crash: >>> >>> - loading virtualbox modules build on 11.1-RELEASE-p6 >>> - loading nvidia module build on 11.1-RELEASE-p6 and start xdm >>> >>> Is this expected? I am mean about loading modules builded on >>> 11.1-RELEASE on any 11.1-STABLE. >> >> This is not expected. Can you bisect to find the stable/11 commit that >> broke this? >> >> If you can roll back to 11.1-RELEASE, you could probably just >> buildkernel and installkernel from various points along stable/11. That >> would save a lot of time by avoiding buildworld. > > r325665 is previos point and is good. > r331615 crashed. > Can I use some script for bisect? I'm not aware of a script for this. The only tool I've used is "git bisect", which is very handy if you're already familiar with git. Eric ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: KBI unexpexted change in stable/11 ?
On Wed, 28 Mar 2018, at 16:49, Stefan Esser wrote: > No, its not confusing at all, if you think about it ... ;-) The bit for me which was confusing is that the PORTS_MODULES reference, as Gregory correctly pointed out, is in the man page for make.conf and not src.conf. So one would think what the man page referenced would go into the file that pertaining to that manpage. The reason I've always had things like KERNCONF=KERNCONF1 GENERIC and PORTS_MODULES in src.conf is because I figured "if it gets built when base gets built then it must go in src.conf". So when Gregory mentioned it goes in make.conf, I thought perhaps I was doing it all wrong, or in an outdated way. I was surprised finding that in these contexts it didn't matter as long as one remembers as you say += ?= in make.conf -- John tech-li...@zyxst.net ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: KBI unexpexted change in stable/11 ?
On Wed, 2018-03-28 at 16:30 +0100, John wrote: > On Wed, 28 Mar 2018, at 15:20, Greg Byshenk wrote: > > > > On Wed, Mar 28, 2018 at 03:11:50PM +0100, tech-lists wrote: > > > > > > On 28/03/2018 14:39, Gregory Byshenk wrote: > > > > > > > > You can do this manually, or by adding a PORTS_MODULES line to > > > > /etc/make.conf. This will rebuild the listed modules from ports > > > > when you build a new kernel. > > > Are you sure it's in /etc/make.conf and not /etc/src.conf? > > No. But it is in the man page for make.conf and not src.conf. > yeah. That is... confusing! at least to me, because [I thought it would be] > src.conf that's consulted when src is built. So I ran a couple of tests and > found that it would work in either file HOWEVER if one ports module statement > was in src.conf and another, different ports module statement was in > make.conf, > that the one in src.conf would get built but the one in make.conf would not. > > how confusing is that. This is on 11.1-stable. > Make.conf is for building anything that uses make(1) and the build infrastructure in /usr/share/mk. Src.conf is for building the freebsd sources only. According to build(7), the freebsd sources include /usr/src, /usr/doc, and /usr/ports. So settings in both make.conf and src.conf affect ports builds. The difference appears when you use make to build your own software that isn't part of freebsd. For that, make.conf settings are applied, but not src.conf. At least, that's what the docs say. When I look at the .mk files in /usr/share/mk, it appears to me that src.conf gets used even for non-freebsd-source builds. When both files exist, variables set in src.conf will override same- named variables set in make.conf. Using VAR += value in both files will accumulate values into VAR rather than replacing them. -- Ian ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: KBI unexpexted change in stable/11 ?
Am 28.03.18 um 17:30 schrieb John: > On Wed, 28 Mar 2018, at 15:20, Greg Byshenk wrote: >> On Wed, Mar 28, 2018 at 03:11:50PM +0100, tech-lists wrote: >>> On 28/03/2018 14:39, Gregory Byshenk wrote: You can do this manually, or by adding a PORTS_MODULES line to /etc/make.conf. This will rebuild the listed modules from ports when you build a new kernel. >>> >>> Are you sure it's in /etc/make.conf and not /etc/src.conf? >> >> No. But it is in the man page for make.conf and not src.conf. > > yeah. That is... confusing! at least to me, because [I thought it would be] > src.conf that's consulted when src is built. So I ran a couple of tests and > found that it would work in either file HOWEVER if one ports module statement > was in src.conf and another, different ports module statement was in > make.conf, > that the one in src.conf would get built but the one in make.conf would not. > > how confusing is that. This is on 11.1-stable. No, its not confusing at all, if you think about it ... ;-) /etc/make.conf is included in any case, but kernel and world builds will include /etc/src.conf thereafter (and overwrite earlier settings read from make.conf, unless ?= is used to preserve the earlier value, or += to append the new values). Regards, STefan ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: KBI unexpexted change in stable/11 ?
On Wed, Mar 28, 2018 at 10:29:10AM -0500, Eric van Gyzen wrote: > On 03/28/2018 08:09, Slawa Olhovchenkov wrote: > > I am upgrade system to latest -STABLE and now see kernel crash: > > > > - loading virtualbox modules build on 11.1-RELEASE-p6 > > - loading nvidia module build on 11.1-RELEASE-p6 and start xdm > > > > Is this expected? I am mean about loading modules builded on > > 11.1-RELEASE on any 11.1-STABLE. > > This is not expected. Can you bisect to find the stable/11 commit that > broke this? > > If you can roll back to 11.1-RELEASE, you could probably just > buildkernel and installkernel from various points along stable/11. That > would save a lot of time by avoiding buildworld. r325665 is previos point and is good. r331615 crashed. Can I use some script for bisect? ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: KBI unexpexted change in stable/11 ?
On Wed, 28 Mar 2018, at 15:20, Greg Byshenk wrote: > On Wed, Mar 28, 2018 at 03:11:50PM +0100, tech-lists wrote: > > On 28/03/2018 14:39, Gregory Byshenk wrote: > > > You can do this manually, or by adding a PORTS_MODULES line to > > > /etc/make.conf. This will rebuild the listed modules from ports > > > when you build a new kernel. > > > > Are you sure it's in /etc/make.conf and not /etc/src.conf? > > No. But it is in the man page for make.conf and not src.conf. yeah. That is... confusing! at least to me, because [I thought it would be] src.conf that's consulted when src is built. So I ran a couple of tests and found that it would work in either file HOWEVER if one ports module statement was in src.conf and another, different ports module statement was in make.conf, that the one in src.conf would get built but the one in make.conf would not. how confusing is that. This is on 11.1-stable. -- John tech-li...@zyxst.net ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: KBI unexpexted change in stable/11 ?
On 03/28/2018 08:09, Slawa Olhovchenkov wrote: > I am upgrade system to latest -STABLE and now see kernel crash: > > - loading virtualbox modules build on 11.1-RELEASE-p6 > - loading nvidia module build on 11.1-RELEASE-p6 and start xdm > > Is this expected? I am mean about loading modules builded on > 11.1-RELEASE on any 11.1-STABLE. This is not expected. Can you bisect to find the stable/11 commit that broke this? If you can roll back to 11.1-RELEASE, you could probably just buildkernel and installkernel from various points along stable/11. That would save a lot of time by avoiding buildworld. Eric ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: KBI unexpexted change in stable/11 ?
On Wed, Mar 28, 2018 at 05:13:48PM +0200, Gregory Byshenk wrote: > On Wed, Mar 28, 2018 at 05:35:51PM +0300, Slawa Olhovchenkov wrote: > > On Wed, Mar 28, 2018 at 03:39:46PM +0200, Gregory Byshenk wrote: > > > > > > Did you rebuild your virtualbox and nvidia modules for your new > > > kernel? If you build a new kernel, then you need to rebuild any > > > modules that were installed from ports for the new version. > > > > Only for -CURRENT, not for -STABLE: > > > > https://lists.freebsd.org/pipermail/svn-src-all/2018-March/159649.html > > > > John Baldwin: "In theory we try to not break existing kernel > > modules on a stable branch. That is, one should be able to kldload an > > if_iwn.ko built on 11.0 on a 11-stable kernel." > > I understand that this is true for standard kernel modules, but > I have frequently run into problems loading _ports_ modules on > a new kernel - including in STABLE. This is general _kernel_ rule, for loading all modules, mostly for third-party modules. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: KBI unexpexted change in stable/11 ?
On Wed, Mar 28, 2018 at 05:35:51PM +0300, Slawa Olhovchenkov wrote: > On Wed, Mar 28, 2018 at 03:39:46PM +0200, Gregory Byshenk wrote: > > > > Did you rebuild your virtualbox and nvidia modules for your new > > kernel? If you build a new kernel, then you need to rebuild any > > modules that were installed from ports for the new version. > > Only for -CURRENT, not for -STABLE: > > https://lists.freebsd.org/pipermail/svn-src-all/2018-March/159649.html > > John Baldwin: "In theory we try to not break existing kernel > modules on a stable branch. That is, one should be able to kldload an > if_iwn.ko built on 11.0 on a 11-stable kernel." I understand that this is true for standard kernel modules, but I have frequently run into problems loading _ports_ modules on a new kernel - including in STABLE. -- gregory byshenk - gbysh...@byshenk.net - Leiden, NL ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: KBI unexpexted change in stable/11 ?
On Wed, Mar 28, 2018 at 03:39:46PM +0200, Gregory Byshenk wrote: > On Wed, Mar 28, 2018 at 04:09:04PM +0300, Slawa Olhovchenkov wrote: > > I am upgrade system to latest -STABLE and now see kernel crash: > > > > - loading virtualbox modules build on 11.1-RELEASE-p6 > > - loading nvidia module build on 11.1-RELEASE-p6 and start xdm > > > > Is this expected? I am mean about loading modules builded on > > 11.1-RELEASE on any 11.1-STABLE. > > Did you rebuild your virtualbox and nvidia modules for your new > kernel? If you build a new kernel, then you need to rebuild any > modules that were installed from ports for the new version. Only for -CURRENT, not for -STABLE: https://lists.freebsd.org/pipermail/svn-src-all/2018-March/159649.html John Baldwin: "In theory we try to not break existing kernel modules on a stable branch. That is, one should be able to kldload an if_iwn.ko built on 11.0 on a 11-stable kernel." ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: KBI unexpexted change in stable/11 ?
On Wed, Mar 28, 2018 at 03:11:50PM +0100, tech-lists wrote: > On 28/03/2018 14:39, Gregory Byshenk wrote: > > You can do this manually, or by adding a PORTS_MODULES line to > > /etc/make.conf. This will rebuild the listed modules from ports > > when you build a new kernel. > > Are you sure it's in /etc/make.conf and not /etc/src.conf? No. But it is in the man page for make.conf and not src.conf. -- gregory byshenk - gbysh...@byshenk.net - Leiden, NL ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: KBI unexpexted change in stable/11 ?
On 28/03/2018 14:39, Gregory Byshenk wrote: > You can do this manually, or by adding a PORTS_MODULES line to > /etc/make.conf. This will rebuild the listed modules from ports > when you build a new kernel. Hi, Are you sure it's in /etc/make.conf and not /etc/src.conf? -- J. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: KBI unexpexted change in stable/11 ?
On Wed, Mar 28, 2018 at 04:09:04PM +0300, Slawa Olhovchenkov wrote: > I am upgrade system to latest -STABLE and now see kernel crash: > > - loading virtualbox modules build on 11.1-RELEASE-p6 > - loading nvidia module build on 11.1-RELEASE-p6 and start xdm > > Is this expected? I am mean about loading modules builded on > 11.1-RELEASE on any 11.1-STABLE. Did you rebuild your virtualbox and nvidia modules for your new kernel? If you build a new kernel, then you need to rebuild any modules that were installed from ports for the new version. You can do this manually, or by adding a PORTS_MODULES line to /etc/make.conf. This will rebuild the listed modules from ports when you build a new kernel. I think this will do what you need (double-check that these are the right port names): PORTS_MODULES= emulators/virtualbox-ose-kmod x11/nvidia-driver -gregory -- gregory byshenk - gbysh...@byshenk.net - Leiden, NL ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: KBI unexpexted change in stable/11 ?
> On Mar 28, 2018, at 9:09 AM, Slawa Olhovchenkovwrote: > > I am upgrade system to latest -STABLE and now see kernel crash: > > - loading virtualbox modules build on 11.1-RELEASE-p6 > - loading nvidia module build on 11.1-RELEASE-p6 and start xdm > > Is this expected? I am mean about loading modules builded on > 11.1-RELEASE on any 11.1-STABLE. I had a similar issue and it caused a kernel coredump I tracked the virtualbox issue to something in the vbox network module . Rebuilding it fixes it but I am not sure what broke . I have a kernel core dump if anyone wants a peak. --- Mark Saad | nones...@longcount.org > ___ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
KBI unexpexted change in stable/11 ?
I am upgrade system to latest -STABLE and now see kernel crash: - loading virtualbox modules build on 11.1-RELEASE-p6 - loading nvidia module build on 11.1-RELEASE-p6 and start xdm Is this expected? I am mean about loading modules builded on 11.1-RELEASE on any 11.1-STABLE. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"