On Sat, May 2, 2015 at 10:25 PM, Adrian Chadd <adr...@freebsd.org> wrote:
> hi, > > Hm, should we be upping this limit automatically? Can we get cpu > counts or memory amount early enough in boot to have a hope of > auto-tuning? > > 64 seems low, 1024 seems high as a default. :) > What is it that's exhausting the boot_pages? I'm semi-guessing it's the number of vm radix tree nodes needed for the TiB of memory. The only thing I'm aware of (allow for ignorance here) that consumes boot_pages and scales with the cpu count is the uma zone used for uma cache objects, but on amd64 this zone only needs 640 + cpus * 128 bytes, or about 4 pages for 120 cpus. vm radix nodes are 144 bytes each on amd64, and by my back-of-the-envelope calculations (using traces of non-vm-radix boot_page use from another amd64 system), 64 boot_pages would be exhausted after about 1000 vm radix nodes were allocated. It would be interesting to know how many boot_pages were actually required for this particular system. In any event, since startup_alloc() is designed to exhaust all the boot_pages before switching to the normal allocators, it doesn't seem necessarily harmful to err on the high side either in bumping up the static default or introducing an auto-tuned value (provided the excess is not so perversely large that startup_alloc() isn't able to make use of an embarrassment of pages due to zone creation timing and usage patterns). We know the number of cpus at the time boot_pages is put to use, but I don't think we know how much memory there is (and even less sure that even if we did, we'd really want to try to estimate things the vm radix tree size in a generic way). Something like a default of boot_pages = max(64, 32 + k * cpus) might be sufficient for k = 4 or 8 (gathering some data points would give a clue here), and palatable since it is at a minimum the current value that's been in use, and at the other end approaches a modest commitment of 16 or 32 KiB per cpu in the worst case (unused and unreclaimed boot_pages with high cpu count). -Patrick > > > On 24 March 2015 at 13:00, Keith White <kwh...@site.uottawa.ca> wrote: > > On Tue, 24 Mar 2015, Rui Paulo wrote: > > > >> On Mar 24, 2015, at 04:19, kwh...@site.uottawa.ca wrote: > >>> > >>> > >>> I'm using /boot/loader.conf. Is there another place I should be doing > >>> this? > >> > >> > >> No, that's correct, but apparently there's a problem: the RDTUN sysctl > is > >> not picked up early enough. Can you try this patch? I haven't really > >> tested it. :-) > >> > >> diff --git a/sys/vm/vm_page.c b/sys/vm/vm_page.c > >> index 79665ba..a764788 100644 > >> --- a/sys/vm/vm_page.c > >> +++ b/sys/vm/vm_page.c > >> @@ -134,8 +134,9 @@ long first_page; > >> int vm_page_zero_count; > >> > >> static int boot_pages = UMA_BOOT_PAGES; > >> -SYSCTL_INT(_vm, OID_AUTO, boot_pages, CTLFLAG_RDTUN, &boot_pages, 0, > >> - "number of pages allocated for bootstrapping the VM system"); > >> +SYSCTL_INT(_vm, OID_AUTO, boot_pages, CTLFLAG_RDTUN | CTLFLAG_NOFETCH, > >> + &boot_pages, 0, > >> + "number of pages allocated for bootstrapping the VM system"); > >> > >> static int pa_tryrelock_restart; > >> SYSCTL_INT(_vm, OID_AUTO, tryrelock_restart, CTLFLAG_RD, > >> @@ -349,6 +350,7 @@ vm_page_startup(vm_offset_t vaddr) > >> * Allocate memory for use when boot strapping the kernel memory > >> * allocator. > >> */ > >> + TUNABLE_INT_FETCH("vm.boot_pages", &boot_pages); > >> new_end = end - (boot_pages * UMA_SLAB_SIZE); > >> new_end = trunc_page(new_end); > >> mapped = pmap_map(&vaddr, new_end, end, > >> @@ -443,7 +445,7 @@ vm_page_startup(vm_offset_t vaddr) > >> > >> > >> -- > >> Rui Paulo > > > > > > Patch tried. Success! > > > > I now get this after setting vm.boot_pages=1024 in /boot/loader.conf: > > > > Booting... > > GDB: no debug ports present > > KDB: debugger backends: ddb > > KDB: current backend: ddb > > Copyright (c) 1992-2015 The FreeBSD Project. > > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > > The Regents of the University of California. All rights reserved. > > FreeBSD is a registered trademark of The FreeBSD Foundation. > > FreeBSD 11.0-CURRENT #1: Tue Mar 24 13:44:48 UTC 2015 > > root@:/usr/obj/usr/src/sys/GENERIC amd64 > > FreeBSD clang version 3.5.1 (tags/RELEASE_351/final 225668) 20150115 > > WARNING: WITNESS option enabled, expect reduced performance. > > UMA startup boot_pages: 1024 > > ... > > > > And can start all 120 processors. > > > > Thanks! > > > > ...keith > > -- > > Keith White, genie.uottawa.ca engineering.uottawa.ca > > kwh...@uottawa.ca [+1 613 562 5800 x6681] > > _______________________________________________ > > firstname.lastname@example.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to " > freebsd-current-unsubscr...@freebsd.org" > _______________________________________________ > email@example.com mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org" > _______________________________________________ firstname.lastname@example.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"