Re: panic: UMA: Increase vm.boot_pages on Dell R920 r279210
On Sat, May 2, 2015 at 10:25 PM, Adrian Chadd wrote:
> hi,
>
> Hm, should we be upping this limit automatically? Can we get cpu
> counts or memory amount early enough in boot to have a hope of
> auto-tuning?
>
> 64 seems low, 1024 seems high as a default. :)
>
What is it that's exhausting the boot_pages? I'm semi-guessing it's the
number of vm radix tree nodes needed for the TiB of memory. The only thing
I'm aware of (allow for ignorance here) that consumes boot_pages and scales
with the cpu count is the uma zone used for uma cache objects, but on amd64
this zone only needs 640 + cpus * 128 bytes, or about 4 pages for 120
cpus. vm radix nodes are 144 bytes each on amd64, and by my
back-of-the-envelope calculations (using traces of non-vm-radix boot_page
use from another amd64 system), 64 boot_pages would be exhausted after
about 1000 vm radix nodes were allocated. It would be interesting to know
how many boot_pages were actually required for this particular system.
In any event, since startup_alloc() is designed to exhaust all the
boot_pages before switching to the normal allocators, it doesn't seem
necessarily harmful to err on the high side either in bumping up the static
default or introducing an auto-tuned value (provided the excess is not so
perversely large that startup_alloc() isn't able to make use of an
embarrassment of pages due to zone creation timing and usage patterns). We
know the number of cpus at the time boot_pages is put to use, but I don't
think we know how much memory there is (and even less sure that even if we
did, we'd really want to try to estimate things the vm radix tree size in a
generic way).
Something like a default of boot_pages = max(64, 32 + k * cpus) might be
sufficient for k = 4 or 8 (gathering some data points would give a clue
here), and palatable since it is at a minimum the current value that's been
in use, and at the other end approaches a modest commitment of 16 or 32 KiB
per cpu in the worst case (unused and unreclaimed boot_pages with high cpu
count).
-Patrick
>
>
> On 24 March 2015 at 13:00, Keith White wrote:
> > On Tue, 24 Mar 2015, Rui Paulo wrote:
> >
> >> On Mar 24, 2015, at 04:19, kwh...@site.uottawa.ca wrote:
> >>>
> >>>
> >>> I'm using /boot/loader.conf. Is there another place I should be doing
> >>> this?
> >>
> >>
> >> No, that's correct, but apparently there's a problem: the RDTUN sysctl
> is
> >> not picked up early enough. Can you try this patch? I haven't really
> >> tested it. :-)
> >>
> >> diff --git a/sys/vm/vm_page.c b/sys/vm/vm_page.c
> >> index 79665ba..a764788 100644
> >> --- a/sys/vm/vm_page.c
> >> +++ b/sys/vm/vm_page.c
> >> @@ -134,8 +134,9 @@ long first_page;
> >> int vm_page_zero_count;
> >>
> >> static int boot_pages = UMA_BOOT_PAGES;
> >> -SYSCTL_INT(_vm, OID_AUTO, boot_pages, CTLFLAG_RDTUN, &boot_pages, 0,
> >> - "number of pages allocated for bootstrapping the VM system");
> >> +SYSCTL_INT(_vm, OID_AUTO, boot_pages, CTLFLAG_RDTUN | CTLFLAG_NOFETCH,
> >> +&boot_pages, 0,
> >> +"number of pages allocated for bootstrapping the VM system");
> >>
> >> static int pa_tryrelock_restart;
> >> SYSCTL_INT(_vm, OID_AUTO, tryrelock_restart, CTLFLAG_RD,
> >> @@ -349,6 +350,7 @@ vm_page_startup(vm_offset_t vaddr)
> >> * Allocate memory for use when boot strapping the kernel memory
> >> * allocator.
> >> */
> >> + TUNABLE_INT_FETCH("vm.boot_pages", &boot_pages);
> >>new_end = end - (boot_pages * UMA_SLAB_SIZE);
> >>new_end = trunc_page(new_end);
> >>mapped = pmap_map(&vaddr, new_end, end,
> >> @@ -443,7 +445,7 @@ vm_page_startup(vm_offset_t vaddr)
> >>
> >>
> >> --
> >> Rui Paulo
> >
> >
> > Patch tried. Success!
> >
> > I now get this after setting vm.boot_pages=1024 in /boot/loader.conf:
> >
> > Booting...
> > GDB: no debug ports present
> > KDB: debugger backends: ddb
> > KDB: current backend: ddb
> > Copyright (c) 1992-2015 The FreeBSD Project.
> > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
> > The Regents of the University of California. All rights reserved.
> > FreeBSD is a registered trademark of The FreeBSD Foundation.
> > FreeBSD 11.0-CURRENT #1: Tue Mar 24 13:44:48 UTC 2015
> > root@:/usr/obj/usr/src/sys/GENERIC amd64
> > FreeBSD clang version 3.5.1 (tags/RELEASE_351/final 225668) 20150115
> > WARNING: WITNESS option enabled, expect reduced performance.
> > UMA startup boot_pages: 1024
> > ...
> >
> > And can start all 120 processors.
> >
> > Thanks!
> >
> > ...keith
> > --
> > Keith White, genie.uottawa.ca engineering.uottawa.ca
> > kwh...@uottawa.ca [+1 613 562 5800 x6681]
> > ___
> > freebsd-current@freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-current
> > To unsubscribe, send any mail to "
> freebsd-current-unsubscr...@freebsd.org"
> ___
> freebsd-current@freebsd.org mailing list
> http://l
Re: panic: UMA: Increase vm.boot_pages on Dell R920 r279210
hi,
Hm, should we be upping this limit automatically? Can we get cpu
counts or memory amount early enough in boot to have a hope of
auto-tuning?
64 seems low, 1024 seems high as a default. :)
-adrian
On 24 March 2015 at 13:00, Keith White wrote:
> On Tue, 24 Mar 2015, Rui Paulo wrote:
>
>> On Mar 24, 2015, at 04:19, kwh...@site.uottawa.ca wrote:
>>>
>>>
>>> I'm using /boot/loader.conf. Is there another place I should be doing
>>> this?
>>
>>
>> No, that's correct, but apparently there's a problem: the RDTUN sysctl is
>> not picked up early enough. Can you try this patch? I haven't really
>> tested it. :-)
>>
>> diff --git a/sys/vm/vm_page.c b/sys/vm/vm_page.c
>> index 79665ba..a764788 100644
>> --- a/sys/vm/vm_page.c
>> +++ b/sys/vm/vm_page.c
>> @@ -134,8 +134,9 @@ long first_page;
>> int vm_page_zero_count;
>>
>> static int boot_pages = UMA_BOOT_PAGES;
>> -SYSCTL_INT(_vm, OID_AUTO, boot_pages, CTLFLAG_RDTUN, &boot_pages, 0,
>> - "number of pages allocated for bootstrapping the VM system");
>> +SYSCTL_INT(_vm, OID_AUTO, boot_pages, CTLFLAG_RDTUN | CTLFLAG_NOFETCH,
>> +&boot_pages, 0,
>> +"number of pages allocated for bootstrapping the VM system");
>>
>> static int pa_tryrelock_restart;
>> SYSCTL_INT(_vm, OID_AUTO, tryrelock_restart, CTLFLAG_RD,
>> @@ -349,6 +350,7 @@ vm_page_startup(vm_offset_t vaddr)
>> * Allocate memory for use when boot strapping the kernel memory
>> * allocator.
>> */
>> + TUNABLE_INT_FETCH("vm.boot_pages", &boot_pages);
>>new_end = end - (boot_pages * UMA_SLAB_SIZE);
>>new_end = trunc_page(new_end);
>>mapped = pmap_map(&vaddr, new_end, end,
>> @@ -443,7 +445,7 @@ vm_page_startup(vm_offset_t vaddr)
>>
>>
>> --
>> Rui Paulo
>
>
> Patch tried. Success!
>
> I now get this after setting vm.boot_pages=1024 in /boot/loader.conf:
>
> Booting...
> GDB: no debug ports present
> KDB: debugger backends: ddb
> KDB: current backend: ddb
> Copyright (c) 1992-2015 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
> The Regents of the University of California. All rights reserved.
> FreeBSD is a registered trademark of The FreeBSD Foundation.
> FreeBSD 11.0-CURRENT #1: Tue Mar 24 13:44:48 UTC 2015
> root@:/usr/obj/usr/src/sys/GENERIC amd64
> FreeBSD clang version 3.5.1 (tags/RELEASE_351/final 225668) 20150115
> WARNING: WITNESS option enabled, expect reduced performance.
> UMA startup boot_pages: 1024
> ...
>
> And can start all 120 processors.
>
> Thanks!
>
> ...keith
> --
> Keith White, genie.uottawa.ca engineering.uottawa.ca
> kwh...@uottawa.ca [+1 613 562 5800 x6681]
> ___
> freebsd-current@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic: UMA: Increase vm.boot_pages on Dell R920 r279210
On Mar 24, 2015, at 04:19, kwh...@site.uottawa.ca wrote:
>
> I'm using /boot/loader.conf. Is there another place I should be doing this?
No, that's correct, but apparently there's a problem: the RDTUN sysctl is not
picked up early enough. Can you try this patch? I haven't really tested it.
:-)
diff --git a/sys/vm/vm_page.c b/sys/vm/vm_page.c
index 79665ba..a764788 100644
--- a/sys/vm/vm_page.c
+++ b/sys/vm/vm_page.c
@@ -134,8 +134,9 @@ long first_page;
int vm_page_zero_count;
static int boot_pages = UMA_BOOT_PAGES;
-SYSCTL_INT(_vm, OID_AUTO, boot_pages, CTLFLAG_RDTUN, &boot_pages, 0,
- "number of pages allocated for bootstrapping the VM system");
+SYSCTL_INT(_vm, OID_AUTO, boot_pages, CTLFLAG_RDTUN | CTLFLAG_NOFETCH,
+&boot_pages, 0,
+"number of pages allocated for bootstrapping the VM system");
static int pa_tryrelock_restart;
SYSCTL_INT(_vm, OID_AUTO, tryrelock_restart, CTLFLAG_RD,
@@ -349,6 +350,7 @@ vm_page_startup(vm_offset_t vaddr)
* Allocate memory for use when boot strapping the kernel memory
* allocator.
*/
+ TUNABLE_INT_FETCH("vm.boot_pages", &boot_pages);
new_end = end - (boot_pages * UMA_SLAB_SIZE);
new_end = trunc_page(new_end);
mapped = pmap_map(&vaddr, new_end, end,
@@ -443,7 +445,7 @@ vm_page_startup(vm_offset_t vaddr)
--
Rui Paulo
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic: UMA: Increase vm.boot_pages on Dell R920 r279210
On Tue, 24 Mar 2015, Rui Paulo wrote:
On Mar 24, 2015, at 04:19, kwh...@site.uottawa.ca wrote:
I'm using /boot/loader.conf. Is there another place I should be doing this?
No, that's correct, but apparently there's a problem: the RDTUN sysctl is not
picked up early enough. Can you try this patch? I haven't really tested it.
:-)
diff --git a/sys/vm/vm_page.c b/sys/vm/vm_page.c
index 79665ba..a764788 100644
--- a/sys/vm/vm_page.c
+++ b/sys/vm/vm_page.c
@@ -134,8 +134,9 @@ long first_page;
int vm_page_zero_count;
static int boot_pages = UMA_BOOT_PAGES;
-SYSCTL_INT(_vm, OID_AUTO, boot_pages, CTLFLAG_RDTUN, &boot_pages, 0,
- "number of pages allocated for bootstrapping the VM system");
+SYSCTL_INT(_vm, OID_AUTO, boot_pages, CTLFLAG_RDTUN | CTLFLAG_NOFETCH,
+&boot_pages, 0,
+"number of pages allocated for bootstrapping the VM system");
static int pa_tryrelock_restart;
SYSCTL_INT(_vm, OID_AUTO, tryrelock_restart, CTLFLAG_RD,
@@ -349,6 +350,7 @@ vm_page_startup(vm_offset_t vaddr)
* Allocate memory for use when boot strapping the kernel memory
* allocator.
*/
+ TUNABLE_INT_FETCH("vm.boot_pages", &boot_pages);
new_end = end - (boot_pages * UMA_SLAB_SIZE);
new_end = trunc_page(new_end);
mapped = pmap_map(&vaddr, new_end, end,
@@ -443,7 +445,7 @@ vm_page_startup(vm_offset_t vaddr)
--
Rui Paulo
Patch tried. Success!
I now get this after setting vm.boot_pages=1024 in /boot/loader.conf:
Booting...
GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
Copyright (c) 1992-2015 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.0-CURRENT #1: Tue Mar 24 13:44:48 UTC 2015
root@:/usr/obj/usr/src/sys/GENERIC amd64
FreeBSD clang version 3.5.1 (tags/RELEASE_351/final 225668) 20150115
WARNING: WITNESS option enabled, expect reduced performance.
UMA startup boot_pages: 1024
...
And can start all 120 processors.
Thanks!
...keith
--
Keith White, genie.uottawa.ca engineering.uottawa.ca
kwh...@uottawa.ca [+1 613 562 5800 x6681]
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: panic: UMA: Increase vm.boot_pages on Dell R920 r279210
> On Mar 19, 2015, at 07:34, Keith White wrote: >> I tried the suggestion "Increase vm.boot_pages" but am unsure how >> much. I tried doubling to 128, and even excessive(?) values like >> 102400; but got the same panic. > > How are you trying to change the value? Can you build a custom kernel and > confirm the value in uma_startup()? > > -- > Rui Paulo > - I'm using /boot/loader.conf. Is there another place I should be doing this? vfs.mountroot.timeout="10" boot_multicons="YES" boot_serial="YES" comconsole_speed="115200" console="comconsole,vidconsole" vm.boot_pages=1024 hw.mfi.mrsas_enable=1 - With a custom kernel, the value printed is 64 not my expected "1024": ... Copyright (c) 1992-2015 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-CURRENT #0: Tue Mar 24 04:55:47 UTC 2015 root@:/usr/obj/usr/src/sys/GENERIC amd64 FreeBSD clang version 3.5.1 (tags/RELEASE_351/final 225668) 20150115 WARNING: WITNESS option enabled, expect reduced performance. === UMA startup boot_pages: 64 === VT: running with driver "vga". CPU: Intel(R) Xeon(R) CPU E7-4870 v2 @ 2.30GHz (2300.05-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306e7 Family=0x6 Model=0x3e Stepping=7 Features=0xbfebfbff Features2=0x7fbee3ff AMD Features=0x2c100800 AMD Features2=0x1 Structured Extended Features=0x281 XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 1099478073344 (1048544 MB) avail memory = 1069175943168 (1019645 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 80 CPUs FreeBSD/SMP: 4 package(s) x 10 core(s) x 2 SMT threads ... -- After boot, the value is as expected: # sysctl vm.boot_pages vm.boot_pages: 1024 ...keith ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
