On Tue, Jan 8, 2013 at 3:12 AM, Andre Oppermann <an...@freebsd.org> wrote: > On 07.01.2013 20:32, Alan Cox wrote: >> >> On 01/07/2013 12:47, Oleksandr Tymoshenko wrote: >>> >>> On 12/27/2012 6:46 PM, Oleksandr Tymoshenko wrote: >>>> >>>> On 12/18/2012 1:59 AM, Alan Cox wrote: >>>>> >>>>> On 12/17/2012 23:40, Oleksandr Tymoshenko wrote: >>>>>> >>>>>> On 2012-12-08, at 1:21 PM, Alan Cox <a...@rice.edu> wrote: >>>>>> >>>>>>> On 12/08/2012 14:32, Andre Oppermann wrote: >>>>>> >>>>>> .. skipped .. >>>>>> >>>>>>>> The trouble seems to come from NSFBUFS which is (512 + maxusers * >>>>>>>> 16) >>>>>>>> resulting in a kernel map of (512 + 400 * 16) * PAGE_SIZE = >>>>>>>> 27MB. This >>>>>>>> seem to be pushing it with the smaller ARM kmap layout. >>>>>>>> >>>>>>>> Does it boot and run when you set the tunable kern.ipc.nsfbufs=3500? >>>>>>>> >>>>>>>> ARM does have a direct map mode as well which doesn't require the >>>>>>>> allocation >>>>>>>> of sfbufs. I'm not sure which other problems that approach has. >>>>>>>> >>>>>>> Only a few (3?) platforms use it. It reduces the size of the user >>>>>>> address space, and translation between physical addresses and >>>>>>> direct map >>>>>>> addresses is not computationally trivial as it is on other >>>>>>> architectures, e.g., amd64, ia64. However, it does try to use large >>>>>>> page mappings. >>>>>>> >>>>>>> >>>>>>>> Hopefully alc@ (added to cc) can answer that and also why the >>>>>>>> kmap of >>>>>>>> 27MB >>>>>>>> manages to wrench the ARM kernel. >>>>>>>> >>>>>>> Arm does not define caps on either the buffer map size (param.h) >>>>>>> or the >>>>>>> kmem map size (vmparam.h). It would probably make sense to copy >>>>>>> these >>>>>>> definitions from i386. >>>>>> >>>>>> Adding caps didn't help. I did some digging and found out that >>>>>> although address range >>>>>> 0xc0000000 .. 0xffffffff is indeed valid for ARM in general actual >>>>>> KVA space varies for >>>>>> each specific hardware platform. This "real" KVA is defined by >>>>>> <virtual_avail, virtual_end> >>>>>> pair and ifI use them instead of <VM_MIN_KERNEL_ADDRESS, >>>>>> VM_MAX_KERNEL_ADDRESS> >>>>>> in init_param2 function my pandaboard successfully boots. Since >>>>>> former pair is used for defining >>>>>> kernel_map boundaries I believe it should be used for auto tuning >>>>>> as well. >>>>> >>>>> >>>>> That makes sense. However, "virtual_avail" isn't the start of the >>>>> kernel address space. The kernel map always starts at >>>>> VM_MIN_KERNEL_ADDRESS. (See kmem_init().) "virtual_avail" represents >>>>> the next unallocated virtual address in the kernel address space at an >>>>> early point in initialization. "virtual_avail" and "virtual_end" >>>>> aren't >>>>> used after that, or outside the VM system. Please use >>>>> vm_map_min(kernel_map) and vm_map_max(kernel_map) instead. >>>> >>>> >>>> I checked: kernel_map is not available (NULL) at this point. So we >>>> can't use it to >>>> determine real KVA size. Closest thing we can get is >>>> virtual_avail/virtual_end pair. >>>> >>>> Andre, could you approve attached patch for commit or suggest better >>>> solution? >>> >>> >>> Any update on this one? Can I proceed with commit? >>> >> >> Sorry, I've been away from my e-mail since the 30th, and I'm now in the >> process of getting caught up. Give me a day or so to look at this.
I see an issue with commit on MIPS XLP platform as well. With 16 GB physical memory, the ncallout is calculated to be 538881 (since it is based on maxfiles - which is now based on the physical memory). Due to this, the callwheel allocation per cpu is 16MB (callwheelsize is 1MB). And on a 32 CPU machine, the total allocation for callouts comes to 32*16MB = 512MB. I have worked around this issue for now by increasing VM_KMEM_SIZE_MAX (which is 200MB now) - but I think a better fix is needed for this. JC. _______________________________________________ svn-src-all@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"