Re: dynamically calculating NKPT [was: Re: huge ktr buffer]

2013-02-05 Thread Alan Cox
On 02/05/2013 09:45, m...@freebsd.org wrote: On Tue, Feb 5, 2013 at 7:14 AM, Konstantin Belousov kostik...@gmail.com wrote: On Mon, Feb 04, 2013 at 03:05:15PM -0800, Neel Natu wrote: Hi, I have a patch to dynamically calculate NKPT for amd64 kernels. This should fix the various issues that

Re: dynamically calculating NKPT [was: Re: huge ktr buffer]

2013-02-05 Thread Alan Cox
On 02/05/2013 10:13, Konstantin Belousov wrote: On Tue, Feb 05, 2013 at 07:45:24AM -0800, m...@freebsd.org wrote: On Tue, Feb 5, 2013 at 7:14 AM, Konstantin Belousov kostik...@gmail.com wrote: On Mon, Feb 04, 2013 at 03:05:15PM -0800, Neel Natu wrote: Hi, I have a patch to dynamically

Re: kmem_map auto-sizing and size dependencies

2013-01-18 Thread Alan Cox
I'll follow up with detailed answers to your questions over the weekend. For now, I will, however, point out that you've misinterpreted the tunables. In fact, they say that your kmem map can hold up to 16GB and the current used space is about 58MB. Like other things, the kmem map is auto-sized

Re: huge ktr buffer

2012-12-06 Thread Alan Cox
On 12/06/2012 09:43, Davide Italiano wrote: On Thu, Dec 6, 2012 at 4:18 PM, Andriy Gapon a...@freebsd.org wrote: So I configured a kernel with the following option: options KTR_ENTRIES=(1024UL*1024) then booted the kernel and did $ sysctl debug.ktr.clear=1 and got an insta-reboot. No

Re: Memory reserves or lack thereof

2012-11-15 Thread Alan Cox
On 11/13/2012 05:54, Konstantin Belousov wrote: On Mon, Nov 12, 2012 at 05:10:01PM -0600, Alan Cox wrote: On 11/12/2012 3:48 PM, Konstantin Belousov wrote: On Mon, Nov 12, 2012 at 01:28:02PM -0800, Sushanth Rai wrote: This patch still doesn't address the issue of M_NOWAIT calls driving

Re: Memory reserves or lack thereof

2012-11-15 Thread Alan Cox
On 11/15/2012 12:21, Konstantin Belousov wrote: On Thu, Nov 15, 2012 at 11:32:18AM -0600, Alan Cox wrote: On 11/13/2012 05:54, Konstantin Belousov wrote: On Mon, Nov 12, 2012 at 05:10:01PM -0600, Alan Cox wrote: On 11/12/2012 3:48 PM, Konstantin Belousov wrote: On Mon, Nov 12, 2012 at 01:28

Re: Memory reserves or lack thereof

2012-11-13 Thread Alan Cox
On 11/12/2012 11:35, Alan Cox wrote: On 11/12/2012 07:36, Konstantin Belousov wrote: On Sun, Nov 11, 2012 at 03:40:24PM -0600, Alan Cox wrote: On Sat, Nov 10, 2012 at 7:20 AM, Konstantin Belousov kostik...@gmail.comwrote: On Fri, Nov 09, 2012 at 07:10:04PM +, Sears, Steven wrote: I

Re: Memory reserves or lack thereof

2012-11-12 Thread Alan Cox
On 11/12/2012 07:36, Konstantin Belousov wrote: On Sun, Nov 11, 2012 at 03:40:24PM -0600, Alan Cox wrote: On Sat, Nov 10, 2012 at 7:20 AM, Konstantin Belousov kostik...@gmail.comwrote: On Fri, Nov 09, 2012 at 07:10:04PM +, Sears, Steven wrote: I have a memory subsystem design question

Re: Memory reserves or lack thereof

2012-11-12 Thread Alan Cox
On 11/12/2012 3:48 PM, Konstantin Belousov wrote: On Mon, Nov 12, 2012 at 01:28:02PM -0800, Sushanth Rai wrote: This patch still doesn't address the issue of M_NOWAIT calls driving the memory the all the way down to 2 pages, right ? It would be nice to have M_NOWAIT just do non-sleep version of

Re: Memory reserves or lack thereof

2012-11-12 Thread Alan Cox
On 11/12/2012 5:24 PM, Adrian Chadd wrote: .. wait, so what exactly would the difference be between M_NOWAIT and M_WAITOK? Whether or not the allocation can sleep until memory becomes available. ___ freebsd-hackers@freebsd.org mailing list

Re: Memory reserves or lack thereof

2012-11-11 Thread Alan Cox
On Sat, Nov 10, 2012 at 7:20 AM, Konstantin Belousov kostik...@gmail.comwrote: On Fri, Nov 09, 2012 at 07:10:04PM +, Sears, Steven wrote: I have a memory subsystem design question that I'm hoping someone can answer. I've been looking at a machine that is completely out of memory, as

Re: Threaded 6.4 code compiled under 9.0 uses a lot more memory?..

2012-11-03 Thread Alan Cox
On Wed, Oct 31, 2012 at 2:06 PM, Konstantin Belousov kostik...@gmail.comwrote: On Wed, Oct 31, 2012 at 11:52:06AM -0700, Adrian Chadd wrote: On 31 October 2012 11:20, Ian Lepore free...@damnhippie.dyndns.org wrote: I think there are some things we should be investigating about the

Re: contigmalloc() breaking Xorg

2012-07-13 Thread Alan Cox
On 07/12/2012 07:26, John Baldwin wrote: [ Adding alc@ for VM stuff, Warner for arm/mips bus dma brokenness ] When the code underlying contigmalloc() fails in its initial attempt to allocate memory and proceeds to launder and reclaim pages, it should almost certainly do as the page daemon

Re: Rtld object tasting [Was: Re: wired memory - again!]

2012-06-13 Thread Alan Cox
On Wed, Jun 13, 2012 at 2:12 PM, Konstantin Belousov kostik...@gmail.comwrote: On Wed, Jun 13, 2012 at 07:14:09AM -0600, Ian Lepore wrote: http://lists.freebsd.org/pipermail/freebsd-arm/2012-January/003288.html The map_object.c patch is step in the almost right direction, I wanted to remove

Re: superpages and kmem on amd64

2012-05-20 Thread Alan Cox
On Sun, May 20, 2012 at 2:01 AM, Marko Zec z...@fer.hr wrote: Hi all, I'm playing with an algorithm which makes use of large contiguous blocks of kernel memory (ranging from 1M to 1G in size), so it would be nice if those could be somehow forcibly mapped to superpages. I was hoping that the

Re: superpages and kmem on amd64

2012-05-20 Thread Alan Cox
On 05/20/2012 09:43, Marko Zec wrote: On Sunday 20 May 2012 09:25:59 Alan Cox wrote: On Sun, May 20, 2012 at 2:01 AM, Marko Zecz...@fer.hr wrote: Hi all, I'm playing with an algorithm which makes use of large contiguous blocks of kernel memory (ranging from 1M to 1G in size), so it would

Re: superpages and kmem on amd64

2012-05-20 Thread Alan Cox
On 05/20/2012 17:48, Marko Zec wrote: On Sunday 20 May 2012 19:34:26 Alan Cox wrote: ... In any case, I wish to be certain that a particular kmem virtual address range is mapped to superpages - how can I enforce that at malloc time, and / or find out later if I really got my kmem mapped

Re: problems with mmap() and disk caching

2012-04-29 Thread Alan Cox
On 04/11/2012 01:07, Andrey Zonov wrote: On 10.04.2012 20:19, Alan Cox wrote: On 04/09/2012 10:26, John Baldwin wrote: On Thursday, April 05, 2012 11:54:31 am Alan Cox wrote: On 04/04/2012 02:17, Konstantin Belousov wrote: On Tue, Apr 03, 2012 at 11:02:53PM +0400, Andrey Zonov wrote: Hi, I

Re: mlockall() on freebsd 7.2 + amd64 returns EAGAIN

2012-04-21 Thread Alan Cox
On 04/13/2012 16:45, Konstantin Belousov wrote: On Fri, Apr 13, 2012 at 11:37:44AM -0700, Sushanth Rai wrote: I've attached the simple program that creates 5 threads. Following is the o/p of /proc/pid/map when this program is running. Note that I modified sys/fs/procfs/procfs_map.c to print

Re: Corrupted pmap pm_vlist - pmap_remove_pte()

2012-04-17 Thread Alan Cox
On 4/17/2012 4:48 AM, Konstantin Belousov wrote: On Mon, Apr 16, 2012 at 03:08:25PM -0400, Ewart Tempest wrote: In FreeBSD 6.*, we have been seeing crashes in pmap_remove_pages() that only seem to occur in scaling scenarios: 2564#ifdef PMAP_REMOVE_PAGES_CURPROC_ONLY 2565

Re: problems with mmap() and disk caching

2012-04-10 Thread Alan Cox
On 04/09/2012 10:26, John Baldwin wrote: On Thursday, April 05, 2012 11:54:31 am Alan Cox wrote: On 04/04/2012 02:17, Konstantin Belousov wrote: On Tue, Apr 03, 2012 at 11:02:53PM +0400, Andrey Zonov wrote: Hi, I open the file, then call mmap() on the whole file and get pointer, then I work

Re: problems with mmap() and disk caching

2012-04-06 Thread Alan Cox
On 04/04/2012 02:17, Konstantin Belousov wrote: On Tue, Apr 03, 2012 at 11:02:53PM +0400, Andrey Zonov wrote: Hi, I open the file, then call mmap() on the whole file and get pointer, then I work with this pointer. I expect that page should be only once touched to get it into the memory (disk

Re: problems with mmap() and disk caching

2012-04-06 Thread Alan Cox
On 04/06/2012 03:38, Konstantin Belousov wrote: On Thu, Apr 05, 2012 at 01:25:49PM -0500, Alan Cox wrote: On 04/05/2012 12:31, Konstantin Belousov wrote: On Thu, Apr 05, 2012 at 10:54:31AM -0500, Alan Cox wrote: On 04/04/2012 02:17, Konstantin Belousov wrote: On Tue, Apr 03, 2012 at 11:02

Re: problems with mmap() and disk caching

2012-04-05 Thread Alan Cox
On 04/04/2012 02:17, Konstantin Belousov wrote: On Tue, Apr 03, 2012 at 11:02:53PM +0400, Andrey Zonov wrote: Hi, I open the file, then call mmap() on the whole file and get pointer, then I work with this pointer. I expect that page should be only once touched to get it into the memory (disk

Re: problems with mmap() and disk caching

2012-04-05 Thread Alan Cox
On 04/04/2012 04:36, Andrey Zonov wrote: On 04.04.2012 11:17, Konstantin Belousov wrote: Calling madvise(MADV_RANDOM) fixes the issue, because the code to deactivate/cache the pages is turned off. On the other hand, it also turns of read-ahead for faulting, and the first loop becomes eternally

Re: problems with mmap() and disk caching

2012-04-05 Thread Alan Cox
On 04/05/2012 12:31, Konstantin Belousov wrote: On Thu, Apr 05, 2012 at 10:54:31AM -0500, Alan Cox wrote: On 04/04/2012 02:17, Konstantin Belousov wrote: On Tue, Apr 03, 2012 at 11:02:53PM +0400, Andrey Zonov wrote: Hi, I open the file, then call mmap() on the whole file and get pointer

Re: Please help me diagnose this crazy VMWare/FreeBSD 8.x crash

2012-03-29 Thread Alan Cox
On Thu, Mar 29, 2012 at 11:27 AM, Mark Felder f...@feld.me wrote: On Thu, 29 Mar 2012 10:55:36 -0500, Hans Petter Selasky hsela...@c2i.net wrote: It almost sounds like the lost interrupt issue I've seen with USB EHCI devices, though disk I/O should have a retry timeout? What does wmstat

Re: mmap performance and memory use

2011-10-28 Thread Alan Cox
On 10/26/2011 06:23, Svatopluk Kraus wrote: Hi, well, I'm working on new port (arm11 mpcore) and pmap_enter_object() is what I'm debugging rigth now. And I did not find any way in userland how to force kernel to call pmap_enter_object() which makes SUPERPAGE mapping without promotion. I tried

Re: mmap performance and memory use

2011-10-25 Thread Alan Cox
On 10/10/2011 4:28 PM, Wojciech Puchar wrote: Notice that vm.pmap.pde.promotions increased by 31. This means that 31 superpage mappings were created by promotion from small page mappings. thank you. i looked at .mappings as it seemed logical for me that is shows total. In contrast,

Re: mmap performance and memory use

2011-10-12 Thread Alan Cox
On 10/11/2011 12:36, Mark Tinguely wrote: On 10/11/2011 11:12 AM, Alan Cox wrote: On 10/10/2011 16:28, Wojciech Puchar wrote: is it possible to force VM subsystem to operate on superpages when possible - i mean swapping in 2MB chunks? Currently, no. For some applications, like the Sun

Re: mmap performance and memory use

2011-10-11 Thread Alan Cox
On 10/10/2011 16:28, Wojciech Puchar wrote: Notice that vm.pmap.pde.promotions increased by 31. This means that 31 superpage mappings were created by promotion from small page mappings. thank you. i looked at .mappings as it seemed logical for me that is shows total. In contrast,

Re: mmap performance and memory use

2011-10-10 Thread Alan Cox
On 10/07/2011 12:23, Wojciech Puchar wrote: You are correct about the page table page. However, a superpage mapping consumes a single PV entry, in place of 512 or 1024 PV entries. This winds up saving about three physical pages worth of memory for every superpage mapping. does it actually

Re: mmap performance and memory use

2011-10-07 Thread Alan Cox
On Thu, Oct 6, 2011 at 11:01 AM, Kostik Belousov kostik...@gmail.comwrote: On Thu, Oct 06, 2011 at 04:41:45PM +0200, Wojciech Puchar wrote: i have few questions. 1) suppose i map 1TB of address space as anonymous and touch just one page. how much memory is used to manage this? I am not

Re: Memory allocation in kernel -- what to use in which situation? What is the best for page-sized allocations?

2011-10-02 Thread Alan Cox
On Sun, Oct 2, 2011 at 1:21 PM, m...@freebsd.org wrote: 2011/10/2 Lev Serebryakov l...@freebsd.org: Hello, Freebsd-hackers. Here are several memory-allocation mechanisms in the kernel. The two I'm aware of is MALLOC_DEFINE()/malloc()/free() and uma_* (zone(9)). As far as I

Re: UMA large allocations issues

2011-07-23 Thread Alan Cox
On Fri, Jul 22, 2011 at 9:07 PM, Davide Italiano davide.itali...@gmail.comwrote: Hi. I'm a student and some time ago I started investigating a bit about the performance/fragmentation issue of large allocations within the UMA allocator. Benckmarks showed up that this problems of performances

Re: SMP question w.r.t. reading kernel variables

2011-04-20 Thread Alan Cox
On Wed, Apr 20, 2011 at 7:42 AM, Rick Macklem rmack...@uoguelph.ca wrote: On Tue, Apr 19, 2011 at 12:00:29PM +, freebsd-hackers-requ...@freebsd.org wrote: Subject: Re: SMP question w.r.t. reading kernel variables To: Rick Macklem rmack...@uoguelph.ca Cc:

Re: Question about Reverse Mappings in FreeBSD.

2011-03-20 Thread Alan Cox
On Fri, Mar 18, 2011 at 7:30 PM, J L dimitar9...@gmail.com wrote: I read an article about Reverse Mappings technique in memory management part. It improves a lot from Linux 2.4 to 2.6. I am wondering is FreeBSD also have this feature? Which source files should I go to find these? I want to

Re: Analyzing wired memory?

2011-02-08 Thread Alan Cox
On Tue, Feb 8, 2011 at 6:20 AM, Ivan Voras ivo...@freebsd.org wrote: Is it possible to track by some way what kernel system, process or thread has wired memory? (including data exists but needs code to extract it) No. I'd like to analyze a system where there is a lot of memory wired but

Re: Analyzing wired memory?

2011-02-08 Thread Alan Cox
On 2/8/2011 12:27 PM, Robert Watson wrote: On Tue, 8 Feb 2011, Alan Cox wrote: On Tue, Feb 8, 2011 at 6:20 AM, Ivan Voras ivo...@freebsd.org wrote: Is it possible to track by some way what kernel system, process or thread has wired memory? (including data exists but needs code to extract

Re: [rfc] allow to boot with = 256GB physmem

2011-01-21 Thread Alan Cox
On Fri, Jan 21, 2011 at 11:44 AM, John Baldwin j...@freebsd.org wrote: On Friday, January 21, 2011 11:09:10 am Sergey Kandaurov wrote: Hello. Some time ago I faced with a problem booting with 400GB physmem. The problem is that vm.max_proc_mmap type overflows with such high value, and

Re: [rfc] allow to boot with = 256GB physmem

2011-01-21 Thread Alan Cox
On Fri, Jan 21, 2011 at 2:58 PM, Alan Cox alan.l@gmail.com wrote: On Fri, Jan 21, 2011 at 11:44 AM, John Baldwin j...@freebsd.org wrote: On Friday, January 21, 2011 11:09:10 am Sergey Kandaurov wrote: Hello. Some time ago I faced with a problem booting with 400GB physmem

Re: vm_map_findspace space search?

2010-12-16 Thread Alan Cox
On Wed, Dec 15, 2010 at 7:37 PM, Venkatesh Srinivas vsrini...@dragonflybsd.org wrote: Hi, In svn r133636, there was a commit to convert the linear search in vm_map_findspace() to use the vm_map splay tree. Just curious, were there any discussions about that particular change? Any

Re: i386 pmap_zero_page() late sched_pin()?

2010-12-12 Thread Alan Cox
On Sun, Dec 12, 2010 at 10:40 AM, Venkatesh Srinivas vsrini...@dragonflybsd.org wrote: Hi, In the i386 pmap's pmap_zero_page(), there is a fragment... sysmaps = sysmaps_pcpu[PCPU_GET(cpuid)]; mtx_lock(sysmaps-lock); * sched_pin(); /*map the page we mean to zero

Re: vm_object ref_count question

2010-10-28 Thread Alan Cox
On Thu, Oct 28, 2010 at 2:48 AM, Eknath Venkataramani eknath.i...@gmail.com wrote: ref_count is defined inside struct vm_object. and it is incremented everytime the object is referenced How is the page reference logged then? rather in which variable? There is no per-page reference. There

Re: Contiguous physical memory

2010-10-27 Thread Alan Cox
On Wed, Oct 27, 2010 at 2:17 PM, Dr. Baud drb...@yahoo.com wrote: Can anyone suggest a method/formula to monitor contiguous physical memory allocations such that one could predict when contigmalloc(), make that bus_dmamem_alloc might fail? From the command line you can obtain this

Re: page table fault, which should map kernel virtual address space

2010-10-03 Thread Alan Cox
On Thu, Sep 30, 2010 at 6:28 AM, Svatopluk Kraus onw...@gmail.com wrote: On Tue, Sep 21, 2010 at 7:38 PM, Alan Cox alan.l@gmail.com wrote: On Mon, Sep 20, 2010 at 9:32 AM, Svatopluk Kraus onw...@gmail.com wrote: Beyond 'kernel_map', some submaps of 'kernel_map' (buffer_map, pager_map

Re: Examining the VM splay tree effectiveness

2010-09-30 Thread Alan Cox
On Thu, Sep 30, 2010 at 12:37 PM, Andre Oppermann an...@freebsd.org wrote: On 30.09.2010 18:37, Andre Oppermann wrote: Just for the kick of it I decided to take a closer look at the use of splay trees (inherited from Mach if I read the history correctly) in the FreeBSD VM system suspecting

Re: zfs + uma

2010-09-21 Thread Alan Cox
On Tue, Sep 21, 2010 at 1:39 AM, Jeff Roberson jrober...@jroberson.netwrote: On Tue, 21 Sep 2010, Andriy Gapon wrote: on 19/09/2010 11:42 Andriy Gapon said the following: on 19/09/2010 11:27 Jeff Roberson said the following: I don't like this because even with very large buffers you can

Re: page table fault, which should map kernel virtual address space

2010-09-21 Thread Alan Cox
On Mon, Sep 20, 2010 at 9:32 AM, Svatopluk Kraus onw...@gmail.com wrote: Hallo, this is about 'NKPT' definition, 'kernel_map' submaps, and 'vm_map_findspace' function. Variable 'kernel_map' is used to manage kernel virtual address space. When 'vm_map_findspace' function deals with

Re: UMA allocations from a specific physical range

2010-09-06 Thread Alan Cox
m...@freebsd.org wrote: [snip] IIRC the memory from vm_phys_alloc_contig() can be released like any other page; the interface should just be fetching a specific page. How far off is the page wire count? I'm assuming it's hitting the assert that it's 1? I think vm_page_free() is the right

Re: Intel TurboBoost in practice

2010-07-27 Thread Alan Cox
On Mon, Jul 26, 2010 at 9:11 AM, Alexander Motin m...@freebsd.org wrote: Robert Watson wrote: On Sun, 25 Jul 2010, Alexander Motin wrote: The numbers that you are showing doesn't show much difference. Have you tried buildworld? If you mean relative difference -- as I have told, it's

Re: Intel TurboBoost in practice

2010-07-24 Thread Alan Cox
2010/7/24 Alexander Motin m...@freebsd.org Hi. I've make small observations of Intel TurboBoost technology under FreeBSD. This technology allows Intel Core i5/i7 CPUs to rise frequency of some cores if other cores are idle and power/thermal conditions permit. CPU core counted as idle, if it

Re: disk I/O, VFS hirunningspace

2010-07-16 Thread Alan Cox
Peter Jeremy wrote: Regarding vfs.lorunningspace and vfs.hirunningspace... On 2010-Jul-15 13:52:43 -0500, Alan Coxalan.l@gmail.com wrote: Keep in mind that we still run on some fairly small systems with limited I/O capabilities, e.g., a typical arm platform. More generally, with the

Re: disk I/O, VFS hirunningspace

2010-07-15 Thread Alan Cox
On Thu, Jul 15, 2010 at 8:01 AM, Ivan Voras ivo...@freebsd.org wrote: On 07/14/10 18:27, Jerry Toung wrote: On Wed, Jul 14, 2010 at 12:04 AM, Gary Jennejohn gljennj...@googlemail.comwrote: Rather than commenting out the code try setting the sysctl vfs.hirunningspace to various

Re: Strange problem with 8-stable, VMWare vSphere 4 AMD CPUs (unexpected shutdowns)

2010-02-11 Thread Alan Cox
On Thu, Feb 11, 2010 at 7:13 AM, John Baldwin j...@freebsd.org wrote: On Wednesday 10 February 2010 1:38:37 pm Ivan Voras wrote: On 10 February 2010 19:35, Andriy Gapon a...@icyb.net.ua wrote: on 10/02/2010 20:26 Ivan Voras said the following: On 10 February 2010 19:10, Andriy Gapon

Re: Superpages on amd64 FreeBSD 7.2-STABLE

2009-12-12 Thread Alan Cox
, or if squid does something particularly horrible to its memory when it forks to cause this, but I wanted to ask about it on the list in case somebody who understands it better might know whats going on. :-) I talked with Alan Cox some about this off-list and there is a case that can

Re: mmap(2) with MAP_ANON honouring offset although it shouldn't

2009-11-04 Thread Alan Cox
Ed Schouten wrote: * John Baldwin j...@freebsd.org wrote: Note that the spec doesn't cover MAP_ANON at all FWIW. Yes. I've noticed Linux also uses MAP_ANONYMOUS instead of MAP_ANON. They do provide MAP_ANON for compatibility, if I remember correctly. For what it's worth, I

Re: mmap(2) with MAP_ANON honouring offset although it shouldn't

2009-11-04 Thread Alan Cox
Ed Schouten wrote: * Alan Cox a...@cs.rice.edu wrote: For what it's worth, I believe that Solaris does the exact opposite. They provide MAP_ANONYMOUS for compatibility. It seems like a good idea for us to do the same. Something like this? Index: mman.h

Re: mmap(2) with MAP_ANON honouring offset although it shouldn't

2009-10-21 Thread Alan Cox
On Wed, Oct 21, 2009 at 10:51 AM, Alexander Best alexbes...@math.uni-muenster.de wrote: although the mmap(2) manual states in section MAP_ANON: The offset argument is ignored. this doesn't seem to be true. running printf(%p\n, mmap((void*)0x1000, 0x1000, PROT_NONE, MAP_ANON, -1,

Re: mmap/munmap with zero length

2009-07-14 Thread Alan Cox
John Baldwin wrote: On Monday 13 July 2009 3:33:51 pm Tijl Coosemans wrote: On Monday 13 July 2009 20:28:08 John Baldwin wrote: On Sunday 05 July 2009 3:32:25 am Alexander Best wrote: so mmap differs from the POSIX recommendation right. the malloc.conf option seems more like a

Re: Problem with vm.pmap.shpgperproc and vm.pmap.pv_entry_max

2009-07-05 Thread Alan Cox
On Fri, Jul 3, 2009 at 8:18 AM, c0re dumped ez.c...@gmail.com wrote: So, I never had problem with this server, but recently it starts to giv me the following messages *every* minute : Jul 3 10:04:00 squid kernel: Approaching the limit on PV entries, consider increasing either the

Re: large pages (amd64)

2009-07-03 Thread Alan Cox
Robert Watson wrote: On Tue, 30 Jun 2009, Mel Flynn wrote: It looks like sys/kern/kern_proc.c could call mincore around the loop at line 1601 (rev 194498), but I know nothing about the vm subsystem to know the implications or locking involved. There's still 16 bytes of spare to consume, in

Re: large pages (amd64)

2009-06-30 Thread Alan Cox
Mel Flynn wrote: On Sunday 28 June 2009 15:41:49 Alan Cox wrote: Wojciech Puchar wrote: how can i check how much (or maybe - what processes) 2MB pages are actually allocated? I'm afraid that you can't with great precision. For a given program execution, on an otherwise

Re: large pages (amd64)

2009-06-28 Thread Alan Cox
On Sun, Jun 28, 2009 at 12:36 PM, Wojciech Puchar woj...@wojtek.tensor.gdynia.pl wrote: i enabled vm.pmap.pg_ps_enabled: 1 could you please explain what exactly this values means? because i don't understand why promotions-demotions!=mappings mappings is not what you seem to think it is.

Re: large pages (amd64)

2009-06-28 Thread Alan Cox
Wojciech Puchar wrote: other question - tried enabling it on my i386 laptop (256 megs ram), always mappings==0, while promitionsdemotions0. The default starting address for executables on i386 is not aligned to a 2/4MB page boundary. Hence, mappings are much less likely to

Re: large pages (amd64)

2009-06-28 Thread Alan Cox
On Sun, Jun 28, 2009 at 7:14 PM, Nathanael Hoyle nho...@hoyletech.comwrote: Wojciech Puchar wrote: i enabled vm.pmap.pg_ps_enabled: 1 could you please explain what exactly this values means? because i don't understand why promotions-demotions!=mappings vm.pmap.pde.promotions: 2703

Re: large pages (amd64)

2009-06-28 Thread Alan Cox
Nathanael Hoyle wrote: [snip] Having been corrected by both you and Joerg (thank you!), I went back to re-verify my understanding. It appears that while I was slightly mixing PAE in with PSE, PSE support for 4MB pages was introduced 'silently' with the Pentium, and documented first with the

Re: Increasing KVM on amd64

2008-09-10 Thread Alan Cox
Artem Belevich wrote: Alan, Thanks a lot for the patch. I've applied it to RELENG_7 and it seems to work great - make -j8 buildworld succeeds, linux emulation seems to work well enough to run linux-sun-jdk14 binaries, ZFS ARC size is bigger, too. So far I didn't see any ZFS-related KVM

Re: Increasing KVM on amd64

2008-06-08 Thread Alan Cox
Tz-Huan Huang wrote: On Sun, Jun 8, 2008 at 7:59 AM, Alan Cox [EMAIL PROTECTED] wrote: You can download a patch from http://www.cs.rice.edu/~alc/amd64_kvm_6GB.patch that increases amd64's kernel virtual address space to 6GB. This patch also increases the default for the kmem map to almost

Increasing KVM on amd64

2008-06-07 Thread Alan Cox
You can download a patch from http://www.cs.rice.edu/~alc/amd64_kvm_6GB.patch that increases amd64's kernel virtual address space to 6GB. This patch also increases the default for the kmem map to almost 2GB. I believe that kernel loadable modules still work. However, I suspect that

Re: Increasing KVM on amd64

2008-06-07 Thread Alan Cox
Kostik Belousov wrote: On Sat, Jun 07, 2008 at 06:59:35PM -0500, Alan Cox wrote: You can download a patch from http://www.cs.rice.edu/~alc/amd64_kvm_6GB.patch that increases amd64's kernel virtual address space to 6GB. This patch also increases the default for the kmem map to almost 2GB

Re: possibly missed wakeup in swap_pager_getpages()

2007-03-09 Thread Alan Cox
Divacky Roman wrote: hi [snip] is my analysis correct? if so, can the race be mitigated by moving the flag setting (hence also the locking) after the msleep()? No. When the wakeup is performed, the VPO_SWAPINPROG flag is also cleared. Both operations occur in the I/O completion

Re: Possible problems with mmap/munmap on FreeBSD ...

2005-03-30 Thread Alan Cox
On Tue, Mar 29, 2005 at 09:18:32PM -0500, David Schultz wrote: On Tue, Mar 29, 2005, Richard Sharpe wrote: Hi, I am having some problems with the tdb package on FreeBSD 4.6.2 and 4.10. One of the things the above package does is: mmap the tdb file to a region of memory

Re: contigmalloc(9) rewrite

2004-06-18 Thread Alan Cox
() into a separate vm_page_alloc_contig() and mapping function as per feedback from Alan Cox and Hiten Pandya. The operation is still the same except for now being able to see memory allocated with it in your vmstat(8) -m output. The patch is still at the same location, and requires sysctl

Re: Update: Debox sendfile modifications

2003-11-05 Thread Alan Cox
On Wed, Nov 05, 2003 at 01:25:43AM -0500, Vivek Pai wrote: Mike Silbersack wrote: On Tue, 4 Nov 2003, Vivek Pai wrote: The one other aspect of this is that sf_bufs mappings are maintained for a configurable amount of time, reducing the number of TLB ops. You can specify the parameter for how

Re: [Fwd: Some mmap observations compared to Linux 2.6/OpenBSD]

2003-10-26 Thread Alan Cox
On Wed, Oct 22, 2003 at 01:34:01PM +1000, Q wrote: It has been suggested that I should direct this question to the VM system guru's. Your comments on this would be appreciated. An address space is represented by a data structure that we call a vm_map. A vm_map is, in the abstact, an ordered

Re: why doesn't aio use at_exit(9)?

2000-12-01 Thread Alan Cox
On Fri, Dec 01, 2000 at 02:02:58AM -0800, Alfred Perlstein wrote: why doesn't aio use at_exit(9) instead of requiring an explicit call in kern_exit.c for aio_rundown? There's no reason that I'm aware of. Unless you're in a hurry, I'll add that change to a cleanup patch that I have in the

Re: why doesn't aio use at_exit(9)?

2000-12-01 Thread Alan Cox
On Fri, Dec 01, 2000 at 12:08:48PM -0800, Alfred Perlstein wrote: * Alan Cox [EMAIL PROTECTED] [001201 11:56] wrote: On Fri, Dec 01, 2000 at 02:02:58AM -0800, Alfred Perlstein wrote: why doesn't aio use at_exit(9) instead of requiring an explicit call in kern_exit.c for aio_rundown

Re: page coloring

2000-11-23 Thread Alan Cox
On Thu, Nov 23, 2000 at 12:48:09PM -0800, Mike Smith wrote: Isn't the page coloring algoritm in _vm_page_list_find totally bogus? No, it's not. The comment is, however, misplaced. It describes the behavior of an inline function in vm_page.h, and not the function it precedes.

Re: minherit(2) API

2000-07-24 Thread Alan Cox
I think I can change it in NetBSD -- anyone willing to do the necessary changes in FreeBSD and OpenBSD? I'll do it. Alan To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message

Re: clearing pages in the idle loop

2000-07-19 Thread Alan Cox
Last year, I tried to reproduce some of the claims/results in this paper on FreeBSD/x86 and couldn't. I also tried limiting the idle loop to clearing pages of one particular color at a time. That way, the cache lines replaced by the second page you clear are the cache lines holding the first

Re: Panic in pmap_remove_pages on 4.0-current

1999-08-28 Thread Alan Cox
This exact problem came up last month. pmap_remove_pages is tripping over a corrupted page table entry (pte). Unfortunately, by the time the panic occurs, pmap_remove_pages has overwritten the corrupted pte with zero. Earlier this month, I added a KASSERT to detect this problem (and panic)

Re: Panic in pmap_remove_pages on 4.0-current

1999-08-28 Thread Alan Cox
This exact problem came up last month. pmap_remove_pages is tripping over a corrupted page table entry (pte). Unfortunately, by the time the panic occurs, pmap_remove_pages has overwritten the corrupted pte with zero. Earlier this month, I added a KASSERT to detect this problem (and panic)

Re: patch for behavior changes and madvise MADV_DONTNEED

1999-07-30 Thread Alan Cox
On Fri, Jul 30, 1999 at 09:51:35AM -0700, Matthew Dillon wrote: I'm not sure I see how MADV_FREE could slow performance down unless, perhaps, it is the overhead of the system call itself. e.g. if malloc is calling it on a page-by-page basis and not implementing any hysteresis.

Re: patch for behavior changes and madvise MADV_DONTNEED

1999-07-30 Thread Alan Cox
On Fri, Jul 30, 1999 at 09:51:35AM -0700, Matthew Dillon wrote: I'm not sure I see how MADV_FREE could slow performance down unless, perhaps, it is the overhead of the system call itself. e.g. if malloc is calling it on a page-by-page basis and not implementing any hysteresis.

Re: patch for behavior changes and madvise MADV_DONTNEED

1999-07-29 Thread Alan Cox
Your new behavior flag isn't known by vm_map_simply_entry, so map simplification could drop the behavior setting on the floor. I would prefer that the behavior is folded into eflags. Overall, I agree with the direction. Behavior is correctly a map attribute rather than an object attribute.

Re: Improving the Unix API

1999-06-28 Thread Alan Cox
As far as sysctl goes, FreeBSD deprecates the use of numbers for OIDs and has a string-based mechanism for exploring the sysctl tree. So we are actually both going the same way. Linus with /proc/sys and his official dislike of sysctl (Oh well I think sysctl using number spaces is the right idea

Re: Improving the Unix API

1999-06-28 Thread Alan Cox
As far as I know, only FreeBSD has a string-based sysctl implementation. Nod. Something which always confused me about Linux' procfs - what have all these kernel variables got to do with process state? We used to have a kernfs which was intended for this kind of thing but it rotted after

Re: 3.2 Freeze date

1999-05-10 Thread Alan Cox
On Mon, May 10, 1999 at 07:33:28PM -0700, Matthew Dillon wrote: My main interest are the NFS/TCP fixes, which Alan now has a -stable patch for. But it's already the tenth so if it goes in now the source will then have to be reviewed by more gurus ( post-commit ). The NFS/TCP