Re: [ast-users] Thank you for the grep builtin!

2013-09-13 Thread Lionel Cons
On 18 June 2013 10:20, Glenn Fowler g...@research.att.com wrote: you showed ast grep strace output but not gnu grep gnu /usr/bin/grep does read of varying chunk sizes on my redhat linux for NFS files the chunk sizes were around 32Ki I added some test options to the SFIO_OPTIONS env var

[ast-users] Documenting SFIO_OPTIONS in ksh93(1) and grep(1)?

2013-09-13 Thread Lionel Cons
Glenn, could you promote SFIO_OPTIONS to the official ksh93(1) and grep(1) manual pages, please? Lionel -- Forwarded message -- From: Glenn Fowler g...@research.att.com Date: 18 June 2013 10:20 Subject: Re: Thank you for the grep builtin! To: g...@research.att.com,

Re: [ast-users] Thank you for the grep builtin!

2013-09-13 Thread Glenn Fowler
On Fri, 13 Sep 2013 17:14:03 +0200 Lionel Cons wrote: On 18 June 2013 10:20, Glenn Fowler g...@research.att.com wrote: you showed ast grep strace output but not gnu grep gnu /usr/bin/grep does read of varying chunk sizes on my redhat linux for NFS files the chunk sizes were around 32Ki

Re: [ast-users] Thank you for the grep builtin!

2013-09-13 Thread ольга крыжановская
Glenn, shared mmap() mapping do not have any impact on fork() performance, at least on VM architectures who can share pages (this is common practice since at least SystemV, and no modern Unix or Linux exists which does not do copy-on-write, but more on that below) The pages are not even touched,

Re: [ast-users] Thank you for the grep builtin!

2013-09-13 Thread Glenn Fowler
On Fri, 13 Sep 2013 23:14:22 +0200 =?KOI8-R?B?z8zYx8Egy9LZ1sHOz9fTy8HR?= wrote: Glenn, shared mmap() mapping do not have any impact on fork() performance, at least on VM architectures who can share pages (this is common practice since at least SystemV, and no modern Unix or Linux exists which

Re: [ast-users] Thank you for the grep builtin!

2013-09-13 Thread ольга крыжановская
No, this is not copy on write, this is check-what-to-do-on-access-when-not-mapped. The short explanation is, that the fork() is not the time when an action in the VM system will happen, its the time of the first access to a page, which is not mapped yet, in the current process, when an action will

Re: [ast-users] Thank you for the grep builtin!

2013-09-13 Thread Glenn Fowler
we're getting close again we're not interested in the pages but the metadata for the pages this may be based on incorrect assumptions ... 1Gib mapped and 8Kib page size = 131072 entries for address-to-page lookup at fork() time the parent process has that 131072 entry table in hand what does the

Re: [ast-users] Thank you for the grep builtin!

2013-09-13 Thread Haller, John H (John)
With any luck, the systems with large allocations will be using transparent huge pages for systems which support it, and up to 2M is just a single page table entry. Unfortunately, that requires that the 2M (or the size of the mmap if lower) be contiguous, and it's easy to run out of contiguous

Re: [ast-users] Thank you for the grep builtin!

2013-09-13 Thread ольга крыжановская
John, please correct me, but AFAIK does not support large pages/huge pages, for mmap() on files, right? AFAIK Solaris 11 was the first Unix which explicitly supports large pages/huge pages for mmap() on files. Olga On Sat, Sep 14, 2013 at 12:43 AM, Haller, John H (John)

Re: [ast-users] Thank you for the grep builtin!

2013-09-13 Thread Haller, John H (John)
You're right, only supported on anonymous memory mapping in Linux currently, page cache layer is a possible future use. Large mmap will wipe PTE cache there until that future. Regards, John Haller -Original Message- From: ольга крыжановская [mailto:olga.kryzhanov...@gmail.com] Sent: