On Tue, 30 Aug 2005, Nick Piggin wrote:
>
> Andrew, did you pick up the patch or should I resend to someone?
I picked it up. If it causes performance regressions, we can fix them, and
if it causes other problems then that will be interesting in itself.
Linus
-
To unsubscribe
Ray Fucillo wrote:
Nick Piggin wrote:
How does the following look? (I changed the comment a bit). Andrew,
please
apply if nobody objects.
Nick, I applied this latest patch to a 2.6.12 kernel and found that it
does resolve the problem. Prior to the patch on this machine, I was
seeing
On Mon, 29 Aug 2005, Ray Fucillo wrote:
>
> FWIW, an interesting side effect of this occurs when I run the database
> with this patch internally on a Linux server that uses NIS. Its an
> unrelated problem and not a kernel problem. Its due to the children
> calling initgroups()...
Nick Piggin wrote:
How does the following look? (I changed the comment a bit). Andrew, please
apply if nobody objects.
Nick, I applied this latest patch to a 2.6.12 kernel and found that it
does resolve the problem. Prior to the patch on this machine, I was
seeing about 23ms spent in fork
Nick Piggin wrote:
How does the following look? (I changed the comment a bit). Andrew, please
apply if nobody objects.
Nick, I applied this latest patch to a 2.6.12 kernel and found that it
does resolve the problem. Prior to the patch on this machine, I was
seeing about 23ms spent in fork
On Mon, 29 Aug 2005, Ray Fucillo wrote:
FWIW, an interesting side effect of this occurs when I run the database
with this patch internally on a Linux server that uses NIS. Its an
unrelated problem and not a kernel problem. Its due to the children
calling initgroups()... apparently
Ray Fucillo wrote:
Nick Piggin wrote:
How does the following look? (I changed the comment a bit). Andrew,
please
apply if nobody objects.
Nick, I applied this latest patch to a 2.6.12 kernel and found that it
does resolve the problem. Prior to the patch on this machine, I was
seeing
On Tue, 30 Aug 2005, Nick Piggin wrote:
Andrew, did you pick up the patch or should I resend to someone?
I picked it up. If it causes performance regressions, we can fix them, and
if it causes other problems then that will be interesting in itself.
Linus
-
To unsubscribe
Hugh Dickins wrote:
On Sun, 28 Aug 2005, Nick Piggin wrote:
This is the condition I ended up with. Any good?
if (!(vma->vm_flags & (VM_HUGETLB|VM_NONLINEAR|VM_RESERVED))) {
if (vma->vm_flags & VM_MAYSHARE)
return 0;
if (vma->vm_file && !vma->anon_vma)
return 0;
}
It's not bad, and
Hugh Dickins wrote:
On Sun, 28 Aug 2005, Nick Piggin wrote:
This is the condition I ended up with. Any good?
if (!(vma-vm_flags (VM_HUGETLB|VM_NONLINEAR|VM_RESERVED))) {
if (vma-vm_flags VM_MAYSHARE)
return 0;
if (vma-vm_file !vma-anon_vma)
return 0;
}
It's not bad, and practical
On Sun, 28 Aug 2005, Nick Piggin wrote:
>
> This is the condition I ended up with. Any good?
>
> if (!(vma->vm_flags & (VM_HUGETLB|VM_NONLINEAR|VM_RESERVED))) {
> if (vma->vm_flags & VM_MAYSHARE)
> return 0;
> if (vma->vm_file && !vma->anon_vma)
>return 0;
> }
It's not bad, and practical
Linus Torvalds wrote:
On Fri, 26 Aug 2005, Rik van Riel wrote:
On Fri, 26 Aug 2005, Hugh Dickins wrote:
Well, I still don't think we need to test vm_file. We can add an
anon_vma test if you like, if we really want to minimize the fork
overhead, in favour of later faults. Do we?
When
Linus Torvalds wrote:
On Fri, 26 Aug 2005, Rik van Riel wrote:
On Fri, 26 Aug 2005, Hugh Dickins wrote:
Well, I still don't think we need to test vm_file. We can add an
anon_vma test if you like, if we really want to minimize the fork
overhead, in favour of later faults. Do we?
When
On Sun, 28 Aug 2005, Nick Piggin wrote:
This is the condition I ended up with. Any good?
if (!(vma-vm_flags (VM_HUGETLB|VM_NONLINEAR|VM_RESERVED))) {
if (vma-vm_flags VM_MAYSHARE)
return 0;
if (vma-vm_file !vma-anon_vma)
return 0;
}
It's not bad, and practical timings are
On Fri, 26 Aug 2005, Rik van Riel wrote:
> On Fri, 26 Aug 2005, Hugh Dickins wrote:
>
> > Well, I still don't think we need to test vm_file. We can add an
> > anon_vma test if you like, if we really want to minimize the fork
> > overhead, in favour of later faults. Do we?
>
> When you
On Fri, 26 Aug 2005, Hugh Dickins wrote:
> Well, I still don't think we need to test vm_file. We can add an
> anon_vma test if you like, if we really want to minimize the fork
> overhead, in favour of later faults. Do we?
When you consider NUMA placement (the child process may
end up running
On Fri, 26 Aug 2005, Hugh Dickins wrote:
>
> Well, I still don't think we need to test vm_file. We can add an
> anon_vma test if you like, if we really want to minimize the fork
> overhead, in favour of later faults. Do we?
I think we might want to do it in -mm for testing. Because quite
On Fri, 26 Aug 2005, Ross Biro wrote:
> On 8/26/05, Rik van Riel <[EMAIL PROTECTED]> wrote:
> >
> > Filling in all the page table entries at the first fault to
> > a VMA doesn't make much sense, IMHO.
> >
> > I suspect we would be better off without that extra complexity,
> > unless there is a
On Fri, 26 Aug 2005, Linus Torvalds wrote:
> On Fri, 26 Aug 2005, Hugh Dickins wrote:
> >
> > I see some flaws in the various patches posted, including Rik's.
> > Here's another version - doing it inside copy_page_range, so this
> > kind of vma special-casing is over in mm/ rather than kernel/.
>
On 8/26/05, Rik van Riel <[EMAIL PROTECTED]> wrote:
>
> Filling in all the page table entries at the first fault to
> a VMA doesn't make much sense, IMHO.
>
>
> I suspect we would be better off without that extra complexity,
> unless there is a demonstrated benefit to it.
You are probably
On Fri, 26 Aug 2005, Hugh Dickins wrote:
>
> I see some flaws in the various patches posted, including Rik's.
> Here's another version - doing it inside copy_page_range, so this
> kind of vma special-casing is over in mm/ rather than kernel/.
I like this approach better, but I don't understand
On Fri, 26 Aug 2005, Ray Fucillo wrote:
> However, there is still a need that the child, once successfully forked, is
> operational reasonably quickly. I suspect that Ross's idea of paging in
> everything after the first fault would not be optimal for us, because we'd
> still be talking about
Nick Piggin wrote:
OK let's see how Ray goes, and try it when 2.6.14 opens...
Working on that now - I'll let you know.
Yeah I guess that's a good idea. Patch looks pretty good.
Just a minor issue with the comment, it is not strictly
just assuming the child will exec... IMO it is worthwhile
On 8/26/05, Hugh Dickins <[EMAIL PROTECTED]> wrote:
> On Fri, 26 Aug 2005, Ross Biro wrote:
> > On 8/26/05, Hugh Dickins <[EMAIL PROTECTED]> wrote:
> > >
> > > The refaulting will hurt the performance of something: let's
> > > just hope that something doesn't turn out to be a show-stopper.
> >
> >
On Fri, 26 Aug 2005, Ross Biro wrote:
> On 8/26/05, Hugh Dickins <[EMAIL PROTECTED]> wrote:
> >
> > The refaulting will hurt the performance of something: let's
> > just hope that something doesn't turn out to be a show-stopper.
>
> Why not just fault in all the pages on the first fault. Then
Hugh Dickins wrote:
On Thu, 25 Aug 2005, Linus Torvalds wrote:
That said, I think it's a valid optimization. Especially as the child
_probably_ doesn't need it (ie there's at least some likelihood of an
execve() or similar).
I agree, seems a great idea to me (sulking because I was too
On Thu, 25 Aug 2005, Linus Torvalds wrote:
> On Fri, 26 Aug 2005, Nick Piggin wrote:
> >
> > > Skipping MAP_SHARED in fork() sounds like a good idea to me...
> >
> > Indeed. Linus, can you remember why we haven't done this before?
>
> Hmm. Historical reasons. Also, if the child ends up needing
On Thu, 25 Aug 2005, Linus Torvalds wrote:
On Fri, 26 Aug 2005, Nick Piggin wrote:
Skipping MAP_SHARED in fork() sounds like a good idea to me...
Indeed. Linus, can you remember why we haven't done this before?
Hmm. Historical reasons. Also, if the child ends up needing it, it will
Hugh Dickins wrote:
On Thu, 25 Aug 2005, Linus Torvalds wrote:
That said, I think it's a valid optimization. Especially as the child
_probably_ doesn't need it (ie there's at least some likelihood of an
execve() or similar).
I agree, seems a great idea to me (sulking because I was too
On Fri, 26 Aug 2005, Ross Biro wrote:
On 8/26/05, Hugh Dickins [EMAIL PROTECTED] wrote:
The refaulting will hurt the performance of something: let's
just hope that something doesn't turn out to be a show-stopper.
Why not just fault in all the pages on the first fault. Then the
On 8/26/05, Hugh Dickins [EMAIL PROTECTED] wrote:
On Fri, 26 Aug 2005, Ross Biro wrote:
On 8/26/05, Hugh Dickins [EMAIL PROTECTED] wrote:
The refaulting will hurt the performance of something: let's
just hope that something doesn't turn out to be a show-stopper.
Why not just fault
Nick Piggin wrote:
OK let's see how Ray goes, and try it when 2.6.14 opens...
Working on that now - I'll let you know.
Yeah I guess that's a good idea. Patch looks pretty good.
Just a minor issue with the comment, it is not strictly
just assuming the child will exec... IMO it is worthwhile
On Fri, 26 Aug 2005, Ray Fucillo wrote:
However, there is still a need that the child, once successfully forked, is
operational reasonably quickly. I suspect that Ross's idea of paging in
everything after the first fault would not be optimal for us, because we'd
still be talking about
On Fri, 26 Aug 2005, Hugh Dickins wrote:
I see some flaws in the various patches posted, including Rik's.
Here's another version - doing it inside copy_page_range, so this
kind of vma special-casing is over in mm/ rather than kernel/.
I like this approach better, but I don't understand
On 8/26/05, Rik van Riel [EMAIL PROTECTED] wrote:
Filling in all the page table entries at the first fault to
a VMA doesn't make much sense, IMHO.
I suspect we would be better off without that extra complexity,
unless there is a demonstrated benefit to it.
You are probably right, but do
On Fri, 26 Aug 2005, Linus Torvalds wrote:
On Fri, 26 Aug 2005, Hugh Dickins wrote:
I see some flaws in the various patches posted, including Rik's.
Here's another version - doing it inside copy_page_range, so this
kind of vma special-casing is over in mm/ rather than kernel/.
I like
On Fri, 26 Aug 2005, Ross Biro wrote:
On 8/26/05, Rik van Riel [EMAIL PROTECTED] wrote:
Filling in all the page table entries at the first fault to
a VMA doesn't make much sense, IMHO.
I suspect we would be better off without that extra complexity,
unless there is a demonstrated
On Fri, 26 Aug 2005, Hugh Dickins wrote:
Well, I still don't think we need to test vm_file. We can add an
anon_vma test if you like, if we really want to minimize the fork
overhead, in favour of later faults. Do we?
I think we might want to do it in -mm for testing. Because quite frankly,
On Fri, 26 Aug 2005, Hugh Dickins wrote:
Well, I still don't think we need to test vm_file. We can add an
anon_vma test if you like, if we really want to minimize the fork
overhead, in favour of later faults. Do we?
When you consider NUMA placement (the child process may
end up running
On Fri, 26 Aug 2005, Rik van Riel wrote:
On Fri, 26 Aug 2005, Hugh Dickins wrote:
Well, I still don't think we need to test vm_file. We can add an
anon_vma test if you like, if we really want to minimize the fork
overhead, in favour of later faults. Do we?
When you consider NUMA
On Fri, 26 Aug 2005, Nick Piggin wrote:
>
> > Skipping MAP_SHARED in fork() sounds like a good idea to me...
> >
>
> Indeed. Linus, can you remember why we haven't done this before?
Hmm. Historical reasons. Also, if the child ends up needing it, it will
now have to fault them in.
That
On Fri, 26 Aug 2005, Nick Piggin wrote:
> > Skipping MAP_SHARED in fork() sounds like a good idea to me...
>
> Indeed. Linus, can you remember why we haven't done this before?
Where "this" looks something like the patch below, shamelessly
merging Nick's and Andy's patches and adding the
Rik van Riel wrote:
On Thu, 25 Aug 2005, Nick Piggin wrote:
fork() can be changed so as not to set up page tables for
MAP_SHARED mappings. I think that has other tradeoffs like
initially causing several unavoidable faults reading
libraries and program text.
Actually, libraries and program
On Thu, 25 Aug 2005, Nick Piggin wrote:
> fork() can be changed so as not to set up page tables for
> MAP_SHARED mappings. I think that has other tradeoffs like
> initially causing several unavoidable faults reading
> libraries and program text.
Actually, libraries and program text are usually
On Thursday 25 August 2005 16:47, Parag Warudkar wrote:
> Exactly - one problem is that this forces all of the hugetlb users to go
> the lazy faulting way.
Actually I disabled it for hugetlbfs (... !is_huge...vma). The reason
is that lazy faulting for huge pages is still not in mainline.
-Andi
On Thu, 2005-08-25 at 16:22 +0200, Andi Kleen wrote:
> But I'm not sure it's a good idea in all cases. Would need a lot of
> benchmarking at least.
>
> -Andi
>
Exactly - one problem is that this forces all of the hugetlb users to go
the lazy faulting way. This is more or less similar to the
Andi Kleen wrote:
Would it be worth trying to do something like this?
Maybe. Shouldn't be very hard though - you just need to check if the VMA is
backed by an object and if yes don't call copy_page_range for it.
I think it just needs (untested)
I think you need to check for MAP_SHARED
Ray Fucillo wrote:
Nick Piggin wrote:
fork() can be changed so as not to set up page tables for
MAP_SHARED mappings. I think that has other tradeoffs like
initially causing several unavoidable faults reading
libraries and program text.
What kind of application are you using?
The
> Would it be worth trying to do something like this?
Maybe. Shouldn't be very hard though - you just need to check if the VMA is
backed by an object and if yes don't call copy_page_range for it.
I think it just needs (untested)
Index: linux-2.6.13-rc5-misc/kernel/fork.c
> Ray Fucillo <[EMAIL PROTECTED]> writes:
> >
> > The application is a database system called Caché. We allocate a
> > large shared memory segment for database cache, which in a large
> > production environment may realistically be 1+GB on 32-bit platforms
> > and much larger on 64-bit. At
Ray Fucillo <[EMAIL PROTECTED]> writes:
>
> The application is a database system called Caché. We allocate a
> large shared memory segment for database cache, which in a large
> production environment may realistically be 1+GB on 32-bit platforms
> and much larger on 64-bit. At these sizes
Nick Piggin wrote:
fork() can be changed so as not to set up page tables for
MAP_SHARED mappings. I think that has other tradeoffs like
initially causing several unavoidable faults reading
libraries and program text.
What kind of application are you using?
The application is a database system
Nick Piggin wrote:
fork() can be changed so as not to set up page tables for
MAP_SHARED mappings. I think that has other tradeoffs like
initially causing several unavoidable faults reading
libraries and program text.
What kind of application are you using?
The application is a database system
Ray Fucillo [EMAIL PROTECTED] writes:
The application is a database system called Caché. We allocate a
large shared memory segment for database cache, which in a large
production environment may realistically be 1+GB on 32-bit platforms
and much larger on 64-bit. At these sizes fork() is
Ray Fucillo [EMAIL PROTECTED] writes:
The application is a database system called Caché. We allocate a
large shared memory segment for database cache, which in a large
production environment may realistically be 1+GB on 32-bit platforms
and much larger on 64-bit. At these sizes
Would it be worth trying to do something like this?
Maybe. Shouldn't be very hard though - you just need to check if the VMA is
backed by an object and if yes don't call copy_page_range for it.
I think it just needs (untested)
Index: linux-2.6.13-rc5-misc/kernel/fork.c
Ray Fucillo wrote:
Nick Piggin wrote:
fork() can be changed so as not to set up page tables for
MAP_SHARED mappings. I think that has other tradeoffs like
initially causing several unavoidable faults reading
libraries and program text.
What kind of application are you using?
The
Andi Kleen wrote:
Would it be worth trying to do something like this?
Maybe. Shouldn't be very hard though - you just need to check if the VMA is
backed by an object and if yes don't call copy_page_range for it.
I think it just needs (untested)
I think you need to check for MAP_SHARED
On Thu, 2005-08-25 at 16:22 +0200, Andi Kleen wrote:
But I'm not sure it's a good idea in all cases. Would need a lot of
benchmarking at least.
-Andi
Exactly - one problem is that this forces all of the hugetlb users to go
the lazy faulting way. This is more or less similar to the
On Thu, 25 Aug 2005, Nick Piggin wrote:
fork() can be changed so as not to set up page tables for
MAP_SHARED mappings. I think that has other tradeoffs like
initially causing several unavoidable faults reading
libraries and program text.
Actually, libraries and program text are usually
Rik van Riel wrote:
On Thu, 25 Aug 2005, Nick Piggin wrote:
fork() can be changed so as not to set up page tables for
MAP_SHARED mappings. I think that has other tradeoffs like
initially causing several unavoidable faults reading
libraries and program text.
Actually, libraries and program
On Fri, 26 Aug 2005, Nick Piggin wrote:
Skipping MAP_SHARED in fork() sounds like a good idea to me...
Indeed. Linus, can you remember why we haven't done this before?
Where this looks something like the patch below, shamelessly
merging Nick's and Andy's patches and adding the
On Fri, 26 Aug 2005, Nick Piggin wrote:
Skipping MAP_SHARED in fork() sounds like a good idea to me...
Indeed. Linus, can you remember why we haven't done this before?
Hmm. Historical reasons. Also, if the child ends up needing it, it will
now have to fault them in.
That said, I
Ray Fucillo wrote:
I am seeing process creation time increase linearly with the size of the
shared memory segment that the parent touches. The attached forktest.c
is a very simple user program that illustrates this behavior, which I
have tested on various kernel versions from 2.4 through 2.6.
I am seeing process creation time increase linearly with the size of the
shared memory segment that the parent touches. The attached forktest.c
is a very simple user program that illustrates this behavior, which I
have tested on various kernel versions from 2.4 through 2.6. Is this a
known
I am seeing process creation time increase linearly with the size of the
shared memory segment that the parent touches. The attached forktest.c
is a very simple user program that illustrates this behavior, which I
have tested on various kernel versions from 2.4 through 2.6. Is this a
known
Ray Fucillo wrote:
I am seeing process creation time increase linearly with the size of the
shared memory segment that the parent touches. The attached forktest.c
is a very simple user program that illustrates this behavior, which I
have tested on various kernel versions from 2.4 through 2.6.
67 matches
Mail list logo
