On Wed, 2007-06-06 at 01:00 -0700, Andrew Morton wrote:
> On Wed, 6 Jun 2007 00:56:10 -0700 Paul Jackson <[EMAIL PROTECTED]> wrote:
>
> > David wrote:
> > > That is, unless you can guarantee this type of problem will not happen
> > > again
> >
> > Well, I certainly cannot guarantee that.
>
>
On Wed, 6 Jun 2007 00:56:10 -0700 Paul Jackson <[EMAIL PROTECTED]> wrote:
> David wrote:
> > That is, unless you can guarantee this type of problem will not happen again
>
> Well, I certainly cannot guarantee that.
The only place I can think of where the kernel will sit there allocating
huge
On Wed, 2007-06-06 at 00:48 -0700, David Rientjes wrote:
> On Wed, 6 Jun 2007, Paul Jackson wrote:
>
> > Seems like that mlock code is able then to get great globs of memory
> > without returning to user space ... perhaps that's where the fix
> > should be ... that code should quit chewing up
David wrote:
> That is, unless you can guarantee this type of problem will not happen again
Well, I certainly cannot guarantee that.
Heck, I can't even guarantee isn't happening right now, somewhere else.
But I'm no memory guru.
--
I won't rest till it's the best ...
On Wed, 6 Jun 2007, Paul Jackson wrote:
> Seems like that mlock code is able then to get great globs of memory
> without returning to user space ... perhaps that's where the fix
> should be ... that code should quit chewing up memory if it's
> marked MEMDIE or some such?
>
That's one case. Are
On Wed, 6 Jun 2007 00:34:21 -0700 Paul Jackson <[EMAIL PROTECTED]> wrote:
> > a separate exclusive cpuset mlock'd a gigantic amount of
> > memory and it could not reliably exit because the mlock continued to
> > allocate outside its own cpuset and eventually OOM'd system-critical tasks
> > or
> a separate exclusive cpuset mlock'd a gigantic amount of
> memory and it could not reliably exit because the mlock continued to
> allocate outside its own cpuset and eventually OOM'd system-critical tasks
> or depleated all system memory.
Seems like that mlock code is able then to get great
On Wed, 6 Jun 2007, Peter Zijlstra wrote:
> Right, I see your point; however considering that its a system
> allocation, and all these constraints get violated by interrupts anyway,
> its more of an application container than a strict allocation container.
>
It is not necessarily system
On Tue, 2007-06-05 at 23:42 -0700, David Rientjes wrote:
> On Wed, 6 Jun 2007, Peter Zijlstra wrote:
>
> > > diff --git a/kernel/cpuset.c b/kernel/cpuset.c
> > > --- a/kernel/cpuset.c
> > > +++ b/kernel/cpuset.c
> > > @@ -2431,12 +2431,6 @@ int __cpuset_zone_allowed_softwall(struct zone *z,
> >
On Wed, 6 Jun 2007, Peter Zijlstra wrote:
> > diff --git a/kernel/cpuset.c b/kernel/cpuset.c
> > --- a/kernel/cpuset.c
> > +++ b/kernel/cpuset.c
> > @@ -2431,12 +2431,6 @@ int __cpuset_zone_allowed_softwall(struct zone *z,
> > gfp_t gfp_mask)
> > might_sleep_if(!(gfp_mask & __GFP_HARDWALL));
On Tue, 2007-06-05 at 15:39 -0700, David Rientjes wrote:
> Reverts git commit c596d9f320aaf30d28c1d793ff3a976dee1db8f5.
>
> OOM-killed tasks, marked as TIF_MEMDIE, should not be able to access
> memory outside its cpuset because it could potentially cause other
> exclusive cpusets to OOM
On Tue, 2007-06-05 at 15:39 -0700, David Rientjes wrote:
Reverts git commit c596d9f320aaf30d28c1d793ff3a976dee1db8f5.
OOM-killed tasks, marked as TIF_MEMDIE, should not be able to access
memory outside its cpuset because it could potentially cause other
exclusive cpusets to OOM themselves.
On Wed, 6 Jun 2007, Peter Zijlstra wrote:
diff --git a/kernel/cpuset.c b/kernel/cpuset.c
--- a/kernel/cpuset.c
+++ b/kernel/cpuset.c
@@ -2431,12 +2431,6 @@ int __cpuset_zone_allowed_softwall(struct zone *z,
gfp_t gfp_mask)
might_sleep_if(!(gfp_mask __GFP_HARDWALL));
if
On Tue, 2007-06-05 at 23:42 -0700, David Rientjes wrote:
On Wed, 6 Jun 2007, Peter Zijlstra wrote:
diff --git a/kernel/cpuset.c b/kernel/cpuset.c
--- a/kernel/cpuset.c
+++ b/kernel/cpuset.c
@@ -2431,12 +2431,6 @@ int __cpuset_zone_allowed_softwall(struct zone *z,
gfp_t gfp_mask)
On Wed, 6 Jun 2007, Peter Zijlstra wrote:
Right, I see your point; however considering that its a system
allocation, and all these constraints get violated by interrupts anyway,
its more of an application container than a strict allocation container.
It is not necessarily system allocations
a separate exclusive cpuset mlock'd a gigantic amount of
memory and it could not reliably exit because the mlock continued to
allocate outside its own cpuset and eventually OOM'd system-critical tasks
or depleated all system memory.
Seems like that mlock code is able then to get great
On Wed, 6 Jun 2007 00:34:21 -0700 Paul Jackson [EMAIL PROTECTED] wrote:
a separate exclusive cpuset mlock'd a gigantic amount of
memory and it could not reliably exit because the mlock continued to
allocate outside its own cpuset and eventually OOM'd system-critical tasks
or depleated
On Wed, 6 Jun 2007, Paul Jackson wrote:
Seems like that mlock code is able then to get great globs of memory
without returning to user space ... perhaps that's where the fix
should be ... that code should quit chewing up memory if it's
marked MEMDIE or some such?
That's one case. Are
David wrote:
That is, unless you can guarantee this type of problem will not happen again
Well, I certainly cannot guarantee that.
Heck, I can't even guarantee isn't happening right now, somewhere else.
But I'm no memory guru.
--
I won't rest till it's the best ...
On Wed, 2007-06-06 at 00:48 -0700, David Rientjes wrote:
On Wed, 6 Jun 2007, Paul Jackson wrote:
Seems like that mlock code is able then to get great globs of memory
without returning to user space ... perhaps that's where the fix
should be ... that code should quit chewing up memory if
On Wed, 6 Jun 2007 00:56:10 -0700 Paul Jackson [EMAIL PROTECTED] wrote:
David wrote:
That is, unless you can guarantee this type of problem will not happen again
Well, I certainly cannot guarantee that.
The only place I can think of where the kernel will sit there allocating
huge amounts
On Wed, 2007-06-06 at 01:00 -0700, Andrew Morton wrote:
On Wed, 6 Jun 2007 00:56:10 -0700 Paul Jackson [EMAIL PROTECTED] wrote:
David wrote:
That is, unless you can guarantee this type of problem will not happen
again
Well, I certainly cannot guarantee that.
The only place I
On Tue, 5 Jun 2007, Christoph Lameter wrote:
> H... But we have sent it a SIGKILL. If the process is following
> conventions then it is exiting. Of course the process could be abusing the
> system and attempting to OOM the whole system as an act of revenge for
> being killed but isnt this
On Tue, 5 Jun 2007, David Rientjes wrote:
> mems_allowed. Regardless, we should not allow allocations outside of the
> cpuset because we have marked it TIF_MEMDIE and we don't have any explicit
> guarantee that it is exiting yet and not mlock'ing an excessive amount of
> memory that exceeds
On Tue, 5 Jun 2007, Christoph Lameter wrote:
> On Tue, 5 Jun 2007, David Rientjes wrote:
>
> > Obviously GFP_KERNEL allocations can allocate regardless of our memory
> > exclusivity, but the point is that a job in one exclusive cpuset should
> > not have the ability to effect the performance
On Tue, 5 Jun 2007, David Rientjes wrote:
> Obviously GFP_KERNEL allocations can allocate regardless of our memory
> exclusivity, but the point is that a job in one exclusive cpuset should
> not have the ability to effect the performance (in terms of reclaim and
> swap), memory usage, or
On Tue, 5 Jun 2007, Christoph Lameter wrote:
> Exclusive is not as absolute as you may think. There is also the
> GFP_KERNEL exception.
>
Memory exclusivity with respect to cpusets should guarantee that memory
nodes do not overlap with siblings if they are marked with mems_exclusive.
The
> Sure, that behavior is unchanged. We're relying on
> nearest_exclusive_ancestor() to determine if such nodes overlap.
Ok ... my points on cpusets semantics having been heard,
I stand back down on the matter of memory semantics, where
I am not the master.
Thanks.
--
I
On Tue, 5 Jun 2007, Paul Jackson wrote:
> Well, I can't speak to the 'real' meaning of TIF_MEMDIE with authority,
> but I can speak to the meaning of cpuset flags.
>
> The mem_exclusive flag doesn't mean this.
>
> It means that you cannot overlap the memory of a sibling cpuset.
>
Which, with
On Tue, 5 Jun 2007, David Rientjes wrote:
> If that fails, we can't allocate elsewhere because then we have taken
> exclusive memory from other applications and is contrary to the definition
> of mem_exclusive. You need to construct your cpuset hierarchy with these
> scenarios in mind; when
> If that fails, we can't allocate elsewhere because then we have taken
> exclusive memory from other applications and is contrary to the definition
> of mem_exclusive.
Well, I can't speak to the 'real' meaning of TIF_MEMDIE with authority,
but I can speak to the meaning of cpuset flags.
The
On Tue, 5 Jun 2007, Christoph Lameter wrote:
> But with the patch the process would be able to terminate. There is no
> global OOM situation. If there would be a global OOM situation then
> TIF_MEMDIE would not help.
>
Sure it would, it would have access to memory reserves because of the
On Tue, 5 Jun 2007, David Rientjes wrote:
> > But the alternative is that the exiting process does not save its
> > data.
> The same condition that occurs when there is a system-wide OOM, yes.
> Exclusive cpusets cannot be violated for such allocations outside of the
> obvious GFP_ATOMIC
On Tue, 5 Jun 2007, Christoph Lameter wrote:
> But the alternative is that the exiting process does not save its
> data.
>
The same condition that occurs when there is a system-wide OOM, yes.
Exclusive cpusets cannot be violated for such allocations outside of the
obvious GFP_ATOMIC
> The intended purpose of TIF_MEMDIE was to allocate pages without being
Ok then ... you probably right. I'll stand down.
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[EMAIL PROTECTED]> 1.925.600.0401
On Tue, 5 Jun 2007, David Rientjes wrote:
> No, it means that it can allocate anywhere based on the zonelist ordering
> and then can OOM a very small exclusive cpuset that would never have had
> any memory pressure if it wasn't violated.
But the alternative is that the exiting process does not
On Tue, 5 Jun 2007, Paul Jackson wrote:
> I'm a little surprised at this suggested change -- I'd have thought
> that it was a good idea to let tasks marked for extinction get memory
> anywhere, as they were going to use that memory to exit, and free up
> lots more memory.
>
The intended purpose
> OOM-killed tasks, marked as TIF_MEMDIE, should not be able to access
> memory outside its cpuset because it could potentially cause other
> exclusive cpusets to OOM themselves.
I'm a little surprised at this suggested change -- I'd have thought
that it was a good idea to let tasks marked for
On Tue, 5 Jun 2007, Christoph Lameter wrote:
> On Tue, 5 Jun 2007, David Rientjes wrote:
>
> > Reverts git commit c596d9f320aaf30d28c1d793ff3a976dee1db8f5.
> >
> > OOM-killed tasks, marked as TIF_MEMDIE, should not be able to access
> > memory outside its cpuset because it could potentially
On Tue, 5 Jun 2007, David Rientjes wrote:
> Reverts git commit c596d9f320aaf30d28c1d793ff3a976dee1db8f5.
>
> OOM-killed tasks, marked as TIF_MEMDIE, should not be able to access
> memory outside its cpuset because it could potentially cause other
> exclusive cpusets to OOM themselves.
Those
Reverts git commit c596d9f320aaf30d28c1d793ff3a976dee1db8f5.
OOM-killed tasks, marked as TIF_MEMDIE, should not be able to access
memory outside its cpuset because it could potentially cause other
exclusive cpusets to OOM themselves.
Cc: Andi Kleen <[EMAIL PROTECTED]>
Cc: Christoph Lameter
Reverts git commit c596d9f320aaf30d28c1d793ff3a976dee1db8f5.
OOM-killed tasks, marked as TIF_MEMDIE, should not be able to access
memory outside its cpuset because it could potentially cause other
exclusive cpusets to OOM themselves.
Cc: Andi Kleen [EMAIL PROTECTED]
Cc: Christoph Lameter
On Tue, 5 Jun 2007, David Rientjes wrote:
Reverts git commit c596d9f320aaf30d28c1d793ff3a976dee1db8f5.
OOM-killed tasks, marked as TIF_MEMDIE, should not be able to access
memory outside its cpuset because it could potentially cause other
exclusive cpusets to OOM themselves.
Those
On Tue, 5 Jun 2007, Christoph Lameter wrote:
On Tue, 5 Jun 2007, David Rientjes wrote:
Reverts git commit c596d9f320aaf30d28c1d793ff3a976dee1db8f5.
OOM-killed tasks, marked as TIF_MEMDIE, should not be able to access
memory outside its cpuset because it could potentially cause other
The intended purpose of TIF_MEMDIE was to allocate pages without being
Ok then ... you probably right. I'll stand down.
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson [EMAIL PROTECTED] 1.925.600.0401
-
On Tue, 5 Jun 2007, David Rientjes wrote:
No, it means that it can allocate anywhere based on the zonelist ordering
and then can OOM a very small exclusive cpuset that would never have had
any memory pressure if it wasn't violated.
But the alternative is that the exiting process does not
OOM-killed tasks, marked as TIF_MEMDIE, should not be able to access
memory outside its cpuset because it could potentially cause other
exclusive cpusets to OOM themselves.
I'm a little surprised at this suggested change -- I'd have thought
that it was a good idea to let tasks marked for
On Tue, 5 Jun 2007, Paul Jackson wrote:
I'm a little surprised at this suggested change -- I'd have thought
that it was a good idea to let tasks marked for extinction get memory
anywhere, as they were going to use that memory to exit, and free up
lots more memory.
The intended purpose of
On Tue, 5 Jun 2007, Christoph Lameter wrote:
But the alternative is that the exiting process does not save its
data.
The same condition that occurs when there is a system-wide OOM, yes.
Exclusive cpusets cannot be violated for such allocations outside of the
obvious GFP_ATOMIC exception.
On Tue, 5 Jun 2007, David Rientjes wrote:
But the alternative is that the exiting process does not save its
data.
The same condition that occurs when there is a system-wide OOM, yes.
Exclusive cpusets cannot be violated for such allocations outside of the
obvious GFP_ATOMIC exception.
On Tue, 5 Jun 2007, Christoph Lameter wrote:
But with the patch the process would be able to terminate. There is no
global OOM situation. If there would be a global OOM situation then
TIF_MEMDIE would not help.
Sure it would, it would have access to memory reserves because of the
change
If that fails, we can't allocate elsewhere because then we have taken
exclusive memory from other applications and is contrary to the definition
of mem_exclusive.
Well, I can't speak to the 'real' meaning of TIF_MEMDIE with authority,
but I can speak to the meaning of cpuset flags.
The
On Tue, 5 Jun 2007, David Rientjes wrote:
If that fails, we can't allocate elsewhere because then we have taken
exclusive memory from other applications and is contrary to the definition
of mem_exclusive. You need to construct your cpuset hierarchy with these
scenarios in mind; when you
On Tue, 5 Jun 2007, Paul Jackson wrote:
Well, I can't speak to the 'real' meaning of TIF_MEMDIE with authority,
but I can speak to the meaning of cpuset flags.
The mem_exclusive flag doesn't mean this.
It means that you cannot overlap the memory of a sibling cpuset.
Which, with this
Sure, that behavior is unchanged. We're relying on
nearest_exclusive_ancestor() to determine if such nodes overlap.
Ok ... my points on cpusets semantics having been heard,
I stand back down on the matter of memory semantics, where
I am not the master.
Thanks.
--
I won't
On Tue, 5 Jun 2007, Christoph Lameter wrote:
Exclusive is not as absolute as you may think. There is also the
GFP_KERNEL exception.
Memory exclusivity with respect to cpusets should guarantee that memory
nodes do not overlap with siblings if they are marked with mems_exclusive.
The patch
On Tue, 5 Jun 2007, David Rientjes wrote:
Obviously GFP_KERNEL allocations can allocate regardless of our memory
exclusivity, but the point is that a job in one exclusive cpuset should
not have the ability to effect the performance (in terms of reclaim and
swap), memory usage, or survival
On Tue, 5 Jun 2007, Christoph Lameter wrote:
On Tue, 5 Jun 2007, David Rientjes wrote:
Obviously GFP_KERNEL allocations can allocate regardless of our memory
exclusivity, but the point is that a job in one exclusive cpuset should
not have the ability to effect the performance (in terms
On Tue, 5 Jun 2007, David Rientjes wrote:
mems_allowed. Regardless, we should not allow allocations outside of the
cpuset because we have marked it TIF_MEMDIE and we don't have any explicit
guarantee that it is exiting yet and not mlock'ing an excessive amount of
memory that exceeds the
On Tue, 5 Jun 2007, Christoph Lameter wrote:
H... But we have sent it a SIGKILL. If the process is following
conventions then it is exiting. Of course the process could be abusing the
system and attempting to OOM the whole system as an act of revenge for
being killed but isnt this a
60 matches
Mail list logo
