Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-13 Thread Tejun Heo
Hello, Tim. On Thu, Dec 12, 2013 at 04:23:18PM -0800, Tim Hockin wrote: > Just to be clear - I say this because it doesn't feel right to impose > my craziness on others, and it sucks when we try and are met with > "you're crazy, go away". And you have to admit that happens to > Google. :)

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-13 Thread Tejun Heo
Hello, Tim. On Thu, Dec 12, 2013 at 04:23:18PM -0800, Tim Hockin wrote: Just to be clear - I say this because it doesn't feel right to impose my craziness on others, and it sucks when we try and are met with you're crazy, go away. And you have to admit that happens to Google. :) Punching an

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-12 Thread Tim Hockin
On Thu, Dec 12, 2013 at 11:23 AM, Tejun Heo wrote: > Hello, Tim. > > On Thu, Dec 12, 2013 at 10:42:20AM -0800, Tim Hockin wrote: >> Yeah sorry. Replying from my phone is awkward at best. I know better :) > > Heh, sorry about being bitchy. :) > >> In my mind, the ONLY point of pulling system-OOM

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-12 Thread Tejun Heo
Hello, Tim. On Thu, Dec 12, 2013 at 10:42:20AM -0800, Tim Hockin wrote: > Yeah sorry. Replying from my phone is awkward at best. I know better :) Heh, sorry about being bitchy. :) > In my mind, the ONLY point of pulling system-OOM handling into > userspace is to make it easier for crazy

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-12 Thread Tim Hockin
On Thu, Dec 12, 2013 at 6:21 AM, Tejun Heo wrote: > Hey, Tim. > > Sidenote: Please don't top-post with the whole body quoted below > unless you're adding new cc's. Please selectively quote the original > message's body to remind the readers of the context and reply below > it. It's a basic lkml

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-12 Thread Michal Hocko
On Thu 12-12-13 09:21:56, Tejun Heo wrote: [...] > There'd still be all the bells and whistles to configure and monitor > system-level OOM and if there's justified need for improvements, we > surely can and should do that; You weren't on the CC of the original thread which has started here

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-12 Thread Tejun Heo
Hello, Michal. On Thu, Dec 12, 2013 at 05:32:22PM +0100, Michal Hocko wrote: > You weren't on the CC of the original thread which has started here > https://lkml.org/lkml/2013/11/19/191. And the original request for > discussion was more about user defined _policies_ for the global > OOM rather

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-12 Thread Tejun Heo
Hey, Tim. Sidenote: Please don't top-post with the whole body quoted below unless you're adding new cc's. Please selectively quote the original message's body to remind the readers of the context and reply below it. It's a basic lkml etiquette and one with good reasons. If you have to top-post

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-12 Thread Tejun Heo
Hey, Tim. Sidenote: Please don't top-post with the whole body quoted below unless you're adding new cc's. Please selectively quote the original message's body to remind the readers of the context and reply below it. It's a basic lkml etiquette and one with good reasons. If you have to top-post

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-12 Thread Tejun Heo
Hello, Michal. On Thu, Dec 12, 2013 at 05:32:22PM +0100, Michal Hocko wrote: You weren't on the CC of the original thread which has started here https://lkml.org/lkml/2013/11/19/191. And the original request for discussion was more about user defined _policies_ for the global OOM rather than

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-12 Thread Michal Hocko
On Thu 12-12-13 09:21:56, Tejun Heo wrote: [...] There'd still be all the bells and whistles to configure and monitor system-level OOM and if there's justified need for improvements, we surely can and should do that; You weren't on the CC of the original thread which has started here

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-12 Thread Tim Hockin
On Thu, Dec 12, 2013 at 6:21 AM, Tejun Heo t...@kernel.org wrote: Hey, Tim. Sidenote: Please don't top-post with the whole body quoted below unless you're adding new cc's. Please selectively quote the original message's body to remind the readers of the context and reply below it. It's a

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-12 Thread Tejun Heo
Hello, Tim. On Thu, Dec 12, 2013 at 10:42:20AM -0800, Tim Hockin wrote: Yeah sorry. Replying from my phone is awkward at best. I know better :) Heh, sorry about being bitchy. :) In my mind, the ONLY point of pulling system-OOM handling into userspace is to make it easier for crazy people

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-12 Thread Tim Hockin
On Thu, Dec 12, 2013 at 11:23 AM, Tejun Heo t...@kernel.org wrote: Hello, Tim. On Thu, Dec 12, 2013 at 10:42:20AM -0800, Tim Hockin wrote: Yeah sorry. Replying from my phone is awkward at best. I know better :) Heh, sorry about being bitchy. :) In my mind, the ONLY point of pulling

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-11 Thread Tim Hockin
The immediate problem I see with setting aside reserves "off the top" is that we don't really know a priori how much memory the kernel itself is going to use, which could still land us in an overcommitted state. In other words, if I have your 128 MB machine, and I set aside 8 MB for OOM handling,

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-11 Thread Tejun Heo
Yo, On Tue, Dec 10, 2013 at 03:55:48PM -0800, David Rientjes wrote: > > Well, the gotcha there is that you won't be able to do that with > > system level OOM handler either unless you create a separately > > reserved memory, which, again, can be achieved using hierarchical > > memcg setup

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-11 Thread Mel Gorman
On Tue, Dec 10, 2013 at 03:55:48PM -0800, David Rientjes wrote: > > Okay, are you saying that userland OOM handlers will be able to dip > > into kernel reserve memory? Maybe I'm mistaken but you realize that > > that reserve is there to make things like task exits work under OOM > > conditions,

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-11 Thread Mel Gorman
On Tue, Dec 10, 2013 at 03:55:48PM -0800, David Rientjes wrote: Okay, are you saying that userland OOM handlers will be able to dip into kernel reserve memory? Maybe I'm mistaken but you realize that that reserve is there to make things like task exits work under OOM conditions, right?

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-11 Thread Tejun Heo
Yo, On Tue, Dec 10, 2013 at 03:55:48PM -0800, David Rientjes wrote: Well, the gotcha there is that you won't be able to do that with system level OOM handler either unless you create a separately reserved memory, which, again, can be achieved using hierarchical memcg setup already. Am I

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-11 Thread Tim Hockin
The immediate problem I see with setting aside reserves off the top is that we don't really know a priori how much memory the kernel itself is going to use, which could still land us in an overcommitted state. In other words, if I have your 128 MB machine, and I set aside 8 MB for OOM handling,

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-10 Thread David Rientjes
On Tue, 10 Dec 2013, Tejun Heo wrote: > > Indeed. The setup I'm specifically trying to attack is where the sum of > > the limits of all non-oom handling memcgs (A/b in my model, A in yours) > > exceed the amount of RAM. If the system has 256MB, > > > > /=256MB > >

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-10 Thread Tejun Heo
Hey, David. On Mon, Dec 09, 2013 at 12:10:44PM -0800, David Rientjes wrote: > Indeed. The setup I'm specifically trying to attack is where the sum of > the limits of all non-oom handling memcgs (A/b in my model, A in yours) > exceed the amount of RAM. If the system has 256MB, > >

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-10 Thread Tejun Heo
Hey, David. On Mon, Dec 09, 2013 at 12:10:44PM -0800, David Rientjes wrote: Indeed. The setup I'm specifically trying to attack is where the sum of the limits of all non-oom handling memcgs (A/b in my model, A in yours) exceed the amount of RAM. If the system has 256MB,

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-10 Thread David Rientjes
On Tue, 10 Dec 2013, Tejun Heo wrote: Indeed. The setup I'm specifically trying to attack is where the sum of the limits of all non-oom handling memcgs (A/b in my model, A in yours) exceed the amount of RAM. If the system has 256MB, /=256MB A=126MB

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-09 Thread Johannes Weiner
On Mon, Dec 09, 2013 at 12:10:44PM -0800, David Rientjes wrote: > On Fri, 6 Dec 2013, Tejun Heo wrote: > > > > Tejun, how are you? > > > > Doing pretty good. How's yourself? :) > > > > Not bad, busy with holidays and all that. > > > > I agree that we wouldn't need such support if we are only

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-09 Thread David Rientjes
On Fri, 6 Dec 2013, Tejun Heo wrote: > > Tejun, how are you? > > Doing pretty good. How's yourself? :) > Not bad, busy with holidays and all that. > > I agree that we wouldn't need such support if we are only addressing memcg > > oom conditions. We could do things like

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-09 Thread David Rientjes
On Fri, 6 Dec 2013, Tejun Heo wrote: Tejun, how are you? Doing pretty good. How's yourself? :) Not bad, busy with holidays and all that. I agree that we wouldn't need such support if we are only addressing memcg oom conditions. We could do things like A/memory.limit_in_bytes ==

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-09 Thread Johannes Weiner
On Mon, Dec 09, 2013 at 12:10:44PM -0800, David Rientjes wrote: On Fri, 6 Dec 2013, Tejun Heo wrote: Tejun, how are you? Doing pretty good. How's yourself? :) Not bad, busy with holidays and all that. I agree that we wouldn't need such support if we are only addressing

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-07 Thread Johannes Weiner
On Sat, Dec 07, 2013 at 10:12:19AM -0800, Tim Hockin wrote: > You more or less described the fundamental change - a score per memcg, with > a recursive OOM killer which evaluates scores between siblings at the same > level. > > It gets a bit complicated because we have need if wider scoring

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-07 Thread Johannes Weiner
Hello Tim! On Sat, Dec 07, 2013 at 08:38:20AM -0800, Tim Hockin wrote: > We actually started with kernel patches all h these lines - per-memcg > scores and all of our crazy policy requirements. > > It turns out that changing policies is hard. > > When David offered the opportunity to manage it

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-07 Thread Johannes Weiner
Hello Tim! On Sat, Dec 07, 2013 at 08:38:20AM -0800, Tim Hockin wrote: We actually started with kernel patches all h these lines - per-memcg scores and all of our crazy policy requirements. It turns out that changing policies is hard. When David offered the opportunity to manage it all in

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-07 Thread Johannes Weiner
On Sat, Dec 07, 2013 at 10:12:19AM -0800, Tim Hockin wrote: You more or less described the fundamental change - a score per memcg, with a recursive OOM killer which evaluates scores between siblings at the same level. It gets a bit complicated because we have need if wider scoring ranges

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-06 Thread Tejun Heo
Yo, David. On Thu, Dec 05, 2013 at 03:49:57PM -0800, David Rientjes wrote: > Tejun, how are you? Doing pretty good. How's yourself? :) > > Umm.. without delving into details, aren't you basically creating a > > memory cgroup inside a memory cgroup? Doesn't sound like a > > particularly well

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-06 Thread Johannes Weiner
On Thu, Dec 05, 2013 at 03:49:57PM -0800, David Rientjes wrote: > On Wed, 4 Dec 2013, Tejun Heo wrote: > > > Hello, > > > > Tejun, how are you? > > > Umm.. without delving into details, aren't you basically creating a > > memory cgroup inside a memory cgroup? Doesn't sound like a > >

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-06 Thread Johannes Weiner
On Thu, Dec 05, 2013 at 03:49:57PM -0800, David Rientjes wrote: On Wed, 4 Dec 2013, Tejun Heo wrote: Hello, Tejun, how are you? Umm.. without delving into details, aren't you basically creating a memory cgroup inside a memory cgroup? Doesn't sound like a particularly well

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-06 Thread Tejun Heo
Yo, David. On Thu, Dec 05, 2013 at 03:49:57PM -0800, David Rientjes wrote: Tejun, how are you? Doing pretty good. How's yourself? :) Umm.. without delving into details, aren't you basically creating a memory cgroup inside a memory cgroup? Doesn't sound like a particularly well

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-05 Thread David Rientjes
On Wed, 4 Dec 2013, Tejun Heo wrote: > Hello, > Tejun, how are you? > Umm.. without delving into details, aren't you basically creating a > memory cgroup inside a memory cgroup? Doesn't sound like a > particularly well thought-out plan to me. > I agree that we wouldn't need such support if

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-05 Thread David Rientjes
On Wed, 4 Dec 2013, Tejun Heo wrote: Hello, Tejun, how are you? Umm.. without delving into details, aren't you basically creating a memory cgroup inside a memory cgroup? Doesn't sound like a particularly well thought-out plan to me. I agree that we wouldn't need such support if we are

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-04 Thread Tejun Heo
Hello, On Wed, Dec 04, 2013 at 05:49:04PM -0800, David Rientjes wrote: > That's not what this series is addressing, though, and in fact it's quite > the opposite. It acknowledges that userspace oom handlers need to > allocate and that anything else would be too difficult to maintain >

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-04 Thread David Rientjes
On Wed, 4 Dec 2013, Johannes Weiner wrote: > > Now that a per-process flag is available, define it for processes that > > handle userspace oom notifications. This is an optimization to avoid > > mantaining a list of such processes attached to a memcg at any given time > > and iterating it at

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-04 Thread David Rientjes
On Wed, 4 Dec 2013, Johannes Weiner wrote: Now that a per-process flag is available, define it for processes that handle userspace oom notifications. This is an optimization to avoid mantaining a list of such processes attached to a memcg at any given time and iterating it at charge

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-04 Thread Tejun Heo
Hello, On Wed, Dec 04, 2013 at 05:49:04PM -0800, David Rientjes wrote: That's not what this series is addressing, though, and in fact it's quite the opposite. It acknowledges that userspace oom handlers need to allocate and that anything else would be too difficult to maintain (thereby

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-03 Thread Johannes Weiner
On Tue, Dec 03, 2013 at 09:20:17PM -0800, David Rientjes wrote: > Now that a per-process flag is available, define it for processes that > handle userspace oom notifications. This is an optimization to avoid > mantaining a list of such processes attached to a memcg at any given time > and

[patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-03 Thread David Rientjes
Now that a per-process flag is available, define it for processes that handle userspace oom notifications. This is an optimization to avoid mantaining a list of such processes attached to a memcg at any given time and iterating it at charge time. This flag gets set whenever a process has

[patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-03 Thread David Rientjes
Now that a per-process flag is available, define it for processes that handle userspace oom notifications. This is an optimization to avoid mantaining a list of such processes attached to a memcg at any given time and iterating it at charge time. This flag gets set whenever a process has

Re: [patch 7/8] mm, memcg: allow processes handling oom notifications to access reserves

2013-12-03 Thread Johannes Weiner
On Tue, Dec 03, 2013 at 09:20:17PM -0800, David Rientjes wrote: Now that a per-process flag is available, define it for processes that handle userspace oom notifications. This is an optimization to avoid mantaining a list of such processes attached to a memcg at any given time and iterating