Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue 31-10-17 12:49:59, Johannes Weiner wrote: > On Tue, Oct 31, 2017 at 09:00:48AM +0100, Michal Hocko wrote: > > On Mon 30-10-17 12:28:13, Shakeel Butt wrote: > > > On Mon, Oct 30, 2017 at 1:29 AM, Michal Hocko wrote: > > > > On Fri 27-10-17 13:50:47, Shakeel Butt wrote: >

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue 31-10-17 12:49:59, Johannes Weiner wrote: > On Tue, Oct 31, 2017 at 09:00:48AM +0100, Michal Hocko wrote: > > On Mon 30-10-17 12:28:13, Shakeel Butt wrote: > > > On Mon, Oct 30, 2017 at 1:29 AM, Michal Hocko wrote: > > > > On Fri 27-10-17 13:50:47, Shakeel Butt wrote: > > > >> > Why is

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue, Oct 31, 2017 at 09:00:48AM +0100, Michal Hocko wrote: > On Mon 30-10-17 12:28:13, Shakeel Butt wrote: > > On Mon, Oct 30, 2017 at 1:29 AM, Michal Hocko wrote: > > > On Fri 27-10-17 13:50:47, Shakeel Butt wrote: > > >> > Why is OOM-disabling a thing? Why isn't this

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue, Oct 31, 2017 at 09:00:48AM +0100, Michal Hocko wrote: > On Mon 30-10-17 12:28:13, Shakeel Butt wrote: > > On Mon, Oct 30, 2017 at 1:29 AM, Michal Hocko wrote: > > > On Fri 27-10-17 13:50:47, Shakeel Butt wrote: > > >> > Why is OOM-disabling a thing? Why isn't this simply a "kill

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Mon 30-10-17 12:28:13, Shakeel Butt wrote: > On Mon, Oct 30, 2017 at 1:29 AM, Michal Hocko wrote: > > On Fri 27-10-17 13:50:47, Shakeel Butt wrote: > >> > Why is OOM-disabling a thing? Why isn't this simply a "kill everything > >> > else before you kill me"? It's crashing

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Mon 30-10-17 12:28:13, Shakeel Butt wrote: > On Mon, Oct 30, 2017 at 1:29 AM, Michal Hocko wrote: > > On Fri 27-10-17 13:50:47, Shakeel Butt wrote: > >> > Why is OOM-disabling a thing? Why isn't this simply a "kill everything > >> > else before you kill me"? It's crashing the kernel in trying

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Mon, Oct 30, 2017 at 1:29 AM, Michal Hocko wrote: > On Fri 27-10-17 13:50:47, Shakeel Butt wrote: >> > Why is OOM-disabling a thing? Why isn't this simply a "kill everything >> > else before you kill me"? It's crashing the kernel in trying to >> > protect a userspace

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Mon, Oct 30, 2017 at 1:29 AM, Michal Hocko wrote: > On Fri 27-10-17 13:50:47, Shakeel Butt wrote: >> > Why is OOM-disabling a thing? Why isn't this simply a "kill everything >> > else before you kill me"? It's crashing the kernel in trying to >> > protect a userspace application. How is that

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Fri 27-10-17 13:50:47, Shakeel Butt wrote: > > Why is OOM-disabling a thing? Why isn't this simply a "kill everything > > else before you kill me"? It's crashing the kernel in trying to > > protect a userspace application. How is that not insane? > > In parallel to other discussion, I think we

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Fri 27-10-17 13:50:47, Shakeel Butt wrote: > > Why is OOM-disabling a thing? Why isn't this simply a "kill everything > > else before you kill me"? It's crashing the kernel in trying to > > protect a userspace application. How is that not insane? > > In parallel to other discussion, I think we

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

> Why is OOM-disabling a thing? Why isn't this simply a "kill everything > else before you kill me"? It's crashing the kernel in trying to > protect a userspace application. How is that not insane? In parallel to other discussion, I think we should definitely move from "completely oom-disabled"

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

> Why is OOM-disabling a thing? Why isn't this simply a "kill everything > else before you kill me"? It's crashing the kernel in trying to > protect a userspace application. How is that not insane? In parallel to other discussion, I think we should definitely move from "completely oom-disabled"

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Thu 26-10-17 12:56:48, Greg Thelen wrote: > Michal Hocko wrote: > > Greg Thelen wrote: > > > So a force charge fallback might be a needed even with oom killer > > > successful > > > invocations. Or we'll need to teach out_of_memory() to return three > > > values > > > (e.g. NO_VICTIM,

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Thu 26-10-17 12:56:48, Greg Thelen wrote: > Michal Hocko wrote: > > Greg Thelen wrote: > > > So a force charge fallback might be a needed even with oom killer > > > successful > > > invocations. Or we'll need to teach out_of_memory() to return three > > > values > > > (e.g. NO_VICTIM,

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

Michal Hocko wrote: > Greg Thelen wrote: > > So a force charge fallback might be a needed even with oom killer successful > > invocations. Or we'll need to teach out_of_memory() to return three values > > (e.g. NO_VICTIM, NEW_VICTIM, PENDING_VICTIM) and try_charge() can loop on > > NEW_VICTIM. >

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

Michal Hocko wrote: > Greg Thelen wrote: > > So a force charge fallback might be a needed even with oom killer successful > > invocations. Or we'll need to teach out_of_memory() to return three values > > (e.g. NO_VICTIM, NEW_VICTIM, PENDING_VICTIM) and try_charge() can loop on > > NEW_VICTIM. >

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Wed, Oct 25, 2017 at 03:49:21PM -0700, Greg Thelen wrote: > Johannes Weiner wrote: > > > On Wed, Oct 25, 2017 at 09:00:57PM +0200, Michal Hocko wrote: > >> On Wed 25-10-17 14:11:06, Johannes Weiner wrote: > >> > "Safe" is a vague term, and it doesn't make much sense to me

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Wed, Oct 25, 2017 at 03:49:21PM -0700, Greg Thelen wrote: > Johannes Weiner wrote: > > > On Wed, Oct 25, 2017 at 09:00:57PM +0200, Michal Hocko wrote: > >> On Wed 25-10-17 14:11:06, Johannes Weiner wrote: > >> > "Safe" is a vague term, and it doesn't make much sense to me in this > >> >

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On 2017/10/26 16:49, Michal Hocko wrote: > On Wed 25-10-17 15:49:21, Greg Thelen wrote: >> Johannes Weiner wrote: >> >>> On Wed, Oct 25, 2017 at 09:00:57PM +0200, Michal Hocko wrote: > [...] So just to make it clear you would be OK with the retry on successful OOM

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On 2017/10/26 16:49, Michal Hocko wrote: > On Wed 25-10-17 15:49:21, Greg Thelen wrote: >> Johannes Weiner wrote: >> >>> On Wed, Oct 25, 2017 at 09:00:57PM +0200, Michal Hocko wrote: > [...] So just to make it clear you would be OK with the retry on successful OOM killer invocation and

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Wed 25-10-17 15:49:21, Greg Thelen wrote: > Johannes Weiner wrote: > > > On Wed, Oct 25, 2017 at 09:00:57PM +0200, Michal Hocko wrote: [...] > >> So just to make it clear you would be OK with the retry on successful > >> OOM killer invocation and force charge on oom

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Wed 25-10-17 15:49:21, Greg Thelen wrote: > Johannes Weiner wrote: > > > On Wed, Oct 25, 2017 at 09:00:57PM +0200, Michal Hocko wrote: [...] > >> So just to make it clear you would be OK with the retry on successful > >> OOM killer invocation and force charge on oom failure, right? > > > >

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

Johannes Weiner wrote: > On Wed, Oct 25, 2017 at 09:00:57PM +0200, Michal Hocko wrote: >> On Wed 25-10-17 14:11:06, Johannes Weiner wrote: >> > "Safe" is a vague term, and it doesn't make much sense to me in this >> > situation. The OOM behavior should be predictable and

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

Johannes Weiner wrote: > On Wed, Oct 25, 2017 at 09:00:57PM +0200, Michal Hocko wrote: >> On Wed 25-10-17 14:11:06, Johannes Weiner wrote: >> > "Safe" is a vague term, and it doesn't make much sense to me in this >> > situation. The OOM behavior should be predictable and consistent. >> > >> >

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Wed, Oct 25, 2017 at 09:00:57PM +0200, Michal Hocko wrote: > On Wed 25-10-17 14:11:06, Johannes Weiner wrote: > > "Safe" is a vague term, and it doesn't make much sense to me in this > > situation. The OOM behavior should be predictable and consistent. > > > > Yes, global might in the rarest

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Wed, Oct 25, 2017 at 09:00:57PM +0200, Michal Hocko wrote: > On Wed 25-10-17 14:11:06, Johannes Weiner wrote: > > "Safe" is a vague term, and it doesn't make much sense to me in this > > situation. The OOM behavior should be predictable and consistent. > > > > Yes, global might in the rarest

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Wed 25-10-17 14:11:06, Johannes Weiner wrote: > On Wed, Oct 25, 2017 at 07:29:24PM +0200, Michal Hocko wrote: > > On Wed 25-10-17 12:44:02, Johannes Weiner wrote: > > > On Wed, Oct 25, 2017 at 04:12:21PM +0200, Michal Hocko wrote: > > > > So how about we start with a BIG FAT WARNING for the

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Wed 25-10-17 14:11:06, Johannes Weiner wrote: > On Wed, Oct 25, 2017 at 07:29:24PM +0200, Michal Hocko wrote: > > On Wed 25-10-17 12:44:02, Johannes Weiner wrote: > > > On Wed, Oct 25, 2017 at 04:12:21PM +0200, Michal Hocko wrote: > > > > So how about we start with a BIG FAT WARNING for the

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Wed, Oct 25, 2017 at 07:29:24PM +0200, Michal Hocko wrote: > On Wed 25-10-17 12:44:02, Johannes Weiner wrote: > > On Wed, Oct 25, 2017 at 04:12:21PM +0200, Michal Hocko wrote: > > > So how about we start with a BIG FAT WARNING for the failure case? > > > Something resembling warn_alloc for the

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Wed, Oct 25, 2017 at 07:29:24PM +0200, Michal Hocko wrote: > On Wed 25-10-17 12:44:02, Johannes Weiner wrote: > > On Wed, Oct 25, 2017 at 04:12:21PM +0200, Michal Hocko wrote: > > > So how about we start with a BIG FAT WARNING for the failure case? > > > Something resembling warn_alloc for the

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Wed 25-10-17 12:44:02, Johannes Weiner wrote: > On Wed, Oct 25, 2017 at 04:12:21PM +0200, Michal Hocko wrote: [...] I yet have to digest the first path of the email but the remaining just sounds we are not on the same page. > > So how about we start with a BIG FAT WARNING for the failure

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Wed 25-10-17 12:44:02, Johannes Weiner wrote: > On Wed, Oct 25, 2017 at 04:12:21PM +0200, Michal Hocko wrote: [...] I yet have to digest the first path of the email but the remaining just sounds we are not on the same page. > > So how about we start with a BIG FAT WARNING for the failure

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Wed, Oct 25, 2017 at 04:12:21PM +0200, Michal Hocko wrote: > On Wed 25-10-17 09:11:51, Johannes Weiner wrote: > > On Wed, Oct 25, 2017 at 09:15:22AM +0200, Michal Hocko wrote: > [...] > > > ... we shouldn't make it more loose though. > > > > Then we can end this discussion right now. I pointed

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Wed, Oct 25, 2017 at 04:12:21PM +0200, Michal Hocko wrote: > On Wed 25-10-17 09:11:51, Johannes Weiner wrote: > > On Wed, Oct 25, 2017 at 09:15:22AM +0200, Michal Hocko wrote: > [...] > > > ... we shouldn't make it more loose though. > > > > Then we can end this discussion right now. I pointed

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Wed 25-10-17 09:11:51, Johannes Weiner wrote: > On Wed, Oct 25, 2017 at 09:15:22AM +0200, Michal Hocko wrote: [...] > > ... we shouldn't make it more loose though. > > Then we can end this discussion right now. I pointed out right from > the start that the only way to replace -ENOMEM with OOM

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Wed 25-10-17 09:11:51, Johannes Weiner wrote: > On Wed, Oct 25, 2017 at 09:15:22AM +0200, Michal Hocko wrote: [...] > > ... we shouldn't make it more loose though. > > Then we can end this discussion right now. I pointed out right from > the start that the only way to replace -ENOMEM with OOM

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Wed, Oct 25, 2017 at 09:15:22AM +0200, Michal Hocko wrote: > On Tue 24-10-17 23:51:30, Greg Thelen wrote: > > Michal Hocko wrote: > [...] > > > I am definitely not pushing that thing right now. It is good to discuss > > > it, though. The more kernel allocations we will track

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Wed, Oct 25, 2017 at 09:15:22AM +0200, Michal Hocko wrote: > On Tue 24-10-17 23:51:30, Greg Thelen wrote: > > Michal Hocko wrote: > [...] > > > I am definitely not pushing that thing right now. It is good to discuss > > > it, though. The more kernel allocations we will track the more careful

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue 24-10-17 23:51:30, Greg Thelen wrote: > Michal Hocko wrote: [...] > > I am definitely not pushing that thing right now. It is good to discuss > > it, though. The more kernel allocations we will track the more careful we > > will have to be. So maybe we will have to

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue 24-10-17 23:51:30, Greg Thelen wrote: > Michal Hocko wrote: [...] > > I am definitely not pushing that thing right now. It is good to discuss > > it, though. The more kernel allocations we will track the more careful we > > will have to be. So maybe we will have to reconsider the current >

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

Michal Hocko wrote: > On Tue 24-10-17 14:58:54, Johannes Weiner wrote: >> On Tue, Oct 24, 2017 at 07:55:58PM +0200, Michal Hocko wrote: >> > On Tue 24-10-17 13:23:30, Johannes Weiner wrote: >> > > On Tue, Oct 24, 2017 at 06:22:13PM +0200, Michal Hocko wrote: >> > [...] >> > >

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

Michal Hocko wrote: > On Tue 24-10-17 14:58:54, Johannes Weiner wrote: >> On Tue, Oct 24, 2017 at 07:55:58PM +0200, Michal Hocko wrote: >> > On Tue 24-10-17 13:23:30, Johannes Weiner wrote: >> > > On Tue, Oct 24, 2017 at 06:22:13PM +0200, Michal Hocko wrote: >> > [...] >> > > > What would

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue 24-10-17 14:58:54, Johannes Weiner wrote: > On Tue, Oct 24, 2017 at 07:55:58PM +0200, Michal Hocko wrote: > > On Tue 24-10-17 13:23:30, Johannes Weiner wrote: > > > On Tue, Oct 24, 2017 at 06:22:13PM +0200, Michal Hocko wrote: > > [...] > > > > What would prevent a runaway in case the only

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue 24-10-17 14:58:54, Johannes Weiner wrote: > On Tue, Oct 24, 2017 at 07:55:58PM +0200, Michal Hocko wrote: > > On Tue 24-10-17 13:23:30, Johannes Weiner wrote: > > > On Tue, Oct 24, 2017 at 06:22:13PM +0200, Michal Hocko wrote: > > [...] > > > > What would prevent a runaway in case the only

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue, Oct 24, 2017 at 07:55:58PM +0200, Michal Hocko wrote: > On Tue 24-10-17 13:23:30, Johannes Weiner wrote: > > On Tue, Oct 24, 2017 at 06:22:13PM +0200, Michal Hocko wrote: > [...] > > > What would prevent a runaway in case the only process in the memcg is > > > oom unkillable then? > > > >

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue, Oct 24, 2017 at 07:55:58PM +0200, Michal Hocko wrote: > On Tue 24-10-17 13:23:30, Johannes Weiner wrote: > > On Tue, Oct 24, 2017 at 06:22:13PM +0200, Michal Hocko wrote: > [...] > > > What would prevent a runaway in case the only process in the memcg is > > > oom unkillable then? > > > >

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue 24-10-17 13:23:30, Johannes Weiner wrote: > On Tue, Oct 24, 2017 at 06:22:13PM +0200, Michal Hocko wrote: [...] > > What would prevent a runaway in case the only process in the memcg is > > oom unkillable then? > > In such a scenario, the page fault handler would busy-loop right now. > >

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue 24-10-17 13:23:30, Johannes Weiner wrote: > On Tue, Oct 24, 2017 at 06:22:13PM +0200, Michal Hocko wrote: [...] > > What would prevent a runaway in case the only process in the memcg is > > oom unkillable then? > > In such a scenario, the page fault handler would busy-loop right now. > >

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue, Oct 24, 2017 at 02:18:59PM +0200, Michal Hocko wrote: > Does this sound something that you would be interested in? I can spend > som more time on it if it is worthwhile. Before you invest too much time in this, I think the rationale for changing the current behavior so far is very weak.

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue, Oct 24, 2017 at 02:18:59PM +0200, Michal Hocko wrote: > Does this sound something that you would be interested in? I can spend > som more time on it if it is worthwhile. Before you invest too much time in this, I think the rationale for changing the current behavior so far is very weak.

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue, Oct 24, 2017 at 06:22:13PM +0200, Michal Hocko wrote: > On Tue 24-10-17 12:06:37, Johannes Weiner wrote: > > >* > > > - * That's why we don't do anything here except remember the > > > - * OOM context and then deal with it at the end of the page > > > - * fault when the stack is

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue, Oct 24, 2017 at 06:22:13PM +0200, Michal Hocko wrote: > On Tue 24-10-17 12:06:37, Johannes Weiner wrote: > > >* > > > - * That's why we don't do anything here except remember the > > > - * OOM context and then deal with it at the end of the page > > > - * fault when the stack is

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue 24-10-17 11:45:11, Johannes Weiner wrote: > On Fri, Oct 13, 2017 at 08:35:55AM +0200, Michal Hocko wrote: > > On Thu 12-10-17 15:03:12, Johannes Weiner wrote: > > > All I'm saying is that, when the syscall-context fails to charge, we > > > should do mem_cgroup_oom() to set up the async OOM

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue 24-10-17 11:45:11, Johannes Weiner wrote: > On Fri, Oct 13, 2017 at 08:35:55AM +0200, Michal Hocko wrote: > > On Thu 12-10-17 15:03:12, Johannes Weiner wrote: > > > All I'm saying is that, when the syscall-context fails to charge, we > > > should do mem_cgroup_oom() to set up the async OOM

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue 24-10-17 12:06:37, Johannes Weiner wrote: > On Fri, Oct 13, 2017 at 05:24:21PM +0200, Michal Hocko wrote: > > Well, it actually occured to me that this would trigger the global oom > > killer in case no memcg specific victim can be found which is definitely > > not something we would like

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue 24-10-17 12:06:37, Johannes Weiner wrote: > On Fri, Oct 13, 2017 at 05:24:21PM +0200, Michal Hocko wrote: > > Well, it actually occured to me that this would trigger the global oom > > killer in case no memcg specific victim can be found which is definitely > > not something we would like

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Fri, Oct 13, 2017 at 05:24:21PM +0200, Michal Hocko wrote: > Well, it actually occured to me that this would trigger the global oom > killer in case no memcg specific victim can be found which is definitely > not something we would like to do. This should work better. I am not > sure we can

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Fri, Oct 13, 2017 at 05:24:21PM +0200, Michal Hocko wrote: > Well, it actually occured to me that this would trigger the global oom > killer in case no memcg specific victim can be found which is definitely > not something we would like to do. This should work better. I am not > sure we can

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Fri, Oct 13, 2017 at 08:35:55AM +0200, Michal Hocko wrote: > On Thu 12-10-17 15:03:12, Johannes Weiner wrote: > > All I'm saying is that, when the syscall-context fails to charge, we > > should do mem_cgroup_oom() to set up the async OOM killer, let the > > charge succeed over the hard limit -

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Fri, Oct 13, 2017 at 08:35:55AM +0200, Michal Hocko wrote: > On Thu 12-10-17 15:03:12, Johannes Weiner wrote: > > All I'm saying is that, when the syscall-context fails to charge, we > > should do mem_cgroup_oom() to set up the async OOM killer, let the > > charge succeed over the hard limit -

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

Does this sound something that you would be interested in? I can spend som more time on it if it is worthwhile. On Fri 13-10-17 17:24:21, Michal Hocko wrote: > Well, it actually occured to me that this would trigger the global oom > killer in case no memcg specific victim can be found which is

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

Does this sound something that you would be interested in? I can spend som more time on it if it is worthwhile. On Fri 13-10-17 17:24:21, Michal Hocko wrote: > Well, it actually occured to me that this would trigger the global oom > killer in case no memcg specific victim can be found which is

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

Well, it actually occured to me that this would trigger the global oom killer in case no memcg specific victim can be found which is definitely not something we would like to do. This should work better. I am not sure we can trigger this corner case but we should cover it and it actually doesn't

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

Well, it actually occured to me that this would trigger the global oom killer in case no memcg specific victim can be found which is definitely not something we would like to do. This should work better. I am not sure we can trigger this corner case but we should cover it and it actually doesn't

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

Just to be explicit what I've had in mind. This hasn't been even compile tested but it should provide at least an idea where I am trying to go.. --- diff --git a/mm/memcontrol.c b/mm/memcontrol.c index d5f3a62887cf..91fa05372114 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1528,26

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

Just to be explicit what I've had in mind. This hasn't been even compile tested but it should provide at least an idea where I am trying to go.. --- diff --git a/mm/memcontrol.c b/mm/memcontrol.c index d5f3a62887cf..91fa05372114 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1528,26

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Thu 12-10-17 16:57:22, Greg Thelen wrote: [...] > Overcharging kmem with deferred reconciliation sounds good to me. > > A few comments (not reasons to avoid this): > > 1) If a task is moved between memcg it seems possible to overcharge >multiple oom memcg for different kmem/user

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Thu 12-10-17 16:57:22, Greg Thelen wrote: [...] > Overcharging kmem with deferred reconciliation sounds good to me. > > A few comments (not reasons to avoid this): > > 1) If a task is moved between memcg it seems possible to overcharge >multiple oom memcg for different kmem/user

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Thu 12-10-17 15:03:12, Johannes Weiner wrote: > On Tue, Oct 10, 2017 at 04:24:34PM +0200, Michal Hocko wrote: [...] > > And we will simply mark the victim MMF_OOM_SKIP and hide it from the oom > > killer if we fail to get the mmap_sem after several attempts. This will > > allow to find a new

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Thu 12-10-17 15:03:12, Johannes Weiner wrote: > On Tue, Oct 10, 2017 at 04:24:34PM +0200, Michal Hocko wrote: [...] > > And we will simply mark the victim MMF_OOM_SKIP and hide it from the oom > > killer if we fail to get the mmap_sem after several attempts. This will > > allow to find a new

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

Johannes Weiner wrote: > On Tue, Oct 10, 2017 at 04:24:34PM +0200, Michal Hocko wrote: >> On Tue 10-10-17 10:17:33, Johannes Weiner wrote: >> > On Tue, Oct 10, 2017 at 11:14:30AM +0200, Michal Hocko wrote: >> > > On Mon 09-10-17 16:26:13, Johannes Weiner wrote: >> > > > It's

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

Johannes Weiner wrote: > On Tue, Oct 10, 2017 at 04:24:34PM +0200, Michal Hocko wrote: >> On Tue 10-10-17 10:17:33, Johannes Weiner wrote: >> > On Tue, Oct 10, 2017 at 11:14:30AM +0200, Michal Hocko wrote: >> > > On Mon 09-10-17 16:26:13, Johannes Weiner wrote: >> > > > It's consistent in the

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue, Oct 10, 2017 at 04:24:34PM +0200, Michal Hocko wrote: > On Tue 10-10-17 10:17:33, Johannes Weiner wrote: > > On Tue, Oct 10, 2017 at 11:14:30AM +0200, Michal Hocko wrote: > > > On Mon 09-10-17 16:26:13, Johannes Weiner wrote: > > > > It's consistent in the sense that only page faults

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue, Oct 10, 2017 at 04:24:34PM +0200, Michal Hocko wrote: > On Tue 10-10-17 10:17:33, Johannes Weiner wrote: > > On Tue, Oct 10, 2017 at 11:14:30AM +0200, Michal Hocko wrote: > > > On Mon 09-10-17 16:26:13, Johannes Weiner wrote: > > > > It's consistent in the sense that only page faults

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue 10-10-17 15:21:53, Shakeel Butt wrote: [...] > On Tue, Oct 10, 2017 at 2:10 AM, Michal Hocko wrote: > > On Mon 09-10-17 20:17:54, Michal Hocko wrote: > >> the primary concern for this patch was whether we really need/want to > >> charge short therm objects which do not

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue 10-10-17 15:21:53, Shakeel Butt wrote: [...] > On Tue, Oct 10, 2017 at 2:10 AM, Michal Hocko wrote: > > On Mon 09-10-17 20:17:54, Michal Hocko wrote: > >> the primary concern for this patch was whether we really need/want to > >> charge short therm objects which do not outlive a single

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Thu, Oct 05, 2017 at 03:21:44PM -0700, Shakeel Butt wrote: > The allocations from filp and names kmem caches can be directly > triggered by user space applications. A buggy application can > consume a significant amount of unaccounted system memory. Though > we have not noticed such buggy

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Thu, Oct 05, 2017 at 03:21:44PM -0700, Shakeel Butt wrote: > The allocations from filp and names kmem caches can be directly > triggered by user space applications. A buggy application can > consume a significant amount of unaccounted system memory. Though > we have not noticed such buggy

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Sun, Oct 8, 2017 at 11:24 PM, Michal Hocko wrote: > On Fri 06-10-17 12:33:03, Shakeel Butt wrote: >> >> names_cachep = kmem_cache_create("names_cache", PATH_MAX, 0, >> >> - SLAB_HWCACHE_ALIGN|SLAB_PANIC, NULL); >> >> +

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Sun, Oct 8, 2017 at 11:24 PM, Michal Hocko wrote: > On Fri 06-10-17 12:33:03, Shakeel Butt wrote: >> >> names_cachep = kmem_cache_create("names_cache", PATH_MAX, 0, >> >> - SLAB_HWCACHE_ALIGN|SLAB_PANIC, NULL); >> >> +

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue 10-10-17 10:17:33, Johannes Weiner wrote: > On Tue, Oct 10, 2017 at 11:14:30AM +0200, Michal Hocko wrote: > > On Mon 09-10-17 16:26:13, Johannes Weiner wrote: > > > It's consistent in the sense that only page faults enable the memcg > > > OOM killer. It's not the type of memory that

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue 10-10-17 10:17:33, Johannes Weiner wrote: > On Tue, Oct 10, 2017 at 11:14:30AM +0200, Michal Hocko wrote: > > On Mon 09-10-17 16:26:13, Johannes Weiner wrote: > > > It's consistent in the sense that only page faults enable the memcg > > > OOM killer. It's not the type of memory that

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue, Oct 10, 2017 at 11:14:30AM +0200, Michal Hocko wrote: > On Mon 09-10-17 16:26:13, Johannes Weiner wrote: > > It's consistent in the sense that only page faults enable the memcg > > OOM killer. It's not the type of memory that decides, it's whether the > > allocation context has a channel

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Tue, Oct 10, 2017 at 11:14:30AM +0200, Michal Hocko wrote: > On Mon 09-10-17 16:26:13, Johannes Weiner wrote: > > It's consistent in the sense that only page faults enable the memcg > > OOM killer. It's not the type of memory that decides, it's whether the > > allocation context has a channel

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Mon 09-10-17 16:26:13, Johannes Weiner wrote: > On Mon, Oct 09, 2017 at 10:52:44AM -0700, Greg Thelen wrote: > > Michal Hocko wrote: > > > > > On Fri 06-10-17 12:33:03, Shakeel Butt wrote: > > >> >> names_cachep = kmem_cache_create("names_cache", PATH_MAX, 0, > > >>

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Mon 09-10-17 16:26:13, Johannes Weiner wrote: > On Mon, Oct 09, 2017 at 10:52:44AM -0700, Greg Thelen wrote: > > Michal Hocko wrote: > > > > > On Fri 06-10-17 12:33:03, Shakeel Butt wrote: > > >> >> names_cachep = kmem_cache_create("names_cache", PATH_MAX, 0, > > >> >> -

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Mon 09-10-17 20:17:54, Michal Hocko wrote: > the primary concern for this patch was whether we really need/want to > charge short therm objects which do not outlive a single syscall. Let me expand on this some more. What is the benefit of kmem accounting of such an object? It cannot stop any

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Mon 09-10-17 20:17:54, Michal Hocko wrote: > the primary concern for this patch was whether we really need/want to > charge short therm objects which do not outlive a single syscall. Let me expand on this some more. What is the benefit of kmem accounting of such an object? It cannot stop any

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Mon, Oct 09, 2017 at 10:52:44AM -0700, Greg Thelen wrote: > Michal Hocko wrote: > > > On Fri 06-10-17 12:33:03, Shakeel Butt wrote: > >> >> names_cachep = kmem_cache_create("names_cache", PATH_MAX, 0, > >> >> - SLAB_HWCACHE_ALIGN|SLAB_PANIC, NULL);

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Mon, Oct 09, 2017 at 10:52:44AM -0700, Greg Thelen wrote: > Michal Hocko wrote: > > > On Fri 06-10-17 12:33:03, Shakeel Butt wrote: > >> >> names_cachep = kmem_cache_create("names_cache", PATH_MAX, 0, > >> >> - SLAB_HWCACHE_ALIGN|SLAB_PANIC, NULL); > >> >> +

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Mon 09-10-17 20:04:09, Michal Hocko wrote: > [CC Johannes - the thread starts > http://lkml.kernel.org/r/20171005222144.123797-1-shake...@google.com] > > On Mon 09-10-17 10:52:44, Greg Thelen wrote: [...] > > A few ideas on how to make it more flexible: > > > > a) Go back to memcg oom killing

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Mon 09-10-17 20:04:09, Michal Hocko wrote: > [CC Johannes - the thread starts > http://lkml.kernel.org/r/20171005222144.123797-1-shake...@google.com] > > On Mon 09-10-17 10:52:44, Greg Thelen wrote: [...] > > A few ideas on how to make it more flexible: > > > > a) Go back to memcg oom killing

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

[CC Johannes - the thread starts http://lkml.kernel.org/r/20171005222144.123797-1-shake...@google.com] On Mon 09-10-17 10:52:44, Greg Thelen wrote: > Michal Hocko wrote: > > > On Fri 06-10-17 12:33:03, Shakeel Butt wrote: > >> >> names_cachep =

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

[CC Johannes - the thread starts http://lkml.kernel.org/r/20171005222144.123797-1-shake...@google.com] On Mon 09-10-17 10:52:44, Greg Thelen wrote: > Michal Hocko wrote: > > > On Fri 06-10-17 12:33:03, Shakeel Butt wrote: > >> >> names_cachep = kmem_cache_create("names_cache", PATH_MAX,

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

Michal Hocko wrote: > On Fri 06-10-17 12:33:03, Shakeel Butt wrote: >> >> names_cachep = kmem_cache_create("names_cache", PATH_MAX, 0, >> >> - SLAB_HWCACHE_ALIGN|SLAB_PANIC, NULL); >> >> +

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

Michal Hocko wrote: > On Fri 06-10-17 12:33:03, Shakeel Butt wrote: >> >> names_cachep = kmem_cache_create("names_cache", PATH_MAX, 0, >> >> - SLAB_HWCACHE_ALIGN|SLAB_PANIC, NULL); >> >> + SLAB_HWCACHE_ALIGN|SLAB_PANIC|SLAB_ACCOUNT, NULL); >> > >> >

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Fri 06-10-17 12:33:03, Shakeel Butt wrote: > >> names_cachep = kmem_cache_create("names_cache", PATH_MAX, 0, > >> - SLAB_HWCACHE_ALIGN|SLAB_PANIC, NULL); > >> + SLAB_HWCACHE_ALIGN|SLAB_PANIC|SLAB_ACCOUNT, NULL); > > > > I might be wrong but isn't

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

On Fri 06-10-17 12:33:03, Shakeel Butt wrote: > >> names_cachep = kmem_cache_create("names_cache", PATH_MAX, 0, > >> - SLAB_HWCACHE_ALIGN|SLAB_PANIC, NULL); > >> + SLAB_HWCACHE_ALIGN|SLAB_PANIC|SLAB_ACCOUNT, NULL); > > > > I might be wrong but isn't

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

>> names_cachep = kmem_cache_create("names_cache", PATH_MAX, 0, >> - SLAB_HWCACHE_ALIGN|SLAB_PANIC, NULL); >> + SLAB_HWCACHE_ALIGN|SLAB_PANIC|SLAB_ACCOUNT, NULL); > > I might be wrong but isn't name cache only holding temporary objects > used for path

Re: [PATCH] fs, mm: account filp and names caches to kmemcg

>> names_cachep = kmem_cache_create("names_cache", PATH_MAX, 0, >> - SLAB_HWCACHE_ALIGN|SLAB_PANIC, NULL); >> + SLAB_HWCACHE_ALIGN|SLAB_PANIC|SLAB_ACCOUNT, NULL); > > I might be wrong but isn't name cache only holding temporary objects > used for path

  1   2   >