Re: [PATCH] mm: idle-page: fix oops because end_pfn is larger than max_pfn

; > cc Vladimir. This seems rather obvious - I'm wondering if the code was > that way for some subtle reason? No subtle reason at all - just a bug. The patch looks good to me, Acked-by: Vladimir Davydov

Re: [PATCH v7 10/10] mm: reparent memcg kmem_caches on cgroup removal

used anywhere except count_shadow_nodes(). But even there it > won't break anything: after reparenting "nodes" will be 0 on child > level (because we're already reparenting shrinker lists), and on > parent level page stats always were 0, and this patch won't change > anything. > > Signed-off-by: Roman Gushchin Acked-by: Vladimir Davydov

Re: [PATCH v7 07/10] mm: synchronize access to kmem_cache dying flag using a spinlock

the irq context, > which will be required in order to implement asynchronous release > of kmem_caches. > > So let's switch over to the irq-save flavor of the spinlock-based > synchronization. > > Signed-off-by: Roman Gushchin Acked-by: Vladimir Davydov

Re: [PATCH v7 06/10] mm: don't check the dying flag on kmem_cache creation

scheduled after the flag is set. And if it was > scheduled before, flush_memcg_workqueue() will wait for it anyway. > > So let's drop this check to simplify the code. > > Signed-off-by: Roman Gushchin Acked-by: Vladimir Davydov

Re: [PATCH v6 10/10] mm: reparent slab memory on cgroup removal

On Tue, Jun 04, 2019 at 07:44:54PM -0700, Roman Gushchin wrote: > Let's reparent memcg slab memory on memcg offlining. This allows us > to release the memory cgroup without waiting for the last outstanding > kernel object (e.g. dentry used by another application). > > So instead of reparenting

Re: [PATCH v6 09/10] mm: stop setting page->mem_cgroup pointer for slab pages

cgroup from going away. Instead rely on kmem_cache > as an intermediate object. > > Make sure that vmstats and shrinker lists are working as previously, > as well as /proc/kpagecgroup interface. > > Signed-off-by: Roman Gushchin Acked-by: Vladimir Davydov

Re: [PATCH v6 08/10] mm: rework non-root kmem_cache lifecycle management

0m0.181s > sys 0m0.824ssys 0m0.864s > > real 0m1.350sreal 0m1.295s > user 0m0.200suser0m0.190s > sys 0m0.842ssys 0m0.811s > > So it looks like the difference is not noticeable in this test. > > Signed-off-by: Roman Gushchin Acked-by: Vladimir Davydov

Re: [PATCH v6 07/10] mm: synchronize access to kmem_cache dying flag using a spinlock

On Tue, Jun 04, 2019 at 07:44:51PM -0700, Roman Gushchin wrote: > Currently the memcg_params.dying flag and the corresponding > workqueue used for the asynchronous deactivation of kmem_caches > is synchronized using the slab_mutex. > > It makes impossible to check this flag from the irq context,

Re: [PATCH v6 05/10] mm: introduce __memcg_kmem_uncharge_memcg()

ter using a new > memcg_kmem_uncharge_memcg() wrapper, which calls > __memcg_kmem_uncharge_memcg() if memcg_kmem_enabled() > check is passed. > > Signed-off-by: Roman Gushchin > Reviewed-by: Shakeel Butt Acked-by: Vladimir Davydov

Re: [PATCH v6 04/10] mm: generalize postponed non-root kmem_cache deactivation

act_after_rcu() SLUB-only > > For consistency, all allocator-specific functions start with "__". > > Signed-off-by: Roman Gushchin Much easier to review after extracting renaming, thanks. Acked-by: Vladimir Davydov

Re: [PATCH v6 03/10] mm: rename slab delayed deactivation functions and fields

act_work -> work > > And RCU/delayed work callbacks in slab common code: > kmemcg_deactivate_rcufn -> kmemcg_rcufn > kmemcg_deactivate_workfn -> kmemcg_workfn > > This patch contains no functional changes, only renamings. > > Signed-off-by: Roman Gushchin Acked-by: Vladimir Davydov

Re: [PATCH v6 01/10] mm: add missing smp read barrier on getting memcg kmem_cache pointer

On Tue, Jun 04, 2019 at 07:44:45PM -0700, Roman Gushchin wrote: > Johannes noticed that reading the memcg kmem_cache pointer in > cache_from_memcg_idx() is performed using READ_ONCE() macro, > which doesn't implement a SMP barrier, which is required > by the logic. > > Add a proper smp_rmb() to

Re: [PATCH v5 6/7] mm: reparent slab memory on cgroup removal

On Tue, May 28, 2019 at 07:58:17PM +, Roman Gushchin wrote: > It looks like outstanding questions are: > 1) synchronization around the dying flag > 2) removing CONFIG_SLOB in 2/7 > 3) early sysfs_slab_remove() > 4) mem_cgroup_from_kmem in 7/7 > > Please, let me know if I missed anything.

Re: [PATCH v5 6/7] mm: reparent slab memory on cgroup removal

On Tue, May 21, 2019 at 01:07:34PM -0700, Roman Gushchin wrote: > Let's reparent memcg slab memory on memcg offlining. This allows us > to release the memory cgroup without waiting for the last outstanding > kernel object (e.g. dentry used by another application). > > So instead of reparenting

Re: [PATCH v5 5/7] mm: rework non-root kmem_cache lifecycle management

On Tue, May 28, 2019 at 08:08:28PM +0300, Vladimir Davydov wrote: > Hello Roman, > > On Tue, May 21, 2019 at 01:07:33PM -0700, Roman Gushchin wrote: > > This commit makes several important changes in the lifecycle > > of a non-root kmem_cache, which also affect the life

Re: [PATCH v5 5/7] mm: rework non-root kmem_cache lifecycle management

On Tue, May 28, 2019 at 01:37:50PM -0400, Waiman Long wrote: > On 5/28/19 1:08 PM, Vladimir Davydov wrote: > >> static void flush_memcg_workqueue(struct kmem_cache *s) > >> { > >> + /* > >> + * memcg_params.dying is synchronized using slab_mutex AND

Re: [PATCH v5 7/7] mm: fix /proc/kpagecgroup interface for slab pages

On Tue, May 21, 2019 at 01:07:35PM -0700, Roman Gushchin wrote: > Switching to an indirect scheme of getting mem_cgroup pointer for > !root slab pages broke /proc/kpagecgroup interface for them. > > Let's fix it by learning page_cgroup_ino() how to get memcg > pointer for slab pages. > >

Re: [PATCH v5 1/7] mm: postpone kmem_cache memcg pointer initialization to memcg_link_cache()

> > For non-root kmem_caches memcg_link_cache() is always called > before the kmem_cache becomes visible to a user, so it's safe. > > Signed-off-by: Roman Gushchin > Reviewed-by: Shakeel Butt This one is a no-brainer. Acked-by: Vladimir Davydov Provided it's necessary for the rest of the series.

Re: [PATCH v5 4/7] mm: unify SLAB and SLUB page accounting

al and memcg-aware) and memcg charging. > So they are replacing direct memcg_(un)charge_slab() calls. > > Signed-off-by: Roman Gushchin > Reviewed-by: Shakeel Butt > Acked-by: Christoph Lameter Makes sense even without the rest of the series, Acked-by: Vladimir Davydov

Re: [PATCH v5 2/7] mm: generalize postponed non-root kmem_cache deactivation

On Tue, May 21, 2019 at 01:07:30PM -0700, Roman Gushchin wrote: > diff --git a/mm/slab_common.c b/mm/slab_common.c > index 6e00bdf8618d..4e5b4292a763 100644 > --- a/mm/slab_common.c > +++ b/mm/slab_common.c > @@ -866,11 +859,12 @@ static void flush_memcg_workqueue(struct kmem_cache *s) >

Re: [PATCH v5 5/7] mm: rework non-root kmem_cache lifecycle management

Hello Roman, On Tue, May 21, 2019 at 01:07:33PM -0700, Roman Gushchin wrote: > This commit makes several important changes in the lifecycle > of a non-root kmem_cache, which also affect the lifecycle > of a memory cgroup. > > Currently each charged slab page has a page->mem_cgroup pointer > to

Re: [PATCH] memcg: make it work on sparse non-0-node systems

On Fri, May 17, 2019 at 06:48:37AM +0200, Jiri Slaby wrote: > On 16. 05. 19, 15:59, Michal Hocko wrote: > >> However, I tend to agree with Michal that (ab)using node[0].memcg_lrus > >> to check if a list_lru is memcg aware looks confusing. I guess we could > >> simply add a bool flag to list_lru

Re: [PATCH] memcg: make it work on sparse non-0-node systems

; > The root cause are list_lru_memcg_aware checks in the list_lru code. > The test in list_lru_memcg_aware is broken: it assumes node 0 is always > present, but it is not true on some systems as can be seen above. > > So fix this by checking the first online node instead of node 0

Re: [PATCH 0/5] mm: reparent slab memory on cgroup removal

Hello Roman, On Wed, Apr 17, 2019 at 02:54:29PM -0700, Roman Gushchin wrote: > There is however a significant problem with reparenting of slab memory: > there is no list of charged pages. Some of them are in shrinker lists, > but not all. Introducing of a new list is really not an option. True,

Re: [PATCH] mm: slowly shrink slabs with a relatively small number of objects

On Tue, Sep 04, 2018 at 10:52:46AM -0700, Roman Gushchin wrote: > Reparenting of all pages is definitely an option to consider, Reparenting pages would be great indeed, but I'm not sure we could do that, because we'd have to walk over page lists of semi-active kmem caches and do it consistently

Re: [PATCH] mm: slowly shrink slabs with a relatively small number of objects

On Tue, Sep 04, 2018 at 10:52:46AM -0700, Roman Gushchin wrote: > Reparenting of all pages is definitely an option to consider, Reparenting pages would be great indeed, but I'm not sure we could do that, because we'd have to walk over page lists of semi-active kmem caches and do it consistently

Re: [PATCH] memcg: Remove memcg_cgroup::id from IDR on mem_cgroup_css_alloc() failure

On Wed, Aug 01, 2018 at 11:55:52AM -0400, Johannes Weiner wrote: > On Tue, Jul 31, 2018 at 04:39:08PM -0700, Andrew Morton wrote: > > On Mon, 30 Jul 2018 11:31:13 -0400 Johannes Weiner > > wrote: > > > > > Subject: [PATCH] mm: memcontrol: simplify memcg idr allocation and error > > > unwinding

Re: [PATCH] memcg: Remove memcg_cgroup::id from IDR on mem_cgroup_css_alloc() failure

On Wed, Aug 01, 2018 at 11:55:52AM -0400, Johannes Weiner wrote: > On Tue, Jul 31, 2018 at 04:39:08PM -0700, Andrew Morton wrote: > > On Mon, 30 Jul 2018 11:31:13 -0400 Johannes Weiner > > wrote: > > > > > Subject: [PATCH] mm: memcontrol: simplify memcg idr allocation and error > > > unwinding

Re: [PATCH] memcg: Remove memcg_cgroup::id from IDR on mem_cgroup_css_alloc() failure

On Mon, Jul 30, 2018 at 11:31:13AM -0400, Johannes Weiner wrote: > On Sun, Jul 29, 2018 at 10:26:21PM +0300, Vladimir Davydov wrote: > > On Fri, Jul 27, 2018 at 03:31:34PM -0400, Johannes Weiner wrote: > > > That said, the lifetime of the root reference on the ID is the online &

Re: [PATCH] memcg: Remove memcg_cgroup::id from IDR on mem_cgroup_css_alloc() failure

On Mon, Jul 30, 2018 at 11:31:13AM -0400, Johannes Weiner wrote: > On Sun, Jul 29, 2018 at 10:26:21PM +0300, Vladimir Davydov wrote: > > On Fri, Jul 27, 2018 at 03:31:34PM -0400, Johannes Weiner wrote: > > > That said, the lifetime of the root reference on the ID is the online &

Re: [PATCH] memcg: Remove memcg_cgroup::id from IDR on mem_cgroup_css_alloc() failure

On Fri, Jul 27, 2018 at 03:31:34PM -0400, Johannes Weiner wrote: > That said, the lifetime of the root reference on the ID is the online > state, we put that in css_offline. Is there a reason we need to have > the ID ready and the memcg in the IDR before onlining it? I fail to see any reason for

Re: [PATCH] memcg: Remove memcg_cgroup::id from IDR on mem_cgroup_css_alloc() failure

On Fri, Jul 27, 2018 at 03:31:34PM -0400, Johannes Weiner wrote: > That said, the lifetime of the root reference on the ID is the online > state, we put that in css_offline. Is there a reason we need to have > the ID ready and the memcg in the IDR before onlining it? I fail to see any reason for

Re: [PATCH v8 05/17] mm: Assign memcg-aware shrinkers bitmap to memcg

On Thu, Jul 05, 2018 at 03:10:30PM -0700, Andrew Morton wrote: > On Wed, 4 Jul 2018 18:51:12 +0300 Kirill Tkhai wrote: > > > > - why aren't we decreasing shrinker_nr_max in > > > unregister_memcg_shrinker()? That's easy to do, avoids pointless > > > work in shrink_slab_memcg() and avoids

Re: [PATCH v8 05/17] mm: Assign memcg-aware shrinkers bitmap to memcg

On Thu, Jul 05, 2018 at 03:10:30PM -0700, Andrew Morton wrote: > On Wed, 4 Jul 2018 18:51:12 +0300 Kirill Tkhai wrote: > > > > - why aren't we decreasing shrinker_nr_max in > > > unregister_memcg_shrinker()? That's easy to do, avoids pointless > > > work in shrink_slab_memcg() and avoids

Re: [PATCH v8 05/17] mm: Assign memcg-aware shrinkers bitmap to memcg

On Tue, Jul 03, 2018 at 01:50:00PM -0700, Andrew Morton wrote: > On Tue, 03 Jul 2018 18:09:26 +0300 Kirill Tkhai wrote: > > > Imagine a big node with many cpus, memory cgroups and containers. > > Let we have 200 containers, every container has 10 mounts, > > and 10 cgroups. All container tasks

Re: [PATCH v8 05/17] mm: Assign memcg-aware shrinkers bitmap to memcg

On Tue, Jul 03, 2018 at 01:50:00PM -0700, Andrew Morton wrote: > On Tue, 03 Jul 2018 18:09:26 +0300 Kirill Tkhai wrote: > > > Imagine a big node with many cpus, memory cgroups and containers. > > Let we have 200 containers, every container has 10 mounts, > > and 10 cgroups. All container tasks

Re: [PATCH v4] mm: fix race between kmem_cache destroy, create and deactivate

k the cache as > dying and flush the workqueue used for memcg kmem cache creation and > deactivation. SLUB's memcg kmem cache deactivation also includes RCU > callback and thus make sure all previous registered RCU callbacks > have completed as well. > > Signed-off-by: Shakeel Butt Acked-by: Vladimir Davydov Thanks.

Re: [PATCH v4] mm: fix race between kmem_cache destroy, create and deactivate

k the cache as > dying and flush the workqueue used for memcg kmem cache creation and > deactivation. SLUB's memcg kmem cache deactivation also includes RCU > callback and thus make sure all previous registered RCU callbacks > have completed as well. > > Signed-off-by: Shakeel Butt Acked-by: Vladimir Davydov Thanks.

Re: [PATCH v3] mm: fix race between kmem_cache destroy, create and deactivate

On Tue, May 29, 2018 at 05:12:04PM -0700, Shakeel Butt wrote: > The memcg kmem cache creation and deactivation (SLUB only) is > asynchronous. If a root kmem cache is destroyed whose memcg cache is in > the process of creation or deactivation, the kernel may crash. > > Example of one such crash: >

Re: [PATCH v3] mm: fix race between kmem_cache destroy, create and deactivate

On Tue, May 29, 2018 at 05:12:04PM -0700, Shakeel Butt wrote: > The memcg kmem cache creation and deactivation (SLUB only) is > asynchronous. If a root kmem cache is destroyed whose memcg cache is in > the process of creation or deactivation, the kernel may crash. > > Example of one such crash: >

Re: [PATCH v2] mm: fix race between kmem_cache destroy, create and deactivate

On Tue, May 22, 2018 at 01:13:36PM -0700, Shakeel Butt wrote: > The memcg kmem cache creation and deactivation (SLUB only) is > asynchronous. If a root kmem cache is destroyed whose memcg cache is in > the process of creation or deactivation, the kernel may crash. > > Example of one such crash: >

Re: [PATCH v2] mm: fix race between kmem_cache destroy, create and deactivate

On Tue, May 22, 2018 at 01:13:36PM -0700, Shakeel Butt wrote: > The memcg kmem cache creation and deactivation (SLUB only) is > asynchronous. If a root kmem cache is destroyed whose memcg cache is in > the process of creation or deactivation, the kernel may crash. > > Example of one such crash: >

Re: [PATCH] memcg: force charge kmem counter too

On Fri, May 25, 2018 at 11:55:01AM -0700, Shakeel Butt wrote: > Based on several conditions the kernel can decide to force charge an > allocation for a memcg i.e. overcharge memcg->memory and memcg->memsw > counters. Do the same for memcg->kmem counter too. In cgroup-v1, this > bug can cause a

Re: [PATCH] memcg: force charge kmem counter too

On Fri, May 25, 2018 at 11:55:01AM -0700, Shakeel Butt wrote: > Based on several conditions the kernel can decide to force charge an > allocation for a memcg i.e. overcharge memcg->memory and memcg->memsw > counters. Do the same for memcg->kmem counter too. In cgroup-v1, this > bug can cause a

Re: [PATCH v7 00/17] Improve shrink_slab() scalability (old complexity was O(n^2), new is O(n))

Hello Kirill, The whole patch set looks good to me now. Acked-by: Vladimir Davydov <vdavydov@gmail.com> Thanks, Vladimir On Tue, May 22, 2018 at 01:07:10PM +0300, Kirill Tkhai wrote: > Hi, > > this patches solves the problem with slow shrink_slab() occuring > on the ma

Re: [PATCH v7 00/17] Improve shrink_slab() scalability (old complexity was O(n^2), new is O(n))

Hello Kirill, The whole patch set looks good to me now. Acked-by: Vladimir Davydov Thanks, Vladimir On Tue, May 22, 2018 at 01:07:10PM +0300, Kirill Tkhai wrote: > Hi, > > this patches solves the problem with slow shrink_slab() occuring > on the machines having many shrinker

Re: [PATCH v6 05/17] mm: Assign memcg-aware shrinkers bitmap to memcg

On Mon, May 21, 2018 at 01:16:40PM +0300, Kirill Tkhai wrote: > >> +static int memcg_expand_one_shrinker_map(struct mem_cgroup *memcg, > >> + int size, int old_size) > > > > Nit: No point in passing old_size here. You can instead use > > memcg_shrinker_map_size

Re: [PATCH v6 05/17] mm: Assign memcg-aware shrinkers bitmap to memcg

On Mon, May 21, 2018 at 01:16:40PM +0300, Kirill Tkhai wrote: > >> +static int memcg_expand_one_shrinker_map(struct mem_cgroup *memcg, > >> + int size, int old_size) > > > > Nit: No point in passing old_size here. You can instead use > > memcg_shrinker_map_size

Re: [PATCH v6 12/17] mm: Set bit in memcg shrinker bitmap on first list_lru item apearance

On Mon, May 21, 2018 at 12:31:34PM +0300, Kirill Tkhai wrote: > On 20.05.2018 10:55, Vladimir Davydov wrote: > > On Fri, May 18, 2018 at 11:43:42AM +0300, Kirill Tkhai wrote: > >> Introduce set_shrinker_bit() function to set shrinker-related > >> bit in memcg shrinker bi

Re: [PATCH v6 12/17] mm: Set bit in memcg shrinker bitmap on first list_lru item apearance

On Mon, May 21, 2018 at 12:31:34PM +0300, Kirill Tkhai wrote: > On 20.05.2018 10:55, Vladimir Davydov wrote: > > On Fri, May 18, 2018 at 11:43:42AM +0300, Kirill Tkhai wrote: > >> Introduce set_shrinker_bit() function to set shrinker-related > >> bit in memcg shrinker bi

Re: [PATCH v6 14/17] mm: Iterate only over charged shrinkers during memcg shrink_slab()

On Mon, May 21, 2018 at 12:17:07PM +0300, Kirill Tkhai wrote: > >> +static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid, > >> + struct mem_cgroup *memcg, int priority) > >> +{ > >> + struct memcg_shrinker_map *map; > >> + unsigned long freed = 0; > >> + int ret, i; >

Re: [PATCH v6 14/17] mm: Iterate only over charged shrinkers during memcg shrink_slab()

On Mon, May 21, 2018 at 12:17:07PM +0300, Kirill Tkhai wrote: > >> +static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid, > >> + struct mem_cgroup *memcg, int priority) > >> +{ > >> + struct memcg_shrinker_map *map; > >> + unsigned long freed = 0; > >> + int ret, i; >

Re: [PATCH v6 15/17] mm: Generalize shrink_slab() calls in shrink_node()

On Fri, May 18, 2018 at 11:44:11AM +0300, Kirill Tkhai wrote: > From: Vladimir Davydov <vdavydov@gmail.com> > > The patch makes shrink_slab() be called for root_mem_cgroup > in the same way as it's called for the rest of cgroups. > This simplifies the logic and imp

Re: [PATCH v6 15/17] mm: Generalize shrink_slab() calls in shrink_node()

On Fri, May 18, 2018 at 11:44:11AM +0300, Kirill Tkhai wrote: > From: Vladimir Davydov > > The patch makes shrink_slab() be called for root_mem_cgroup > in the same way as it's called for the rest of cgroups. > This simplifies the logic and improves the readability. > > Si

Re: [PATCH v6 14/17] mm: Iterate only over charged shrinkers during memcg shrink_slab()

On Fri, May 18, 2018 at 11:44:01AM +0300, Kirill Tkhai wrote: > Using the preparations made in previous patches, in case of memcg > shrink, we may avoid shrinkers, which are not set in memcg's shrinkers > bitmap. To do that, we separate iterations over memcg-aware and > !memcg-aware shrinkers, and

Re: [PATCH v6 14/17] mm: Iterate only over charged shrinkers during memcg shrink_slab()

On Fri, May 18, 2018 at 11:44:01AM +0300, Kirill Tkhai wrote: > Using the preparations made in previous patches, in case of memcg > shrink, we may avoid shrinkers, which are not set in memcg's shrinkers > bitmap. To do that, we separate iterations over memcg-aware and > !memcg-aware shrinkers, and

Re: [PATCH v6 13/17] mm: Export mem_cgroup_is_root()

On Fri, May 18, 2018 at 11:43:53AM +0300, Kirill Tkhai wrote: > This will be used in next patch. > > Signed-off-by: Kirill Tkhai > --- > include/linux/memcontrol.h | 10 ++ > mm/memcontrol.c|5 - > 2 files changed, 10 insertions(+), 5

Re: [PATCH v6 13/17] mm: Export mem_cgroup_is_root()

On Fri, May 18, 2018 at 11:43:53AM +0300, Kirill Tkhai wrote: > This will be used in next patch. > > Signed-off-by: Kirill Tkhai > --- > include/linux/memcontrol.h | 10 ++ > mm/memcontrol.c|5 - > 2 files changed, 10 insertions(+), 5 deletions(-) > > diff --git

Re: [PATCH v6 12/17] mm: Set bit in memcg shrinker bitmap on first list_lru item apearance

On Fri, May 18, 2018 at 11:43:42AM +0300, Kirill Tkhai wrote: > Introduce set_shrinker_bit() function to set shrinker-related > bit in memcg shrinker bitmap, and set the bit after the first > item is added and in case of reparenting destroyed memcg's items. > > This will allow next patch to make

Re: [PATCH v6 12/17] mm: Set bit in memcg shrinker bitmap on first list_lru item apearance

On Fri, May 18, 2018 at 11:43:42AM +0300, Kirill Tkhai wrote: > Introduce set_shrinker_bit() function to set shrinker-related > bit in memcg shrinker bitmap, and set the bit after the first > item is added and in case of reparenting destroyed memcg's items. > > This will allow next patch to make

Re: [PATCH v6 05/17] mm: Assign memcg-aware shrinkers bitmap to memcg

On Fri, May 18, 2018 at 11:42:37AM +0300, Kirill Tkhai wrote: > Imagine a big node with many cpus, memory cgroups and containers. > Let we have 200 containers, every container has 10 mounts, > and 10 cgroups. All container tasks don't touch foreign > containers mounts. If there is intensive pages

Re: [PATCH v6 05/17] mm: Assign memcg-aware shrinkers bitmap to memcg

On Fri, May 18, 2018 at 11:42:37AM +0300, Kirill Tkhai wrote: > Imagine a big node with many cpus, memory cgroups and containers. > Let we have 200 containers, every container has 10 mounts, > and 10 cgroups. All container tasks don't touch foreign > containers mounts. If there is intensive pages

Re: [PATCH v6 03/17] mm: Assign id to every memcg-aware shrinker

Hello Kirill, Generally, all patches in the series look OK to me, but I'm going to do some nitpicking before I ack them. See below. On Fri, May 18, 2018 at 11:42:08AM +0300, Kirill Tkhai wrote: > The patch introduces shrinker::id number, which is used to enumerate > memcg-aware shrinkers. The

Re: [PATCH v6 03/17] mm: Assign id to every memcg-aware shrinker

Hello Kirill, Generally, all patches in the series look OK to me, but I'm going to do some nitpicking before I ack them. See below. On Fri, May 18, 2018 at 11:42:08AM +0300, Kirill Tkhai wrote: > The patch introduces shrinker::id number, which is used to enumerate > memcg-aware shrinkers. The

Re: [PATCH v5 11/13] mm: Iterate only over charged shrinkers during memcg shrink_slab()

On Thu, May 17, 2018 at 02:49:26PM +0300, Kirill Tkhai wrote: > On 17.05.2018 07:16, Vladimir Davydov wrote: > > On Tue, May 15, 2018 at 05:49:59PM +0300, Kirill Tkhai wrote: > >>>> @@ -589,13 +647,7 @@ static unsigned long shrink_slab(gfp_t g

Re: [PATCH v5 11/13] mm: Iterate only over charged shrinkers during memcg shrink_slab()

On Thu, May 17, 2018 at 02:49:26PM +0300, Kirill Tkhai wrote: > On 17.05.2018 07:16, Vladimir Davydov wrote: > > On Tue, May 15, 2018 at 05:49:59PM +0300, Kirill Tkhai wrote: > >>>> @@ -589,13 +647,7 @@ static unsigned long shrink_slab(gfp_t g

Re: [PATCH v5 13/13] mm: Clear shrinker bit if there are no objects related to memcg

On Tue, May 15, 2018 at 11:55:04AM +0300, Kirill Tkhai wrote: > >> @@ -586,8 +586,23 @@ static unsigned long shrink_slab_memcg(gfp_t > >> gfp_mask, int nid, > >>continue; > >> > >>ret = do_shrink_slab(, shrinker, priority); > >> - if (ret ==

Re: [PATCH v5 13/13] mm: Clear shrinker bit if there are no objects related to memcg

On Tue, May 15, 2018 at 11:55:04AM +0300, Kirill Tkhai wrote: > >> @@ -586,8 +586,23 @@ static unsigned long shrink_slab_memcg(gfp_t > >> gfp_mask, int nid, > >>continue; > >> > >>ret = do_shrink_slab(, shrinker, priority); > >> - if (ret ==

Re: [PATCH v5 11/13] mm: Iterate only over charged shrinkers during memcg shrink_slab()

On Tue, May 15, 2018 at 01:12:20PM +0300, Kirill Tkhai wrote: > >> +#define root_mem_cgroup NULL > > > > Let's instead export mem_cgroup_is_root(). In case if MEMCG is disabled > > it will always return false. > > export == move to header file That and adding a stub function in case !MEMCG. >

Re: [PATCH v5 11/13] mm: Iterate only over charged shrinkers during memcg shrink_slab()

On Tue, May 15, 2018 at 01:12:20PM +0300, Kirill Tkhai wrote: > >> +#define root_mem_cgroup NULL > > > > Let's instead export mem_cgroup_is_root(). In case if MEMCG is disabled > > it will always return false. > > export == move to header file That and adding a stub function in case !MEMCG. >

Re: [PATCH v5 11/13] mm: Iterate only over charged shrinkers during memcg shrink_slab()

On Tue, May 15, 2018 at 05:49:59PM +0300, Kirill Tkhai wrote: > >> @@ -589,13 +647,7 @@ static unsigned long shrink_slab(gfp_t gfp_mask, int > >> nid, > >>.memcg = memcg, > >>}; > >> > >> - /* > >> - * If kernel memory accounting is disabled,

Re: [PATCH v5 11/13] mm: Iterate only over charged shrinkers during memcg shrink_slab()

On Tue, May 15, 2018 at 05:49:59PM +0300, Kirill Tkhai wrote: > >> @@ -589,13 +647,7 @@ static unsigned long shrink_slab(gfp_t gfp_mask, int > >> nid, > >>.memcg = memcg, > >>}; > >> > >> - /* > >> - * If kernel memory accounting is disabled,

Re: [PATCH v5 13/13] mm: Clear shrinker bit if there are no objects related to memcg

On Thu, May 10, 2018 at 12:54:15PM +0300, Kirill Tkhai wrote: > To avoid further unneed calls of do_shrink_slab() > for shrinkers, which already do not have any charged > objects in a memcg, their bits have to be cleared. > > This patch introduces a lockless mechanism to do that > without races

Re: [PATCH v5 13/13] mm: Clear shrinker bit if there are no objects related to memcg

On Thu, May 10, 2018 at 12:54:15PM +0300, Kirill Tkhai wrote: > To avoid further unneed calls of do_shrink_slab() > for shrinkers, which already do not have any charged > objects in a memcg, their bits have to be cleared. > > This patch introduces a lockless mechanism to do that > without races

Re: [PATCH v5 11/13] mm: Iterate only over charged shrinkers during memcg shrink_slab()

On Thu, May 10, 2018 at 12:53:55PM +0300, Kirill Tkhai wrote: > Using the preparations made in previous patches, in case of memcg > shrink, we may avoid shrinkers, which are not set in memcg's shrinkers > bitmap. To do that, we separate iterations over memcg-aware and > !memcg-aware shrinkers, and

Re: [PATCH v5 11/13] mm: Iterate only over charged shrinkers during memcg shrink_slab()

On Thu, May 10, 2018 at 12:53:55PM +0300, Kirill Tkhai wrote: > Using the preparations made in previous patches, in case of memcg > shrink, we may avoid shrinkers, which are not set in memcg's shrinkers > bitmap. To do that, we separate iterations over memcg-aware and > !memcg-aware shrinkers, and

Re: [PATCH v5 10/13] mm: Set bit in memcg shrinker bitmap on first list_lru item apearance

On Thu, May 10, 2018 at 12:53:45PM +0300, Kirill Tkhai wrote: > Introduce set_shrinker_bit() function to set shrinker-related > bit in memcg shrinker bitmap, and set the bit after the first > item is added and in case of reparenting destroyed memcg's items. > > This will allow next patch to make

Re: [PATCH v5 10/13] mm: Set bit in memcg shrinker bitmap on first list_lru item apearance

On Thu, May 10, 2018 at 12:53:45PM +0300, Kirill Tkhai wrote: > Introduce set_shrinker_bit() function to set shrinker-related > bit in memcg shrinker bitmap, and set the bit after the first > item is added and in case of reparenting destroyed memcg's items. > > This will allow next patch to make

Re: [PATCH v5 03/13] mm: Assign memcg-aware shrinkers bitmap to memcg

On Mon, May 14, 2018 at 12:34:45PM +0300, Kirill Tkhai wrote: > >> +static void memcg_free_shrinker_maps(struct mem_cgroup *memcg) > >> +{ > >> + struct mem_cgroup_per_node *pn; > >> + struct memcg_shrinker_map *map; > >> + int nid; > >> + > >> + if (memcg == root_mem_cgroup) > >> +

Re: [PATCH v5 03/13] mm: Assign memcg-aware shrinkers bitmap to memcg

On Mon, May 14, 2018 at 12:34:45PM +0300, Kirill Tkhai wrote: > >> +static void memcg_free_shrinker_maps(struct mem_cgroup *memcg) > >> +{ > >> + struct mem_cgroup_per_node *pn; > >> + struct memcg_shrinker_map *map; > >> + int nid; > >> + > >> + if (memcg == root_mem_cgroup) > >> +

Re: [PATCH v5 01/13] mm: Assign id to every memcg-aware shrinker

On Mon, May 14, 2018 at 12:03:38PM +0300, Kirill Tkhai wrote: > On 13.05.2018 08:15, Vladimir Davydov wrote: > > On Thu, May 10, 2018 at 12:52:18PM +0300, Kirill Tkhai wrote: > >> The patch introduces shrinker::id number, which is used to enumerate > >> memcg-aware shrin

Re: [PATCH v5 01/13] mm: Assign id to every memcg-aware shrinker

On Mon, May 14, 2018 at 12:03:38PM +0300, Kirill Tkhai wrote: > On 13.05.2018 08:15, Vladimir Davydov wrote: > > On Thu, May 10, 2018 at 12:52:18PM +0300, Kirill Tkhai wrote: > >> The patch introduces shrinker::id number, which is used to enumerate > >> memcg-aware shrin

Re: [PATCH v5 06/13] fs: Propagate shrinker::id to list_lru

On Thu, May 10, 2018 at 12:53:06PM +0300, Kirill Tkhai wrote: > The patch adds list_lru::shrinker_id field, and populates > it by registered shrinker id. > > This will be used to set correct bit in memcg shrinkers > map by lru code in next patches, after there appeared > the first related to

Re: [PATCH v5 06/13] fs: Propagate shrinker::id to list_lru

On Thu, May 10, 2018 at 12:53:06PM +0300, Kirill Tkhai wrote: > The patch adds list_lru::shrinker_id field, and populates > it by registered shrinker id. > > This will be used to set correct bit in memcg shrinkers > map by lru code in next patches, after there appeared > the first related to

Re: [PATCH v5 03/13] mm: Assign memcg-aware shrinkers bitmap to memcg

On Thu, May 10, 2018 at 12:52:36PM +0300, Kirill Tkhai wrote: > Imagine a big node with many cpus, memory cgroups and containers. > Let we have 200 containers, every container has 10 mounts, > and 10 cgroups. All container tasks don't touch foreign > containers mounts. If there is intensive pages

Re: [PATCH v5 03/13] mm: Assign memcg-aware shrinkers bitmap to memcg

On Thu, May 10, 2018 at 12:52:36PM +0300, Kirill Tkhai wrote: > Imagine a big node with many cpus, memory cgroups and containers. > Let we have 200 containers, every container has 10 mounts, > and 10 cgroups. All container tasks don't touch foreign > containers mounts. If there is intensive pages

Re: [PATCH v5 01/13] mm: Assign id to every memcg-aware shrinker

On Thu, May 10, 2018 at 12:52:18PM +0300, Kirill Tkhai wrote: > The patch introduces shrinker::id number, which is used to enumerate > memcg-aware shrinkers. The number start from 0, and the code tries > to maintain it as small as possible. > > This will be used as to represent a memcg-aware

Re: [PATCH v5 01/13] mm: Assign id to every memcg-aware shrinker

On Thu, May 10, 2018 at 12:52:18PM +0300, Kirill Tkhai wrote: > The patch introduces shrinker::id number, which is used to enumerate > memcg-aware shrinkers. The number start from 0, and the code tries > to maintain it as small as possible. > > This will be used as to represent a memcg-aware

Re: [PATCH v2 04/12] mm: Assign memcg-aware shrinkers bitmap to memcg

On Tue, Apr 24, 2018 at 03:24:53PM +0300, Kirill Tkhai wrote: > >> +int expand_shrinker_maps(int old_nr, int nr) > >> +{ > >> + int id, size, old_size, node, ret; > >> + struct mem_cgroup *memcg; > >> + > >> + old_size = old_nr / BITS_PER_BYTE; > >> +

Re: [PATCH v2 04/12] mm: Assign memcg-aware shrinkers bitmap to memcg

On Tue, Apr 24, 2018 at 03:24:53PM +0300, Kirill Tkhai wrote: > >> +int expand_shrinker_maps(int old_nr, int nr) > >> +{ > >> + int id, size, old_size, node, ret; > >> + struct mem_cgroup *memcg; > >> + > >> + old_size = old_nr / BITS_PER_BYTE; > >> +

Re: [PATCH v2] mm: introduce memory.min

On Tue, Apr 24, 2018 at 02:54:15PM +0100, Roman Gushchin wrote: > > On Mon, Apr 23, 2018 at 01:36:10PM +0100, Roman Gushchin wrote: > > > + memory.min > > > + A read-write single value file which exists on non-root > > > + cgroups. The default is "0". > > > + > > > + Hard memory protection. If

Re: [PATCH v2] mm: introduce memory.min

On Tue, Apr 24, 2018 at 02:54:15PM +0100, Roman Gushchin wrote: > > On Mon, Apr 23, 2018 at 01:36:10PM +0100, Roman Gushchin wrote: > > > + memory.min > > > + A read-write single value file which exists on non-root > > > + cgroups. The default is "0". > > > + > > > + Hard memory protection. If

Re: [PATCH v2] mm: introduce memory.min

Hi Roman, On Mon, Apr 23, 2018 at 01:36:10PM +0100, Roman Gushchin wrote: > + memory.min > + A read-write single value file which exists on non-root > + cgroups. The default is "0". > + > + Hard memory protection. If the memory usage of a cgroup > + is within its effective min

Re: [PATCH v2] mm: introduce memory.min

Hi Roman, On Mon, Apr 23, 2018 at 01:36:10PM +0100, Roman Gushchin wrote: > + memory.min > + A read-write single value file which exists on non-root > + cgroups. The default is "0". > + > + Hard memory protection. If the memory usage of a cgroup > + is within its effective min

Re: [PATCH v2 04/12] mm: Assign memcg-aware shrinkers bitmap to memcg

On Tue, Apr 24, 2018 at 02:38:51PM +0300, Kirill Tkhai wrote: > On 24.04.2018 14:28, Vladimir Davydov wrote: > > On Mon, Apr 23, 2018 at 01:54:50PM +0300, Kirill Tkhai wrote: > >>>> @@ -1200,6 +1206,8 @@ extern int memcg_nr_cache_ids; > >>>> void

Re: [PATCH v2 04/12] mm: Assign memcg-aware shrinkers bitmap to memcg

On Tue, Apr 24, 2018 at 02:38:51PM +0300, Kirill Tkhai wrote: > On 24.04.2018 14:28, Vladimir Davydov wrote: > > On Mon, Apr 23, 2018 at 01:54:50PM +0300, Kirill Tkhai wrote: > >>>> @@ -1200,6 +1206,8 @@ extern int memcg_nr_cache_ids; > >>>> void

Re: [PATCH v2 04/12] mm: Assign memcg-aware shrinkers bitmap to memcg

On Mon, Apr 23, 2018 at 01:54:50PM +0300, Kirill Tkhai wrote: > >> @@ -1200,6 +1206,8 @@ extern int memcg_nr_cache_ids; > >> void memcg_get_cache_ids(void); > >> void memcg_put_cache_ids(void); > >> > >> +extern int shrinkers_max_nr; > >> + > > > > memcg_shrinker_id_max? > >

Re: [PATCH v2 04/12] mm: Assign memcg-aware shrinkers bitmap to memcg

On Mon, Apr 23, 2018 at 01:54:50PM +0300, Kirill Tkhai wrote: > >> @@ -1200,6 +1206,8 @@ extern int memcg_nr_cache_ids; > >> void memcg_get_cache_ids(void); > >> void memcg_put_cache_ids(void); > >> > >> +extern int shrinkers_max_nr; > >> + > > > > memcg_shrinker_id_max? > >

Re: [PATCH v2 04/12] mm: Assign memcg-aware shrinkers bitmap to memcg

On Mon, Apr 23, 2018 at 02:06:31PM +0300, Kirill Tkhai wrote: > >> diff --git a/mm/vmscan.c b/mm/vmscan.c > >> index 4f02fe83537e..f63eb5596c35 100644 > >> --- a/mm/vmscan.c > >> +++ b/mm/vmscan.c > >> @@ -172,6 +172,22 @@ static DECLARE_RWSEM(shrinker_rwsem); > >> #if defined(CONFIG_MEMCG) &&

Re: [PATCH v2 04/12] mm: Assign memcg-aware shrinkers bitmap to memcg

On Mon, Apr 23, 2018 at 02:06:31PM +0300, Kirill Tkhai wrote: > >> diff --git a/mm/vmscan.c b/mm/vmscan.c > >> index 4f02fe83537e..f63eb5596c35 100644 > >> --- a/mm/vmscan.c > >> +++ b/mm/vmscan.c > >> @@ -172,6 +172,22 @@ static DECLARE_RWSEM(shrinker_rwsem); > >> #if defined(CONFIG_MEMCG) &&

  1   2   3   4   5   6   7   8   9   10   >