Re: [PATCH] mm: slowly shrink slabs with a relatively small number of objects

2018-09-04 Thread Vladimir Davydov
On Tue, Sep 04, 2018 at 10:52:46AM -0700, Roman Gushchin wrote: > Reparenting of all pages is definitely an option to consider, Reparenting pages would be great indeed, but I'm not sure we could do that, because we'd have to walk over page lists of semi-active kmem caches and do it consistently

Re: [PATCH] memcg: Remove memcg_cgroup::id from IDR on mem_cgroup_css_alloc() failure

2018-08-01 Thread Vladimir Davydov
On Wed, Aug 01, 2018 at 11:55:52AM -0400, Johannes Weiner wrote: > On Tue, Jul 31, 2018 at 04:39:08PM -0700, Andrew Morton wrote: > > On Mon, 30 Jul 2018 11:31:13 -0400 Johannes Weiner > > wrote: > > > > > Subject: [PATCH] mm: memcontrol: simplify memcg idr allocation and error > > > unwinding

Re: [PATCH] memcg: Remove memcg_cgroup::id from IDR on mem_cgroup_css_alloc() failure

2018-08-01 Thread Vladimir Davydov
On Mon, Jul 30, 2018 at 11:31:13AM -0400, Johannes Weiner wrote: > On Sun, Jul 29, 2018 at 10:26:21PM +0300, Vladimir Davydov wrote: > > On Fri, Jul 27, 2018 at 03:31:34PM -0400, Johannes Weiner wrote: > > > That said, the lifetime of the root reference on the ID is the online &

Re: [PATCH] memcg: Remove memcg_cgroup::id from IDR on mem_cgroup_css_alloc() failure

2018-07-29 Thread Vladimir Davydov
On Fri, Jul 27, 2018 at 03:31:34PM -0400, Johannes Weiner wrote: > That said, the lifetime of the root reference on the ID is the online > state, we put that in css_offline. Is there a reason we need to have > the ID ready and the memcg in the IDR before onlining it? I fail to see any reason for

Re: [PATCH v8 05/17] mm: Assign memcg-aware shrinkers bitmap to memcg

2018-07-06 Thread Vladimir Davydov
On Thu, Jul 05, 2018 at 03:10:30PM -0700, Andrew Morton wrote: > On Wed, 4 Jul 2018 18:51:12 +0300 Kirill Tkhai wrote: > > > > - why aren't we decreasing shrinker_nr_max in > > > unregister_memcg_shrinker()? That's easy to do, avoids pointless > > > work in shrink_slab_memcg() and avoids

Re: [PATCH v8 05/17] mm: Assign memcg-aware shrinkers bitmap to memcg

2018-07-06 Thread Vladimir Davydov
On Tue, Jul 03, 2018 at 01:50:00PM -0700, Andrew Morton wrote: > On Tue, 03 Jul 2018 18:09:26 +0300 Kirill Tkhai wrote: > > > Imagine a big node with many cpus, memory cgroups and containers. > > Let we have 200 containers, every container has 10 mounts, > > and 10 cgroups. All container tasks

Re: [PATCH v4] mm: fix race between kmem_cache destroy, create and deactivate

2018-06-12 Thread Vladimir Davydov
k the cache as > dying and flush the workqueue used for memcg kmem cache creation and > deactivation. SLUB's memcg kmem cache deactivation also includes RCU > callback and thus make sure all previous registered RCU callbacks > have completed as well. > > Signed-off-by: Shakeel Butt Acked-by: Vladimir Davydov Thanks.

Re: [PATCH v3] mm: fix race between kmem_cache destroy, create and deactivate

2018-06-09 Thread Vladimir Davydov
On Tue, May 29, 2018 at 05:12:04PM -0700, Shakeel Butt wrote: > The memcg kmem cache creation and deactivation (SLUB only) is > asynchronous. If a root kmem cache is destroyed whose memcg cache is in > the process of creation or deactivation, the kernel may crash. > > Example of one such crash: >

Re: [PATCH v2] mm: fix race between kmem_cache destroy, create and deactivate

2018-05-26 Thread Vladimir Davydov
On Tue, May 22, 2018 at 01:13:36PM -0700, Shakeel Butt wrote: > The memcg kmem cache creation and deactivation (SLUB only) is > asynchronous. If a root kmem cache is destroyed whose memcg cache is in > the process of creation or deactivation, the kernel may crash. > > Example of one such crash: >

Re: [PATCH] memcg: force charge kmem counter too

2018-05-26 Thread Vladimir Davydov
On Fri, May 25, 2018 at 11:55:01AM -0700, Shakeel Butt wrote: > Based on several conditions the kernel can decide to force charge an > allocation for a memcg i.e. overcharge memcg->memory and memcg->memsw > counters. Do the same for memcg->kmem counter too. In cgroup-v1, this > bug can cause a

Re: [PATCH v7 00/17] Improve shrink_slab() scalability (old complexity was O(n^2), new is O(n))

2018-05-26 Thread Vladimir Davydov
Hello Kirill, The whole patch set looks good to me now. Acked-by: Vladimir Davydov <vdavydov@gmail.com> Thanks, Vladimir On Tue, May 22, 2018 at 01:07:10PM +0300, Kirill Tkhai wrote: > Hi, > > this patches solves the problem with slow shrink_slab() occuring > on the ma

Re: [PATCH v6 05/17] mm: Assign memcg-aware shrinkers bitmap to memcg

2018-05-21 Thread Vladimir Davydov
On Mon, May 21, 2018 at 01:16:40PM +0300, Kirill Tkhai wrote: > >> +static int memcg_expand_one_shrinker_map(struct mem_cgroup *memcg, > >> + int size, int old_size) > > > > Nit: No point in passing old_size here. You can instead use > > memcg_shrinker_map_size

Re: [PATCH v6 12/17] mm: Set bit in memcg shrinker bitmap on first list_lru item apearance

2018-05-21 Thread Vladimir Davydov
On Mon, May 21, 2018 at 12:31:34PM +0300, Kirill Tkhai wrote: > On 20.05.2018 10:55, Vladimir Davydov wrote: > > On Fri, May 18, 2018 at 11:43:42AM +0300, Kirill Tkhai wrote: > >> Introduce set_shrinker_bit() function to set shrinker-related > >> bit in memcg shrinker bi

Re: [PATCH v6 14/17] mm: Iterate only over charged shrinkers during memcg shrink_slab()

2018-05-21 Thread Vladimir Davydov
On Mon, May 21, 2018 at 12:17:07PM +0300, Kirill Tkhai wrote: > >> +static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid, > >> + struct mem_cgroup *memcg, int priority) > >> +{ > >> + struct memcg_shrinker_map *map; > >> + unsigned long freed = 0; > >> + int ret, i; >

Re: [PATCH v6 15/17] mm: Generalize shrink_slab() calls in shrink_node()

2018-05-20 Thread Vladimir Davydov
On Fri, May 18, 2018 at 11:44:11AM +0300, Kirill Tkhai wrote: > From: Vladimir Davydov <vdavydov@gmail.com> > > The patch makes shrink_slab() be called for root_mem_cgroup > in the same way as it's called for the rest of cgroups. > This simplifies the logic and imp

Re: [PATCH v6 14/17] mm: Iterate only over charged shrinkers during memcg shrink_slab()

2018-05-20 Thread Vladimir Davydov
On Fri, May 18, 2018 at 11:44:01AM +0300, Kirill Tkhai wrote: > Using the preparations made in previous patches, in case of memcg > shrink, we may avoid shrinkers, which are not set in memcg's shrinkers > bitmap. To do that, we separate iterations over memcg-aware and > !memcg-aware shrinkers, and

Re: [PATCH v6 13/17] mm: Export mem_cgroup_is_root()

2018-05-20 Thread Vladimir Davydov
On Fri, May 18, 2018 at 11:43:53AM +0300, Kirill Tkhai wrote: > This will be used in next patch. > > Signed-off-by: Kirill Tkhai > --- > include/linux/memcontrol.h | 10 ++ > mm/memcontrol.c|5 - > 2 files changed, 10 insertions(+), 5

Re: [PATCH v6 12/17] mm: Set bit in memcg shrinker bitmap on first list_lru item apearance

2018-05-20 Thread Vladimir Davydov
On Fri, May 18, 2018 at 11:43:42AM +0300, Kirill Tkhai wrote: > Introduce set_shrinker_bit() function to set shrinker-related > bit in memcg shrinker bitmap, and set the bit after the first > item is added and in case of reparenting destroyed memcg's items. > > This will allow next patch to make

Re: [PATCH v6 05/17] mm: Assign memcg-aware shrinkers bitmap to memcg

2018-05-20 Thread Vladimir Davydov
On Fri, May 18, 2018 at 11:42:37AM +0300, Kirill Tkhai wrote: > Imagine a big node with many cpus, memory cgroups and containers. > Let we have 200 containers, every container has 10 mounts, > and 10 cgroups. All container tasks don't touch foreign > containers mounts. If there is intensive pages

Re: [PATCH v6 03/17] mm: Assign id to every memcg-aware shrinker

2018-05-20 Thread Vladimir Davydov
Hello Kirill, Generally, all patches in the series look OK to me, but I'm going to do some nitpicking before I ack them. See below. On Fri, May 18, 2018 at 11:42:08AM +0300, Kirill Tkhai wrote: > The patch introduces shrinker::id number, which is used to enumerate > memcg-aware shrinkers. The

Re: [PATCH v5 11/13] mm: Iterate only over charged shrinkers during memcg shrink_slab()

2018-05-17 Thread Vladimir Davydov
On Thu, May 17, 2018 at 02:49:26PM +0300, Kirill Tkhai wrote: > On 17.05.2018 07:16, Vladimir Davydov wrote: > > On Tue, May 15, 2018 at 05:49:59PM +0300, Kirill Tkhai wrote: > >>>> @@ -589,13 +647,7 @@ static unsigned long shrink_slab(gfp_t g

Re: [PATCH v5 13/13] mm: Clear shrinker bit if there are no objects related to memcg

2018-05-16 Thread Vladimir Davydov
On Tue, May 15, 2018 at 11:55:04AM +0300, Kirill Tkhai wrote: > >> @@ -586,8 +586,23 @@ static unsigned long shrink_slab_memcg(gfp_t > >> gfp_mask, int nid, > >>continue; > >> > >>ret = do_shrink_slab(, shrinker, priority); > >> - if (ret ==

Re: [PATCH v5 11/13] mm: Iterate only over charged shrinkers during memcg shrink_slab()

2018-05-16 Thread Vladimir Davydov
On Tue, May 15, 2018 at 01:12:20PM +0300, Kirill Tkhai wrote: > >> +#define root_mem_cgroup NULL > > > > Let's instead export mem_cgroup_is_root(). In case if MEMCG is disabled > > it will always return false. > > export == move to header file That and adding a stub function in case !MEMCG. >

Re: [PATCH v5 11/13] mm: Iterate only over charged shrinkers during memcg shrink_slab()

2018-05-16 Thread Vladimir Davydov
On Tue, May 15, 2018 at 05:49:59PM +0300, Kirill Tkhai wrote: > >> @@ -589,13 +647,7 @@ static unsigned long shrink_slab(gfp_t gfp_mask, int > >> nid, > >>.memcg = memcg, > >>}; > >> > >> - /* > >> - * If kernel memory accounting is disabled,

Re: [PATCH v5 13/13] mm: Clear shrinker bit if there are no objects related to memcg

2018-05-14 Thread Vladimir Davydov
On Thu, May 10, 2018 at 12:54:15PM +0300, Kirill Tkhai wrote: > To avoid further unneed calls of do_shrink_slab() > for shrinkers, which already do not have any charged > objects in a memcg, their bits have to be cleared. > > This patch introduces a lockless mechanism to do that > without races

Re: [PATCH v5 11/13] mm: Iterate only over charged shrinkers during memcg shrink_slab()

2018-05-14 Thread Vladimir Davydov
On Thu, May 10, 2018 at 12:53:55PM +0300, Kirill Tkhai wrote: > Using the preparations made in previous patches, in case of memcg > shrink, we may avoid shrinkers, which are not set in memcg's shrinkers > bitmap. To do that, we separate iterations over memcg-aware and > !memcg-aware shrinkers, and

Re: [PATCH v5 10/13] mm: Set bit in memcg shrinker bitmap on first list_lru item apearance

2018-05-14 Thread Vladimir Davydov
On Thu, May 10, 2018 at 12:53:45PM +0300, Kirill Tkhai wrote: > Introduce set_shrinker_bit() function to set shrinker-related > bit in memcg shrinker bitmap, and set the bit after the first > item is added and in case of reparenting destroyed memcg's items. > > This will allow next patch to make

Re: [PATCH v5 03/13] mm: Assign memcg-aware shrinkers bitmap to memcg

2018-05-14 Thread Vladimir Davydov
On Mon, May 14, 2018 at 12:34:45PM +0300, Kirill Tkhai wrote: > >> +static void memcg_free_shrinker_maps(struct mem_cgroup *memcg) > >> +{ > >> + struct mem_cgroup_per_node *pn; > >> + struct memcg_shrinker_map *map; > >> + int nid; > >> + > >> + if (memcg == root_mem_cgroup) > >> +

Re: [PATCH v5 01/13] mm: Assign id to every memcg-aware shrinker

2018-05-14 Thread Vladimir Davydov
On Mon, May 14, 2018 at 12:03:38PM +0300, Kirill Tkhai wrote: > On 13.05.2018 08:15, Vladimir Davydov wrote: > > On Thu, May 10, 2018 at 12:52:18PM +0300, Kirill Tkhai wrote: > >> The patch introduces shrinker::id number, which is used to enumerate > >> memcg-aware shrin

Re: [PATCH v5 06/13] fs: Propagate shrinker::id to list_lru

2018-05-13 Thread Vladimir Davydov
On Thu, May 10, 2018 at 12:53:06PM +0300, Kirill Tkhai wrote: > The patch adds list_lru::shrinker_id field, and populates > it by registered shrinker id. > > This will be used to set correct bit in memcg shrinkers > map by lru code in next patches, after there appeared > the first related to

Re: [PATCH v5 03/13] mm: Assign memcg-aware shrinkers bitmap to memcg

2018-05-13 Thread Vladimir Davydov
On Thu, May 10, 2018 at 12:52:36PM +0300, Kirill Tkhai wrote: > Imagine a big node with many cpus, memory cgroups and containers. > Let we have 200 containers, every container has 10 mounts, > and 10 cgroups. All container tasks don't touch foreign > containers mounts. If there is intensive pages

Re: [PATCH v5 01/13] mm: Assign id to every memcg-aware shrinker

2018-05-12 Thread Vladimir Davydov
On Thu, May 10, 2018 at 12:52:18PM +0300, Kirill Tkhai wrote: > The patch introduces shrinker::id number, which is used to enumerate > memcg-aware shrinkers. The number start from 0, and the code tries > to maintain it as small as possible. > > This will be used as to represent a memcg-aware

Re: [PATCH v2 04/12] mm: Assign memcg-aware shrinkers bitmap to memcg

2018-04-28 Thread Vladimir Davydov
On Tue, Apr 24, 2018 at 03:24:53PM +0300, Kirill Tkhai wrote: > >> +int expand_shrinker_maps(int old_nr, int nr) > >> +{ > >> + int id, size, old_size, node, ret; > >> + struct mem_cgroup *memcg; > >> + > >> + old_size = old_nr / BITS_PER_BYTE; > >> +

Re: [PATCH v2] mm: introduce memory.min

2018-04-25 Thread Vladimir Davydov
On Tue, Apr 24, 2018 at 02:54:15PM +0100, Roman Gushchin wrote: > > On Mon, Apr 23, 2018 at 01:36:10PM +0100, Roman Gushchin wrote: > > > + memory.min > > > + A read-write single value file which exists on non-root > > > + cgroups. The default is "0". > > > + > > > + Hard memory protection. If

Re: [PATCH v2] mm: introduce memory.min

2018-04-24 Thread Vladimir Davydov
Hi Roman, On Mon, Apr 23, 2018 at 01:36:10PM +0100, Roman Gushchin wrote: > + memory.min > + A read-write single value file which exists on non-root > + cgroups. The default is "0". > + > + Hard memory protection. If the memory usage of a cgroup > + is within its effective min

Re: [PATCH v2 04/12] mm: Assign memcg-aware shrinkers bitmap to memcg

2018-04-24 Thread Vladimir Davydov
On Tue, Apr 24, 2018 at 02:38:51PM +0300, Kirill Tkhai wrote: > On 24.04.2018 14:28, Vladimir Davydov wrote: > > On Mon, Apr 23, 2018 at 01:54:50PM +0300, Kirill Tkhai wrote: > >>>> @@ -1200,6 +1206,8 @@ extern int memcg_nr_cache_ids; > >>>> void

Re: [PATCH v2 04/12] mm: Assign memcg-aware shrinkers bitmap to memcg

2018-04-24 Thread Vladimir Davydov
On Mon, Apr 23, 2018 at 01:54:50PM +0300, Kirill Tkhai wrote: > >> @@ -1200,6 +1206,8 @@ extern int memcg_nr_cache_ids; > >> void memcg_get_cache_ids(void); > >> void memcg_put_cache_ids(void); > >> > >> +extern int shrinkers_max_nr; > >> + > > > > memcg_shrinker_id_max? > >

Re: [PATCH v2 04/12] mm: Assign memcg-aware shrinkers bitmap to memcg

2018-04-24 Thread Vladimir Davydov
On Mon, Apr 23, 2018 at 02:06:31PM +0300, Kirill Tkhai wrote: > >> diff --git a/mm/vmscan.c b/mm/vmscan.c > >> index 4f02fe83537e..f63eb5596c35 100644 > >> --- a/mm/vmscan.c > >> +++ b/mm/vmscan.c > >> @@ -172,6 +172,22 @@ static DECLARE_RWSEM(shrinker_rwsem); > >> #if defined(CONFIG_MEMCG) &&

Re: [PATCH v2 12/12] mm: Clear shrinker bit if there are no objects related to memcg

2018-04-24 Thread Vladimir Davydov
On Mon, Apr 23, 2018 at 01:01:08PM +0300, Kirill Tkhai wrote: > On 22.04.2018 21:21, Vladimir Davydov wrote: > > On Tue, Apr 17, 2018 at 09:54:51PM +0300, Kirill Tkhai wrote: > >> To avoid further unneed calls of do_shrink_slab() > >> for shrinkers, which alre

Re: [PATCH v2 12/12] mm: Clear shrinker bit if there are no objects related to memcg

2018-04-22 Thread Vladimir Davydov
On Tue, Apr 17, 2018 at 09:54:51PM +0300, Kirill Tkhai wrote: > To avoid further unneed calls of do_shrink_slab() > for shrinkers, which already do not have any charged > objects in a memcg, their bits have to be cleared. > > This patch introduces a lockless mechanism to do that > without races

Re: [PATCH v2 10/12] mm: Iterate only over charged shrinkers during memcg shrink_slab()

2018-04-22 Thread Vladimir Davydov
On Tue, Apr 17, 2018 at 09:54:34PM +0300, Kirill Tkhai wrote: > Using the preparations made in previous patches, in case of memcg > shrink, we may avoid shrinkers, which are not set in memcg's shrinkers > bitmap. To do that, we separate iterations over memcg-aware and > !memcg-aware shrinkers, and

Re: [PATCH v2 05/12] fs: Propagate shrinker::id to list_lru

2018-04-22 Thread Vladimir Davydov
On Tue, Apr 17, 2018 at 09:53:47PM +0300, Kirill Tkhai wrote: > The patch adds list_lru::shrinker_id field, and populates > it by registered shrinker id. > > This will be used to set correct bit in memcg shrinkers > map by lru code in next patches, after there appeared > the first related to

Re: [PATCH v2 04/12] mm: Assign memcg-aware shrinkers bitmap to memcg

2018-04-22 Thread Vladimir Davydov
On Tue, Apr 17, 2018 at 09:53:31PM +0300, Kirill Tkhai wrote: > Imagine a big node with many cpus, memory cgroups and containers. > Let we have 200 containers, every container has 10 mounts, > and 10 cgroups. All container tasks don't touch foreign > containers mounts. If there is intensive pages

Re: [PATCH v2 01/12] mm: Assign id to every memcg-aware shrinker

2018-04-22 Thread Vladimir Davydov
On Tue, Apr 17, 2018 at 09:53:02PM +0300, Kirill Tkhai wrote: > The patch introduces shrinker::id number, which is used to enumerate > memcg-aware shrinkers. The number start from 0, and the code tries > to maintain it as small as possible. > > This will be used as to represent a memcg-aware

Re: [PATCH 01/10] mm: Assign id to every memcg-aware shrinker

2018-03-28 Thread Vladimir Davydov
On Wed, Mar 28, 2018 at 01:30:20PM +0300, Kirill Tkhai wrote: > On 27.03.2018 18:48, Vladimir Davydov wrote: > > On Tue, Mar 27, 2018 at 06:09:20PM +0300, Kirill Tkhai wrote: > >>>>>> diff --git a/mm/vmscan.c b/mm/vmscan.c > >>>>>> index 8fcd9f8d7

Re: [PATCH 01/10] mm: Assign id to every memcg-aware shrinker

2018-03-27 Thread Vladimir Davydov
On Tue, Mar 27, 2018 at 06:09:20PM +0300, Kirill Tkhai wrote: > diff --git a/mm/vmscan.c b/mm/vmscan.c > index 8fcd9f8d7390..91b5120b924f 100644 > --- a/mm/vmscan.c > +++ b/mm/vmscan.c > @@ -159,6 +159,56 @@ unsigned long vm_total_pages; > static

Re: [PATCH 03/10] mm: Assign memcg-aware shrinkers bitmap to memcg

2018-03-27 Thread Vladimir Davydov
On Mon, Mar 26, 2018 at 06:29:05PM +0300, Kirill Tkhai wrote: > >> @@ -182,6 +187,9 @@ struct mem_cgroup { > >>unsigned long low; > >>unsigned long high; > >> > >> + /* Bitmap of shrinker ids suitable to call for this memcg */ > >> + struct shrinkers_map __rcu *shrinkers_map; > >> + >

Re: [PATCH 02/10] mm: Maintain memcg-aware shrinkers in mcg_shrinkers array

2018-03-27 Thread Vladimir Davydov
On Mon, Mar 26, 2018 at 06:20:55PM +0300, Kirill Tkhai wrote: > On 24.03.2018 21:45, Vladimir Davydov wrote: > > On Wed, Mar 21, 2018 at 04:21:29PM +0300, Kirill Tkhai wrote: > >> The patch introduces mcg_shrinkers array to keep memcg-aware > >> shrinkers in

Re: [PATCH 01/10] mm: Assign id to every memcg-aware shrinker

2018-03-27 Thread Vladimir Davydov
On Mon, Mar 26, 2018 at 06:09:35PM +0300, Kirill Tkhai wrote: > Hi, Vladimir, > > thanks for your review! > > On 24.03.2018 21:40, Vladimir Davydov wrote: > > Hello Kirill, > > > > I don't have any objections to the idea behind this patch set. > > We

Re: [PATCH] mm/list_lru: replace spinlock with RCU in __list_lru_count_one

2018-03-27 Thread Vladimir Davydov
[Cc Kirill] AFAIU this has already been fixed in exactly the same fashion by Kirill (mmotm commit 8e7d1201ec71 "mm: make counting of list_lru_one::nr_items lockless"). Kirill is working on further optimizations right now, see

Re: [PATCH 10/10] mm: Clear shrinker bit if there are no objects related to memcg

2018-03-24 Thread Vladimir Davydov
On Wed, Mar 21, 2018 at 04:23:01PM +0300, Kirill Tkhai wrote: > To avoid further unneed calls of do_shrink_slab() > for shrinkers, which already do not have any charged > objects in a memcg, their bits have to be cleared. > > This patch introduces new return value SHRINK_EMPTY, > which will be

Re: [PATCH 09/10] mm: Iterate only over charged shrinkers during memcg shrink_slab()

2018-03-24 Thread Vladimir Davydov
On Wed, Mar 21, 2018 at 04:22:51PM +0300, Kirill Tkhai wrote: > Using the preparations made in previous patches, in case of memcg > shrink, we may avoid shrinkers, which are not set in memcg's shrinkers > bitmap. To do that, we separate iterations over memcg-aware and > !memcg-aware shrinkers, and

Re: [PATCH 08/10] mm: Set bit in memcg shrinker bitmap on first list_lru item apearance

2018-03-24 Thread Vladimir Davydov
On Wed, Mar 21, 2018 at 04:22:40PM +0300, Kirill Tkhai wrote: > Introduce set_shrinker_bit() function to set shrinker-related > bit in memcg shrinker bitmap, and set the bit after the first > item is added and in case of reparenting destroyed memcg's items. > > This will allow next patch to make

Re: [PATCH 06/10] list_lru: Pass dst_memcg argument to memcg_drain_list_lru_node()

2018-03-24 Thread Vladimir Davydov
On Wed, Mar 21, 2018 at 04:22:10PM +0300, Kirill Tkhai wrote: > This is just refactoring to allow next patches to have > dst_memcg pointer in memcg_drain_list_lru_node(). > > Signed-off-by: Kirill Tkhai > --- > include/linux/list_lru.h |2 +- > mm/list_lru.c

Re: [PATCH 03/10] mm: Assign memcg-aware shrinkers bitmap to memcg

2018-03-24 Thread Vladimir Davydov
On Wed, Mar 21, 2018 at 04:21:40PM +0300, Kirill Tkhai wrote: > Imagine a big node with many cpus, memory cgroups and containers. > Let we have 200 containers, every container has 10 mounts, > and 10 cgroups. All container tasks don't touch foreign > containers mounts. If there is intensive pages

Re: [PATCH 04/10] fs: Propagate shrinker::id to list_lru

2018-03-24 Thread Vladimir Davydov
On Wed, Mar 21, 2018 at 04:21:51PM +0300, Kirill Tkhai wrote: > The patch adds list_lru::shrk_id field, and populates > it by registered shrinker id. > > This will be used to set correct bit in memcg shrinkers > map by lru code in next patches, after there appeared > the first related to memcg

Re: [PATCH 02/10] mm: Maintain memcg-aware shrinkers in mcg_shrinkers array

2018-03-24 Thread Vladimir Davydov
On Wed, Mar 21, 2018 at 04:21:29PM +0300, Kirill Tkhai wrote: > The patch introduces mcg_shrinkers array to keep memcg-aware > shrinkers in order of their shrinker::id. > > This allows to access the shrinkers dirrectly by the id, > without iteration over shrinker_list list. Why don't you simply

Re: [PATCH 01/10] mm: Assign id to every memcg-aware shrinker

2018-03-24 Thread Vladimir Davydov
Hello Kirill, I don't have any objections to the idea behind this patch set. Well, at least I don't know how to better tackle the problem you describe in the cover letter. Please, see below for my comments regarding implementation details. On Wed, Mar 21, 2018 at 04:21:17PM +0300, Kirill Tkhai

Re: [PATCH] mm, slab: eagerly delete inactive offlined SLABs

2018-03-24 Thread Vladimir Davydov
Hello Shakeel, The patch makes sense to me, but I have a concern about synchronization of cache destruction vs concurrent kmem_cache_free. Please, see my comments inline. On Wed, Mar 21, 2018 at 03:43:01PM -0700, Shakeel Butt wrote: > With kmem cgroup support, high memcgs churn can leave behind

Re: [PATCH] slab, slub: remove size disparity on debug kernel

2018-03-14 Thread Vladimir Davydov
On Tue, Mar 13, 2018 at 10:36:52AM -0700, Shakeel Butt wrote: > On Tue, Mar 13, 2018 at 10:19 AM, Christopher Lameter wrote: > > On Tue, 13 Mar 2018, Shakeel Butt wrote: > > > >> However for SLUB in debug kernel, the sizes were same. On further > >> inspection it is found that

Re: [PATCH 3/3] mm: memcontrol: fix excessive complexity in memory.stat reporting

2017-11-07 Thread Vladimir Davydov
lude/linux/memcontrol.h | 96 +++--- > mm/memcontrol.c| 101 > +++++++-- > 2 files changed, 113 insertions(+), 84 deletions(-) Acked-by: Vladimir Davydov <vdavydov@gmail.com>

Re: [PATCH 2/3] mm: memcontrol: implement lruvec stat functions on top of each other

2017-11-07 Thread Vladimir Davydov
gt; --- > include/linux/memcontrol.h | 44 ++-- > 1 file changed, 22 insertions(+), 22 deletions(-) Acked-by: Vladimir Davydov <vdavydov@gmail.com>

Re: [PATCH 1/3] mm: memcontrol: eliminate raw access to stat and event counters

2017-11-07 Thread Vladimir Davydov
gt; 2 files changed, 45 insertions(+), 45 deletions(-) Acked-by: Vladimir Davydov <vdavydov@gmail.com>

Re: [BUG] fs/super: a possible sleep-in-atomic bug in put_super

2017-10-09 Thread Vladimir Davydov
On Sun, Oct 08, 2017 at 10:13:57PM +0100, Al Viro wrote: > On Sun, Oct 08, 2017 at 06:47:46PM +0300, Vladimir Davydov wrote: > > On Sun, Oct 08, 2017 at 03:03:32AM +0100, Al Viro wrote: > > > On Sun, Oct 08, 2017 at 01:56:08AM +0100, Al Viro wrote: > > > > > >

Re: [BUG] fs/super: a possible sleep-in-atomic bug in put_super

2017-10-08 Thread Vladimir Davydov
On Sun, Oct 08, 2017 at 03:03:32AM +0100, Al Viro wrote: > On Sun, Oct 08, 2017 at 01:56:08AM +0100, Al Viro wrote: > > > What's more, we need to be careful about resize vs. drain. Right now it's > > on list_lrus_mutex, but if we drop that around actual resize of an > > individual > > list_lru,

Re: [BUG] fs/super: a possible sleep-in-atomic bug in put_super

2017-10-07 Thread Vladimir Davydov
Hello, On Fri, Oct 06, 2017 at 11:06:04AM +0200, Michal Hocko wrote: > On Fri 06-10-17 16:59:18, Jia-Ju Bai wrote: > > According to fs/super.c, the kernel may sleep under a spinlock. > > The function call path is: > > put_super (acquire the spinlock) > > __put_super > > destroy_super > >

Re: [PATCH] mm: memcontrol: use vmalloc fallback for large kmem memcg arrays

2017-09-19 Thread Vladimir Davydov
ting them physically contiguous. > > Signed-off-by: Johannes Weiner <han...@cmpxchg.org> Thanks to kvmalloc/kvfree helpers this looks really neat. Should've been done like that long ago. Acked-by: Vladimir Davydov <vdavydov@gmail.com> Thanks, Vladimir

Re: [PATCH 3/3] mm: Count list_lru_one::nr_items lockless

2017-08-26 Thread Vladimir Davydov
On Wed, Aug 23, 2017 at 03:26:12PM +0300, Kirill Tkhai wrote: > On 23.08.2017 11:27, Vladimir Davydov wrote: > > On Wed, Aug 23, 2017 at 11:00:56AM +0300, Kirill Tkhai wrote: > >> On 22.08.2017 22:47, Vladimir Davydov wrote: > >>> On Tue, Aug 22, 2017 at 03:29:

Re: [PATCH 3/3] mm: Count list_lru_one::nr_items lockless

2017-08-23 Thread Vladimir Davydov
On Wed, Aug 23, 2017 at 11:00:56AM +0300, Kirill Tkhai wrote: > On 22.08.2017 22:47, Vladimir Davydov wrote: > > On Tue, Aug 22, 2017 at 03:29:35PM +0300, Kirill Tkhai wrote: > >> During the reclaiming slab of a memcg, shrink_slab iterates > >> over all registe

Re: [PATCH 3/3] mm: Count list_lru_one::nr_items lockless

2017-08-22 Thread Vladimir Davydov
On Tue, Aug 22, 2017 at 03:29:35PM +0300, Kirill Tkhai wrote: > During the reclaiming slab of a memcg, shrink_slab iterates > over all registered shrinkers in the system, and tries to count > and consume objects related to the cgroup. In case of memory > pressure, this behaves bad: I observe high

Re: [PATCH 2/3] mm: Make list_lru_node::memcg_lrus RCU protected

2017-08-22 Thread Vladimir Davydov
Hello Kirill, On Tue, Aug 22, 2017 at 03:29:26PM +0300, Kirill Tkhai wrote: > The array list_lru_node::memcg_lrus::list_lru_one[] only grows, > and it never shrinks. The growths happens in memcg_update_list_lru_node(), > and old array's members remain the same after it. > > So, the access to the

[PATCH] slub: fix per memcg cache leak on css offline

2017-08-12 Thread Vladimir Davydov
f Fix the leak by adding the missing call to kobject_put() to sysfs_slab_remove_workfn(). Signed-off-by: Vladimir Davydov <vdavydov@gmail.com> Reported-and-tested-by: Andrei Vagin <ava...@gmail.com> Acked-by: Tejun Heo <t...@kernel.org> Cc: Michal Hocko <

Re: [PATCH] mm, memcg: reset low limit during memcg offlining

2017-07-26 Thread Vladimir Davydov
On Tue, Jul 25, 2017 at 01:31:13PM +0100, Roman Gushchin wrote: > On Tue, Jul 25, 2017 at 03:05:37PM +0300, Vladimir Davydov wrote: > > On Tue, Jul 25, 2017 at 12:40:47PM +0100, Roman Gushchin wrote: > > > A removed memory cgroup with a defined low limit and some belonging > &

Re: [PATCH] mm, memcg: reset low limit during memcg offlining

2017-07-25 Thread Vladimir Davydov
r <han...@cmpxchg.org> > Cc: Michal Hocko <mho...@kernel.org> > Cc: Vladimir Davydov <vdavydov@gmail.com> > Cc: kernel-t...@fb.com > Cc: cgro...@vger.kernel.org > Cc: linux...@kvack.org > Cc: linux-kernel@vger.kernel.org > --- > mm/memcontrol.c | 2 ++

Re: [PATCH] mm/memory-hotplug: Switch locking to a percpu rwsem

2017-07-03 Thread Vladimir Davydov
ification does look compelling. FWIW, Acked-by: Vladimir Davydov <vdavydov@gmail.com> > > Two steps to fix this: > > 1) Convert the memory hotplug locking to a per cpu rwsem so the potential >issues get reported proper by lockdep. > > 2) Lock the online cpus in

Re: [PATCH v4 2/2] fs/dcache.c: fix spin lockup issue on nlru->lock

2017-07-01 Thread Vladimir Davydov
x10 > > Fix this lockup by reducing the number of entries to be shrinked > from the lru list to 1024 at once. Also, add cond_resched() before > processing the lru list again. > > Link: http://marc.info/?t=14972286491=1=2 > Fix-suggested-by: Jan kara <j...@suse.cz>

Re: [PATCH v4 1/2] mm/list_lru.c: fix list_lru_count_node() to be race free

2017-07-01 Thread Vladimir Davydov
s can return incorrect number of > entries from list_lru_count_node(). > > Fix this by keeping track of entries per node and simply return > it in list_lru_count_node(). > > Signed-off-by: Sahitya Tummala <stumm...@codeaurora.org> Acked-by: Vladimir Davydov <vdavydov@gmail.com>

Re: [PATCH v3 1/2] mm/list_lru.c: fix list_lru_count_node() to be race free

2017-06-28 Thread Vladimir Davydov
On Wed, Jun 28, 2017 at 11:37:23AM +0530, Sahitya Tummala wrote: > list_lru_count_node() iterates over all memcgs to get > the total number of entries on the node but it can race with > memcg_drain_all_list_lrus(), which migrates the entries from > a dead cgroup to another. This can return

Re: [PATCH v2] fs/dcache.c: fix spin lockup issue on nlru->lock

2017-06-22 Thread Vladimir Davydov
On Thu, Jun 22, 2017 at 10:01:39PM +0530, Sahitya Tummala wrote: > > > On 6/21/2017 10:01 PM, Vladimir Davydov wrote: > > > >>index cddf397..c8ca150 100644 > >>--- a/fs/dcache.c > >>+++ b/fs/dcache.c > >>@@ -1133,10 +1133

Re: [PATCH v2] fs/dcache.c: fix spin lockup issue on nlru->lock

2017-06-21 Thread Vladimir Davydov
x10 > > Fix this lockup by reducing the number of entries to be shrinked > from the lru list to 1024 at once. Also, add cond_resched() before > processing the lru list again. > > Link: http://marc.info/?t=14972286491=1=2 > Fix-suggested-by: Jan kara <j...@suse.cz&

Re: [PATCH] mm/list_lru.c: use cond_resched_lock() for nlru->lock

2017-06-17 Thread Vladimir Davydov
Hello, On Thu, Jun 15, 2017 at 02:05:23PM -0700, Andrew Morton wrote: > On Mon, 12 Jun 2017 06:17:20 +0530 Sahitya Tummala > wrote: > > > __list_lru_walk_one() can hold the spin lock for longer duration > > if there are more number of entries to be isolated. > > > >

Re: [RFC PATCH v2 6/7] mm, oom: cgroup-aware OOM killer

2017-06-04 Thread Vladimir Davydov
On Thu, Jun 01, 2017 at 07:35:14PM +0100, Roman Gushchin wrote: > Traditionally, the OOM killer is operating on a process level. > Under oom conditions, it finds a process with the highest oom score > and kills it. > > This behavior doesn't suit well the system with many running > containers.

Re: [PATCH v2] memcg: refactor mem_cgroup_resize_limit()

2017-06-04 Thread Vladimir Davydov
On Sun, Jun 04, 2017 at 01:04:37PM -0700, Yu Zhao wrote: > @@ -2498,22 +2449,24 @@ static int mem_cgroup_resize_memsw_limit(struct > mem_cgroup *memcg, > } > > mutex_lock(_limit_mutex); > - if (limit < memcg->memory.limit) { > + inverted =

Re: [RFC PATCH v2 5/7] mm, oom: introduce oom_score_adj for memory cgroups

2017-06-04 Thread Vladimir Davydov
On Thu, Jun 01, 2017 at 07:35:13PM +0100, Roman Gushchin wrote: > Introduce a per-memory-cgroup oom_score_adj setting. > A read-write single value file which exits on non-root > cgroups. The default is "0". > > It will have a similar meaning to a per-process value, > available via

Re: [RFC PATCH v2 4/7] mm, oom: introduce oom_kill_all_tasks option for memory cgroups

2017-06-04 Thread Vladimir Davydov
On Thu, Jun 01, 2017 at 07:35:12PM +0100, Roman Gushchin wrote: > This option defines whether a cgroup should be treated > as a single entity by the OOM killer. > > If set, the OOM killer will compare the whole cgroup with other > memory consumers (other cgroups and tasks in the root cgroup), >

Re: [RFC PATCH v2 1/7] mm, oom: refactor select_bad_process() to take memcg as an argument

2017-06-04 Thread Vladimir Davydov
On Thu, Jun 01, 2017 at 07:35:09PM +0100, Roman Gushchin wrote: > The select_bad_process() function will be used further > to select a process to kill in the victim cgroup. > This cgroup doesn't necessary match oc->memcg, > which is a cgroup, which limits were caused cgroup-wide OOM > (or NULL in

Re: [PATCH 6/6] mm: memcontrol: account slab stats per lruvec

2017-06-03 Thread Vladimir Davydov
| 12 > mm/slab.h | 18 +- > mm/slub.c | 4 ++-- > 3 files changed, 7 insertions(+), 27 deletions(-) Acked-by: Vladimir Davydov <vdavydov@gmail.com>

Re: [PATCH 5/6] mm: memcontrol: per-lruvec stats infrastructure

2017-06-03 Thread Vladimir Davydov
pn->on_tree = false; I don't see the matching free_percpu() anywhere, forget to patch free_mem_cgroup_per_node_info()? Other than that and with the follow-up fix applied, this patch is good IMO. Acked-by: Vladimir Davydov <vdavydov@gmail.com>

Re: [PATCH 4/6] mm: memcontrol: use generic mod_memcg_page_state for kmem pages

2017-06-03 Thread Vladimir Davydov
| 8 > mm/slab.h | 16 > 3 files changed, 12 insertions(+), 29 deletions(-) Acked-by: Vladimir Davydov <vdavydov@gmail.com>

Re: [PATCH 3/6] mm: memcontrol: use the node-native slab memory counters

2017-06-03 Thread Vladimir Davydov
f OOM, especially on 32 bit hosts, but provided the previous patch is accepted, this one looks good to me. Acked-by: Vladimir Davydov <vdavydov@gmail.com>

Re: [PATCH] memcg: refactor mem_cgroup_resize_limit()

2017-06-03 Thread Vladimir Davydov
e.com> > --- > mm/memcontrol.c | 71 > + > 1 file changed, 11 insertions(+), 60 deletions(-) Makes sense to me. Acked-by: Vladimir Davydov <vdavydov@gmail.com>

Re: [PATCH] swap: cond_resched in swap_cgroup_prepare()

2017-06-03 Thread Vladimir Davydov
me for the page allocation. > > Signed-off-by: Yu Zhao <yuz...@google.com> Acked-by: Vladimir Davydov <vdavydov@gmail.com> > --- > mm/swap_cgroup.c | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/mm/swap_cgroup.c b/mm/swap_cgroup.c > index ac6318a

Re: [PATCH] mm: per-cgroup memory reclaim stats

2017-05-20 Thread Vladimir Davydov
annes Weiner <han...@cmpxchg.org> > Cc: Johannes Weiner <han...@cmpxchg.org> > Cc: Tejun Heo <t...@kernel.org> > Cc: Li Zefan <lize...@huawei.com> > Cc: Michal Hocko <mho...@kernel.org> > Cc: Vladimir Davydov <vdavydov@gmail.com> > Cc:

Re: [RFC PATCH] mm, oom: cgroup-aware OOM-killer

2017-05-20 Thread Vladimir Davydov
Hello Roman, On Thu, May 18, 2017 at 05:28:04PM +0100, Roman Gushchin wrote: ... > +5-2-4. Cgroup-aware OOM Killer > + > +Cgroup v2 memory controller implements a cgroup-aware OOM killer. > +It means that it treats memory cgroups as memory consumers > +rather then individual processes. Under the

Re: [PATCH 3/4] mm: memcontrol: re-use node VM page state enum

2017-04-06 Thread Vladimir Davydov
im to track all these stats on a per-cgroup level anyway. > > Signed-off-by: Johannes Weiner <han...@cmpxchg.org> Acked-by: Vladimir Davydov <vdavydov@gmail.com>

Re: [PATCH 4/4] mm: memcontrol: use node page state naming scheme for memcg

2017-04-06 Thread Vladimir Davydov
e_state()] > dec_memcg_page_state() [dec_node_page_state()] > > Signed-off-by: Johannes Weiner <han...@cmpxchg.org> Looks neat. Acked-by: Vladimir Davydov <vdavydov@gmail.com>

Re: [PATCH 2/4] mm: memcontrol: re-use global VM event enum

2017-04-06 Thread Vladimir Davydov
up basis anyway. > > Signed-off-by: Johannes Weiner <han...@cmpxchg.org> Although the increase in the mem_cgroup struct introduced by this patch looks scary, I agree this is a reasonable step toward unification of vmstat, as most vm_even_item entries do make sense to be accounted per cgroup a

Re: [PATCH 1/4] mm: memcontrol: clean up memory.events counting function

2017-04-06 Thread Vladimir Davydov
On Tue, Apr 04, 2017 at 06:01:45PM -0400, Johannes Weiner wrote: > We only ever count single events, drop the @nr parameter. Rename the > function accordingly. Remove low-information kerneldoc. > > Signed-off-by: Johannes Weiner <han...@cmpxchg.org> Acked-by: Vladimir

Re: [PATCH] mm: workingset: fix premature shadow node shrinking with cgroups

2017-03-23 Thread Vladimir Davydov
ttle cache. > > Enable memcg-mode on the shadow node LRUs, such that per-cgroup limits > are applied to per-cgroup lists. > > Fixes: 0a6b76dd23fa ("mm: workingset: make shadow node shrinker memcg aware") > Signed-off-by: Johannes Weiner <han...@cmpxchg.org> >

Re: [PATCH 10/10] slab: use memcg_kmem_cache_wq for slab destruction operations

2017-01-29 Thread Vladimir Davydov
ugh. > > This is suggested by Joonsoo Kim. > > Signed-off-by: Tejun Heo <t...@kernel.org> > Reported-by: Jay Vana <jsv...@fb.com> > Cc: Vladimir Davydov <vdavydov@gmail.com> > Cc: Christoph Lameter <c...@linux.com> > Cc: Pekka Enberg <pe

  1   2   3   4   5   6   7   8   9   10   >