Johannes noticed that reading the memcg kmem_cache pointer in
cache_from_memcg_idx() is performed using READ_ONCE() macro,
which doesn't implement a SMP barrier, which is required
by the logic.
Add a proper smp_rmb() to be paired with smp_wmb() in
memcg_create_kmem_cache().
The same applies to memcg_create_kmem_cache() itself,
which reads the same value without barriers and READ_ONCE().
Suggested-by: Johannes Weiner <han...@cmpxchg.org>
Signed-off-by: Roman Gushchin <g...@fb.com>
---
mm/slab.h | 1 +
mm/slab_common.c | 3 ++-
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/mm/slab.h b/mm/slab.h
index 739099af6cbb..1176b61bb8fc 100644
--- a/mm/slab.h
+++ b/mm/slab.h
@@ -260,6 +260,7 @@ cache_from_memcg_idx(struct kmem_cache *s, int idx)
* memcg_caches issues a write barrier to match this (see
* memcg_create_kmem_cache()).
*/
+ smp_rmb();
cachep = READ_ONCE(arr->entries[idx]);
rcu_read_unlock();
diff --git a/mm/slab_common.c b/mm/slab_common.c
index 58251ba63e4a..8092bdfc05d5 100644
--- a/mm/slab_common.c
+++ b/mm/slab_common.c
@@ -652,7 +652,8 @@ void memcg_create_kmem_cache(struct mem_cgroup *memcg,
* allocation (see memcg_kmem_get_cache()), several threads can try to
* create the same cache, but only one of them may succeed.
*/
- if (arr->entries[idx])
+ smp_rmb();
+ if (READ_ONCE(arr->entries[idx]))
goto out_unlock;
cgroup_name(css->cgroup, memcg_name_buf, sizeof(memcg_name_buf));
--
2.20.1
