Re: Very slow unlockall()

2021-02-10 Thread Michal Hocko
On Wed 10-02-21 17:57:29, Michal Hocko wrote: > On Wed 10-02-21 16:18:50, Vlastimil Babka wrote: [...] > > And the munlock (munlock_vma_pages_range()) is slow, because it uses > > follow_page_mask() in a loop incrementing addresses by PAGE_SIZE, so that's > > always tra

Re: Very slow unlockall()

2021-02-10 Thread Michal Hocko
M, Milan Broz wrote: > >>>>> On 08/01/2021 14:41, Michal Hocko wrote: > >>>>>> On Wed 06-01-21 16:20:15, Milan Broz wrote: > >>>>>>> Hi, > >>>>>>> > >>>>>>> we use

Re: [page-reclaim] Augmented Page Reclaim

2021-02-10 Thread Michal Hocko
ecially when you do not plan to build on an existing code and rather plan to do things considerably differently. I confess I haven't checked your repository but it would have been much better to post a patch series -- Michal Hocko SUSE Labs

Re: [PATCH] kswapd: no need reclaim cma pages triggered by unmovable allocation

2021-02-10 Thread Michal Hocko
On Wed 10-02-21 12:07:57, zhou xianrong wrote: > > On 2021/2/9 下午5:23, Michal Hocko wrote: > > On Tue 09-02-21 16:23:13, zhou wrote: > > > From: zhou xianrong > > > > > > For purpose of better migration cma pages are allocated after > > > failu

Re: [PATCH] mm: remove lru_add_drain_all in alloc_contig_range

2021-02-10 Thread Michal Hocko
e page instead of putback_lru_page()") This would be a great addition to the changelog. Thanks a lot Vlastimil, you saved me from some archeology. With that mentioned feel free to add Acked-by: Michal Hocko > > > [1] > > https://lore.kernel.org/linux-mm/20210208103935.GA32103@linux/T/#md651fc6e73c656105179382f92f8b2d6073051d1 > > > > > -- Michal Hocko SUSE Labs

Re: [PATCH 3/3] mm: Fix missing mem cgroup soft limit tree updates

2021-02-10 Thread Michal Hocko
tatic void uncharge_page(struct page *page, struct > uncharge_gather *ug) > ug->pgpgout++; > > ug->dummy_page = page; > + ug->nid = page_to_nid(page); > page->memcg_data = 0; > css_put(&ug->memcg->css); > } > -- > 2.20.1 -- Michal Hocko SUSE Labs

Re: [PATCH 2/3] mm: Force update of mem cgroup soft limit tree on usage excess

2021-02-10 Thread Michal Hocko
limit; > > do_softlimit = mem_cgroup_event_ratelimit(memcg, > MEM_CGROUP_TARGET_SOFTLIMIT); > mem_cgroup_threshold(memcg); > - if (unlikely(do_softlimit)) > + if (unlikely((force_update) || do_softlimit)) > mem_cgroup_update_tree(memcg, page); > } > } > -- > 2.20.1 -- Michal Hocko SUSE Labs

Re: [PATCH 1/3] mm: Fix dropped memcg from mem cgroup soft limit tree

2021-02-10 Thread Michal Hocko
It seems this goes all the way to when it has been introduced by 4e41695356fb ("memory controller: soft limit reclaim on contention"). Please add a Fixes tag pointing to the above one. While this looks like a rare event to happen because there should be some reclaimable memory

Re: [PATCH] fs: buffer: use raw page_memcg() on locked page

2021-02-10 Thread Michal Hocko
n Song > Signed-off-by: Johannes Weiner Acked-by: Michal Hocko Thanks! > --- > fs/buffer.c| 4 ++-- > include/linux/memcontrol.h | 7 --- > mm/memcontrol.c| 23 --- > 3 files changed, 2 insertions(+), 32 deletions(-)

Re: [PATCH v17 00/10] mm: introduce memfd_secret system call to create "secret" memory areas

2021-02-09 Thread Michal Hocko
On Tue 09-02-21 17:17:22, David Hildenbrand wrote: > On 09.02.21 14:25, Michal Hocko wrote: > > On Tue 09-02-21 11:23:35, David Hildenbrand wrote: > > [...] > > > I am constantly trying to fight for making more stuff MOVABLE instead of > > > going into the ot

Re: [PATCH v17 00/10] mm: introduce memfd_secret system call to create "secret" memory areas

2021-02-09 Thread Michal Hocko
e movable zone is documented with all the potential problems we have encountered over time and explicitly state which features are fully/partially incompatible. -- Michal Hocko SUSE Labs

Re: [PATCH v17 07/10] mm: introduce memfd_secret system call to create "secret" memory areas

2021-02-09 Thread Michal Hocko
On Tue 09-02-21 11:09:38, Mike Rapoport wrote: > On Tue, Feb 09, 2021 at 09:47:08AM +0100, Michal Hocko wrote: > > On Mon 08-02-21 23:26:05, Mike Rapoport wrote: > > > On Mon, Feb 08, 2021 at 11:49:22AM +0100, Michal Hocko wrote: > > > > On Mon 08-02-21

Re: [PATCH v17 00/10] mm: introduce memfd_secret system call to create "secret" memory areas

2021-02-09 Thread Michal Hocko
On Tue 09-02-21 10:15:17, David Hildenbrand wrote: > On 09.02.21 09:59, Michal Hocko wrote: > > On Mon 08-02-21 22:38:03, David Hildenbrand wrote: > > > > > > > Am 08.02.2021 um 22:13 schrieb Mike Rapoport : > > > > > > > > On Mon, Feb

Re: [PATCH] kswapd: no need reclaim cma pages triggered by unmovable allocation

2021-02-09 Thread Michal Hocko
highest_zoneidx); > @@ -3974,6 +4012,9 @@ void wakeup_kswapd(struct zone *zone, gfp_t gfp_flags, > int order, > if (READ_ONCE(pgdat->kswapd_order) < order) > WRITE_ONCE(pgdat->kswapd_order, order); > > + if (curr_migratetype == MIGRATE_TYPES || > is_migrate_movable(migratetype)) > + WRITE_ONCE(pgdat->kswapd_migratetype, migratetype); > + > if (!waitqueue_active(&pgdat->kswapd_wait)) > return; > > @@ -3994,7 +4035,7 @@ void wakeup_kswapd(struct zone *zone, gfp_t gfp_flags, > int order, > } > > trace_mm_vmscan_wakeup_kswapd(pgdat->node_id, highest_zoneidx, order, > - gfp_flags); > + migratetype, gfp_flags); > wake_up_interruptible(&pgdat->kswapd_wait); > } > > @@ -4017,6 +4058,7 @@ unsigned long shrink_all_memory(unsigned long > nr_to_reclaim) > .may_writepage = 1, > .may_unmap = 1, > .may_swap = 1, > + .may_cma = 1, > .hibernation_mode = 1, > }; > struct zonelist *zonelist = node_zonelist(numa_node_id(), sc.gfp_mask); > @@ -4176,6 +4218,7 @@ static int __node_reclaim(struct pglist_data *pgdat, > gfp_t gfp_mask, unsigned in > .may_writepage = !!(node_reclaim_mode & RECLAIM_WRITE), > .may_unmap = !!(node_reclaim_mode & RECLAIM_UNMAP), > .may_swap = 1, > + .may_cma = movable_reclaim(gfp_mask), > .reclaim_idx = gfp_zone(gfp_mask), > }; > > -- > 2.25.1 -- Michal Hocko SUSE Labs

Re: [PATCH v17 00/10] mm: introduce memfd_secret system call to create "secret" memory areas

2021-02-09 Thread Michal Hocko
has to be very careful when relying on CMA or movable zones. This is definitely worth a comment in the kernel command line parameter documentation. But this is not a new problem. -- Michal Hocko SUSE Labs

Re: [PATCH v17 07/10] mm: introduce memfd_secret system call to create "secret" memory areas

2021-02-09 Thread Michal Hocko
On Mon 08-02-21 23:26:05, Mike Rapoport wrote: > On Mon, Feb 08, 2021 at 11:49:22AM +0100, Michal Hocko wrote: > > On Mon 08-02-21 10:49:17, Mike Rapoport wrote: [...] > > > The file descriptor based memory has several advantages over the > > > "tradition

Re: [PATCH 6/7] mm: memcontrol: switch to rstat

2021-02-08 Thread Michal Hocko
On Fri 05-02-21 11:34:19, Johannes Weiner wrote: > On Fri, Feb 05, 2021 at 04:05:20PM +0100, Michal Hocko wrote: > > On Tue 02-02-21 13:47:45, Johannes Weiner wrote: > > > Replace the memory controller's custom hierarchical stats code with > > > the generic rsta

Re: [PATCH 7/8] mm: memcontrol: consolidate lruvec stat flushing

2021-02-08 Thread Michal Hocko
llected, but the rest of the flushing code is the > same. Merge them into one function and share the common code. > > Signed-off-by: Johannes Weiner Yes, this looks much better/cleaner. Acked-by: Michal Hocko Thanks! > --- > mm/memcontrol.c | 74 +++

Re: [PATCH v17 08/10] PM: hibernate: disable when there are active secretmem users

2021-02-08 Thread Michal Hocko
, Michal Hocko wrote: > On Mon 08-02-21 12:26:31, David Hildenbrand wrote: > [...] > > My F33 system happily hibernates to disk, even with an application that > > succeeded in din doing an mlockall(). > > > > And it somewhat makes sense. Even my freshly-booted, idle F33 ha

Re: [PATCH v17 08/10] PM: hibernate: disable when there are active secretmem users

2021-02-08 Thread Michal Hocko
k whether the expectated mlock semantic really works for those. This should be documented at least. -- Michal Hocko SUSE Labs

[tip: sched/core] preempt: Introduce CONFIG_PREEMPT_DYNAMIC

2021-02-08 Thread tip-bot2 for Michal Hocko
The following commit has been merged into the sched/core branch of tip: Commit-ID: 5759bcdb871f7f73b033643cd27d6cec33280540 Gitweb: https://git.kernel.org/tip/5759bcdb871f7f73b033643cd27d6cec33280540 Author:Michal Hocko AuthorDate:Mon, 18 Jan 2021 15:12:19 +01:00

Re: [PATCH v17 08/10] PM: hibernate: disable when there are active secretmem users

2021-02-08 Thread Michal Hocko
On Mon 08-02-21 11:53:58, David Hildenbrand wrote: > On 08.02.21 11:51, Michal Hocko wrote: > > On Mon 08-02-21 11:32:11, David Hildenbrand wrote: > > > On 08.02.21 11:18, Michal Hocko wrote: > > > > On Mon 08-02-21 10:49:18, Mike Rapoport wrote:

Re: [PATCH v17 08/10] PM: hibernate: disable when there are active secretmem users

2021-02-08 Thread Michal Hocko
On Mon 08-02-21 11:32:11, David Hildenbrand wrote: > On 08.02.21 11:18, Michal Hocko wrote: > > On Mon 08-02-21 10:49:18, Mike Rapoport wrote: > > > From: Mike Rapoport > > > > > > It is unsafe to allow saving of secretmem areas to the hibernation > >

Re: [PATCH v17 07/10] mm: introduce memfd_secret system call to create "secret" memory areas

2021-02-08 Thread Michal Hocko
> https://lore.kernel.org/linux-mm/213b4567-46ce-f116-9cdf-bbd0c884e...@linux.intel.com/ I have only glanced through the implementation and it looks sane. I will have a closer look later but this should be pretty simple with the proposed semantic. -- Michal Hocko SUSE Labs

Re: [PATCH v17 08/10] PM: hibernate: disable when there are active secretmem users

2021-02-08 Thread Michal Hocko
; Prevent hibernation whenever there are active secret memory users. Does this feature need any special handling? As it is effectivelly unevictable memory then it should behave the same as other mlock, ramfs which should already disable hibernation as those cannot be swapped out, no? -- Michal Hocko SUSE Labs

Re: [External] Re: [PATCH] mm: memcontrol: remove rcu_read_lock from get_mem_cgroup_from_page

2021-02-08 Thread Michal Hocko
On Fri 05-02-21 13:15:40, Johannes Weiner wrote: > On Fri, Feb 05, 2021 at 11:32:24AM +0100, Michal Hocko wrote: > > On Fri 05-02-21 17:14:30, Muchun Song wrote: > > > On Fri, Feb 5, 2021 at 4:36 PM Michal Hocko wrote: > > > > > > > > On Fri 05-02-21

Re: [External] Re: [PATCH] mm: memcontrol: fix missing wakeup oom task

2021-02-05 Thread Michal Hocko
On Fri 05-02-21 19:04:19, Muchun Song wrote: > On Fri, Feb 5, 2021 at 6:21 PM Michal Hocko wrote: > > > > On Fri 05-02-21 17:55:10, Muchun Song wrote: > > > On Fri, Feb 5, 2021 at 4:24 PM Michal Hocko wrote: > > > > > > > > On Fri 05-02-2

Re: [PATCH 7/7] mm: memcontrol: consolidate lruvec stat flushing

2021-02-05 Thread Michal Hocko
. > Signed-off-by: Johannes Weiner Anyway the above is not really a fundamental objection. It is more important to unify the flushing. Acked-by: Michal Hocko > --- > mm/memcontrol.c | 88 +++-- > 1 file changed, 42 insertions(+), 46 del

Re: [PATCH v2] mm: memcontrol: replace the loop with a list_for_each_entry()

2021-02-05 Thread Michal Hocko
ch_entry(). > > There is only one caller of the uncharge_list(). So just fold it into > mem_cgroup_uncharge_list() and remove it. > > Signed-off-by: Muchun Song Acked-by: Michal Hocko Thanks! > --- > v2: > - Fold uncharge_list() to mem_cgroup_uncharge_list(). > >

Re: [External] Re: [PATCH] mm: memcontrol: fix missing wakeup oom task

2021-02-05 Thread Michal Hocko
On Fri 05-02-21 23:30:36, Muchun Song wrote: > On Fri, Feb 5, 2021 at 8:20 PM Michal Hocko wrote: > > > > On Fri 05-02-21 19:04:19, Muchun Song wrote: > > > On Fri, Feb 5, 2021 at 6:21 PM Michal Hocko wrote: > > > > > > > > On Fri 05-02-21 17:5

Re: [External] Re: [PATCH] mm: memcontrol: remove rcu_read_lock from get_mem_cgroup_from_page

2021-02-05 Thread Michal Hocko
On Fri 05-02-21 07:59:06, Shakeel Butt wrote: > +Cc Roman > > On Fri, Feb 5, 2021 at 2:49 AM Michal Hocko wrote: > > > [snip] > > > > > Also, css_get is enough because page > > > > > has a reference to the memcg. > > > > > > >

Re: [PATCH 6/7] mm: memcontrol: switch to rstat

2021-02-05 Thread Michal Hocko
ugh it would be really nice if this could be optimized in the future. All that being said, the code looks more manageable now. Acked-by: Michal Hocko > --- > include/linux/memcontrol.h | 67 ++- > mm/memcontrol.c| 224 +++-- > 2

Re: Possible deny of service with memfd_create()

2021-02-05 Thread Michal Hocko
On Fri 05-02-21 11:57:09, Christian König wrote: > Am 05.02.21 um 11:50 schrieb Michal Hocko: > > On Fri 05-02-21 08:54:31, Christian König wrote: > > > Am 05.02.21 um 01:32 schrieb Hugh Dickins: > > > > On Thu, 4 Feb 2021, Michal Hocko wrote: [...] > > > >

Re: Possible deny of service with memfd_create()

2021-02-05 Thread Michal Hocko
On Fri 05-02-21 08:54:31, Christian König wrote: > Am 05.02.21 um 01:32 schrieb Hugh Dickins: > > On Thu, 4 Feb 2021, Michal Hocko wrote: > > > On Thu 04-02-21 17:32:20, Christian Koenig wrote: > > > > Hi Michal, > > > > > > > > as requested i

Re: [External] Re: [PATCH] mm: memcontrol: remove rcu_read_lock from get_mem_cgroup_from_page

2021-02-05 Thread Michal Hocko
On Fri 05-02-21 17:14:30, Muchun Song wrote: > On Fri, Feb 5, 2021 at 4:36 PM Michal Hocko wrote: > > > > On Fri 05-02-21 14:27:19, Muchun Song wrote: > > > The get_mem_cgroup_from_page() is called under page lock, so the page > > > memcg cannot be changed under u

Re: [External] Re: [PATCH] mm: memcontrol: fix missing wakeup oom task

2021-02-05 Thread Michal Hocko
On Fri 05-02-21 17:55:10, Muchun Song wrote: > On Fri, Feb 5, 2021 at 4:24 PM Michal Hocko wrote: > > > > On Fri 05-02-21 14:23:10, Muchun Song wrote: > > > We call memcg_oom_recover() in the uncharge_batch() to wakeup OOM task > > > when page uncharged, but for

Re: [PATCH] mm: memcontrol: remove rcu_read_lock from get_mem_cgroup_from_page

2021-02-05 Thread Michal Hocko
emcg || WARN_ON_ONCE(!css_tryget(&memcg->css))) > - memcg = root_mem_cgroup; > - rcu_read_unlock(); > + memcg = page_memcg(page) ? : root_mem_cgroup; > + css_get(&memcg->css); > + > return memcg; > } > EXPORT_SYMBOL(get_mem_cgroup_from_page); > -- > 2.11.0 -- Michal Hocko SUSE Labs

Re: [PATCH] mm: memcontrol: fix missing wakeup oom task

2021-02-05 Thread Michal Hocko
3,7 @@ static void drain_stock(struct memcg_stock_pcp *stock) > page_counter_uncharge(&old->memory, stock->nr_pages); > if (do_memsw_account()) > page_counter_uncharge(&old->memsw, stock->nr_pages); > + memcg_oom_recover(old); > stock->nr_pages = 0; > } > > -- > 2.11.0 -- Michal Hocko SUSE Labs

Re: Possible deny of service with memfd_create()

2021-02-04 Thread Michal Hocko
who can write to it which is not the case for memfd but then we hit the backward compatibility wall. -- Michal Hocko SUSE Labs

Re: [PATCH 6/7] mm: memcontrol: switch to rstat

2021-02-04 Thread Michal Hocko
On Thu 04-02-21 11:15:06, Johannes Weiner wrote: > Hello Michal, > > On Thu, Feb 04, 2021 at 03:19:17PM +0100, Michal Hocko wrote: > > On Tue 02-02-21 13:47:45, Johannes Weiner wrote: > > > Replace the memory controller's custom hierarchical stats code with > >

Re: [PATCH 4/7] cgroup: rstat: support cgroup1

2021-02-04 Thread Michal Hocko
On Thu 04-02-21 11:01:30, Johannes Weiner wrote: > On Thu, Feb 04, 2021 at 02:39:25PM +0100, Michal Hocko wrote: > > On Tue 02-02-21 13:47:43, Johannes Weiner wrote: > > > Rstat currently only supports the default hierarchy in cgroup2. In > > > order to replace memcg

Re: [PATCH 6/7] mm: memcontrol: switch to rstat

2021-02-04 Thread Michal Hocko
flushes are needed when you want an exact value (e.g. when values are presented to the userspace). I do not see any flushes to be done by the core pro-actively except for clean up on a release. Is the above correct understanding? -- Michal Hocko SUSE Labs

Re: [PATCH 4/7] cgroup: rstat: support cgroup1

2021-02-04 Thread Michal Hocko
moves back to the default hierarchy. Annotate > rebind_subsystems() to move the root css linkage along between roots. I am not familiar with rstat API and from this patch it is not really clear to me how does it deal with memcg v1 use_hierarchy oddness. -- Michal Hocko SUSE Labs

Re: [PATCH 3/7] mm: memcontrol: privatize memcg_page_state query functions

2021-02-04 Thread Michal Hocko
On Tue 02-02-21 13:47:42, Johannes Weiner wrote: > There are no users outside of the memory controller itself. The rest > of the kernel cares either about node or lruvec stats. > > Signed-off-by: Johannes Weiner Acked-by: Michal Hocko > --- > include/linux

Re: [PATCH 2/7] mm: memcontrol: kill mem_cgroup_nodeinfo()

2021-02-04 Thread Michal Hocko
On Tue 02-02-21 13:47:41, Johannes Weiner wrote: > No need to encapsulate a simple struct member access. > > Signed-off-by: Johannes Weiner Acked-by: Michal Hocko > --- > include/linux/memcontrol.h | 8 +--- > mm/memcontrol.c| 21 +++-- >

Re: [PATCH 1/7] mm: memcontrol: fix cpuhotplug statistics flushing

2021-02-04 Thread Michal Hocko
ng mod_memcg_state() and > count_memcg_events() instead of open-coding the upward flush - this is > comparable to how vmstat.c handles hotunplug flushing. > > Signed-off-by: Johannes Weiner Acked-by: Michal Hocko Shakeel has already pointed out Fixes.

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-02-04 Thread Michal Hocko
On Thu 04-02-21 11:58:55, Mike Rapoport wrote: > On Wed, Feb 03, 2021 at 10:12:22AM +0100, Michal Hocko wrote: [...] > > Wrt to the specific syscall, please document why existing interfaces are > > not a good fit as well. It would be also great to describe interaction > >

Re: [PATCH v16 06/11] mm: introduce memfd_secret system call to create "secret" memory areas

2021-02-03 Thread Michal Hocko
OVABLE and you do not support migration so no pages from movable zones should be allowed. -- Michal Hocko SUSE Labs

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-02-03 Thread Michal Hocko
On Tue 02-02-21 10:55:40, James Bottomley wrote: > On Tue, 2021-02-02 at 20:15 +0200, Mike Rapoport wrote: > > On Tue, Feb 02, 2021 at 03:34:29PM +0100, David Hildenbrand wrote: > > > On 02.02.21 15:32, Michal Hocko wrote: > > > > On Tue 02-02-21 15:26:20, David H

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-02-03 Thread Michal Hocko
On Tue 02-02-21 21:10:40, Mike Rapoport wrote: > On Tue, Feb 02, 2021 at 02:27:14PM +0100, Michal Hocko wrote: > > On Tue 02-02-21 14:48:57, Mike Rapoport wrote: > > > On Tue, Feb 02, 2021 at 10:35:05AM +0100, Michal Hocko wrote: > > > > On Mon 01-02-21 0

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-02-02 Thread Michal Hocko
On Tue 02-02-21 15:26:20, David Hildenbrand wrote: > On 02.02.21 15:22, Michal Hocko wrote: > > On Tue 02-02-21 15:12:21, David Hildenbrand wrote: > > [...] > > > I think secretmem behaves much more like longterm GUP right now > > > ("unmigratable", &quo

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-02-02 Thread Michal Hocko
ntrol/limit it or > make it behave more like mlocked pages. I thought I have already asked but I must have forgotten. Is there any actual reason why the memory is not movable? Timing attacks? -- Michal Hocko SUSE Labs

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-02-02 Thread Michal Hocko
t (e.g. RT). Another example is ramdisk or even tmpfs (with swap storage depleted or not configured). Both are PITA from the OOM POV but they are manageable if people are careful. If secretmem behaves along those existing models then we know what to expect at least. -- Michal Hocko SUSE Labs

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-02-02 Thread Michal Hocko
On Tue 02-02-21 14:48:57, Mike Rapoport wrote: > On Tue, Feb 02, 2021 at 10:35:05AM +0100, Michal Hocko wrote: > > On Mon 01-02-21 08:56:19, James Bottomley wrote: > > > > I have also proposed potential ways out of this. Either the pool is not > > fixed sized and you ma

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-02-02 Thread Michal Hocko
On Mon 01-02-21 08:56:19, James Bottomley wrote: > On Fri, 2021-01-29 at 09:23 +0100, Michal Hocko wrote: > > On Thu 28-01-21 13:05:02, James Bottomley wrote: > > > Obviously the API choice could be revisited > > > but do you have anything to add over the previous discus

Re: [PATCH v1 2/2] Documentation: sysfs/memory: clarify some memory block device properties

2021-02-01 Thread Michal Hocko
l supports memory offlining. > > Let's update documentation, stating that the interface is legacy. Also > update documentation of the "state" property and "valid_zones" > properties. > > Cc: Andrew Morton > Cc: Dave Hansen > Cc: Michal Hocko > C

Re: [PATCH v1 1/2] drivers/base/memory: don't store phys_device in memory blocks

2021-02-01 Thread Michal Hocko
different > interfaces [1]. > > [1] https://patchwork.kernel.org/patch/2163871/ > [2] https://github.com/ibm-s390-tools/s390-tools/blob/v2.1.0/zconf/lsmem > [3] https://github.com/ibm-s390-tools/s390-tools/blob/v2.1.0/zconf/chmem > [4] https://bugzilla.redhat.com/show_bug.cg

Re: [PATCH v3 3/5] hugetlb: only set HPageMigratable for migratable hstates

2021-02-01 Thread Michal Hocko
; From: Mike Kravetz > Date: Fri, 29 Jan 2021 10:36:12 -0800 > Subject: [PATCH] huegtlb: add synchronization information for new hugetlb > specific flags > > Adding comments, no functional change. > > Signed-off-by: Mike Kravetz Acked-by: Michal Hocko Thanks Mike! >

Re: [PATCH 1/3] kvfree_rcu: Allocate a page for a single argument

2021-02-01 Thread Michal Hocko
On Fri 29-01-21 17:35:31, Uladzislau Rezki wrote: > On Fri, Jan 29, 2021 at 09:56:29AM +0100, Michal Hocko wrote: > > On Thu 28-01-21 19:02:37, Uladzislau Rezki wrote: > > [...] > > > >From 0bdb8ca1ae62088790e0a452c4acec3821e06989 Mon Sep 17 00:00:00 2001 > >

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-01-29 Thread Michal Hocko
ons would be that direct map would be handled on instantiation/tear down paths, migration would deal with the same (if possible). Other than that it would be mlock like. -- Michal Hocko SUSE Labs

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-01-28 Thread Michal Hocko
On Thu 28-01-21 15:56:36, Cristopher Lameter wrote: > On Thu, 28 Jan 2021, Michal Hocko wrote: > > > > > If you kill the allocating process then yes, it would work, but your > > > > process might be the very last to be selected. > > > > > > OOMs are

Re: [PATCH 1/3] kvfree_rcu: Allocate a page for a single argument

2021-01-28 Thread Michal Hocko
On Thu 28-01-21 16:11:52, Uladzislau Rezki wrote: > On Mon, Jan 25, 2021 at 05:25:59PM +0100, Uladzislau Rezki wrote: > > On Mon, Jan 25, 2021 at 04:39:43PM +0100, Michal Hocko wrote: > > > On Mon 25-01-21 15:31:50, Uladzislau Rezki wrote: > > > > > On Wed 20-01-21

Re: [PATCH v16 08/11] secretmem: add memcg accounting

2021-01-28 Thread Michal Hocko
On Thu 28-01-21 06:05:11, Shakeel Butt wrote: > On Wed, Jan 27, 2021 at 11:59 PM Michal Hocko wrote: > > > > On Wed 27-01-21 10:42:13, Roman Gushchin wrote: > > > On Tue, Jan 26, 2021 at 04:05:55PM +0100, Michal Hocko wrote: > > > > On Tue 26-01-21 14:48:38, Ma

Re: [RFC PATCH v0] mm/slub: Let number of online CPUs determine the slub page order

2021-01-28 Thread Michal Hocko
r pages to keep pace with allocation and reduce the churn or deploy some other techniques to reduce the direct page allocator involvement. -- Michal Hocko SUSE Labs

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-01-28 Thread Michal Hocko
On Thu 28-01-21 13:28:10, Cristopher Lameter wrote: > On Thu, 28 Jan 2021, Michal Hocko wrote: > > > > So, if I understand your concerns correct this implementation has two > > > issues: > > > 1) allocation failure at page fault that causes unrecoverable OOM

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-01-28 Thread Michal Hocko
On Thu 28-01-21 11:22:59, Mike Rapoport wrote: > On Tue, Jan 26, 2021 at 01:08:23PM +0100, Michal Hocko wrote: > > On Tue 26-01-21 12:56:48, David Hildenbrand wrote: > > > On 26.01.21 12:46, Michal Hocko wrote: > > > > On Thu 21-01-21 14:27:19, Mike Rapoport wrote:

Re: [PATCH v16 08/11] secretmem: add memcg accounting

2021-01-28 Thread Michal Hocko
On Wed 27-01-21 10:42:13, Roman Gushchin wrote: > On Tue, Jan 26, 2021 at 04:05:55PM +0100, Michal Hocko wrote: > > On Tue 26-01-21 14:48:38, Matthew Wilcox wrote: > > > On Mon, Jan 25, 2021 at 11:38:17PM +0200, Mike Rapoport wrote: > > > > I cannot use __GFP_ACCOUN

Re: [PATCH v4 2/4] mm: failfast mode with __GFP_NORETRY in alloc_contig_range

2021-01-27 Thread Michal Hocko
On Wed 27-01-21 12:42:45, Minchan Kim wrote: > On Tue, Jan 26, 2021 at 08:44:49AM +0100, Michal Hocko wrote: > > On Mon 25-01-21 11:33:36, Minchan Kim wrote: > > > On Mon, Jan 25, 2021 at 02:12:00PM +0100, Michal Hocko wrote: > > > > On Thu 21-01-

Re: [PATCH] procfs/dmabuf: Add /proc//task//dmabuf_fds

2021-01-27 Thread Michal Hocko
r overhead as being the main concern as oom is a real cold path. Anyway, if you want to revamp those patches then feel free to CC me and we can discuss further. I do not want to hijack this thread by an unrelated topic. -- Michal Hocko SUSE Labs

Re: [PATCH] procfs/dmabuf: Add /proc//task//dmabuf_fds

2021-01-27 Thread Michal Hocko
On Wed 27-01-21 12:08:50, Christian König wrote: > Am 27.01.21 um 12:02 schrieb Michal Hocko: > > On Wed 27-01-21 11:53:55, Christian König wrote: > > [...] > > > In general processes are currently not held accountable for memory they > > > reference through their

Re: [PATCH v3 5/5] hugetlb: convert PageHugeFreed to HPageFreed flag

2021-01-27 Thread Michal Hocko
On Fri 22-01-21 11:52:31, Mike Kravetz wrote: > Use new hugetlb specific HPageFreed flag to replace the > PageHugeFreed interfaces. > > Signed-off-by: Mike Kravetz > Reviewed-by: Oscar Salvador > Reviewed-by: Muchun Song Acked-by: Michal Hocko Again I would love to s

Re: [PATCH] procfs/dmabuf: Add /proc//task//dmabuf_fds

2021-01-27 Thread Michal Hocko
ss level and/or oom_badness doesn't take them into consideration. -- Michal Hocko SUSE Labs

Re: [PATCH] procfs/dmabuf: Add /proc//task//dmabuf_fds

2021-01-27 Thread Michal Hocko
y consider shmem in badness so this wouldn't go out of line. Kernel oom killer could be more clever with these special fds though and query for buffer size directly. -- Michal Hocko SUSE Labs

Re: [PATCH v3 4/5] hugetlb: convert PageHugeTemporary() to HPageTemporary flag

2021-01-27 Thread Michal Hocko
aling with a hugetlb page. > Therefore, the check can be eliminated. A dedicated flag is definitely something I would go for if that was available back then when I introduced it. > Signed-off-by: Mike Kravetz > Reviewed-by: Oscar Salvador Acked-by: Michal Hocko Thanks! > --- > i

Re: [PATCH v3 3/5] hugetlb: only set HPageMigratable for migratable hstates

2021-01-27 Thread Michal Hocko
igration can fail even on those pages for which migration is supported right? So I am not really sure this is an improvement in the end. But up to you I do not really have a strong opinion here. -- Michal Hocko SUSE Labs

Re: [PATCH v3 1/5] hugetlb: use page.private for hugetlb specific page flags

2021-01-27 Thread Michal Hocko
. Can we document that the pool is stored in head+1->private here please? > + */ > +enum hugetlb_page_flags { > + HPG_restore_reserve = 0, > + __NR_HPAGEFLAGS, > +}; -- Michal Hocko SUSE Labs

Re: [PATCH v3 2/5] hugetlb: convert page_huge_active() HPageMigratable flag

2021-01-27 Thread Michal Hocko
Offline code calling > scan_movable_pages already deals with these races, so removing the check > is acceptable. Add comment to racy code. > > Signed-off-by: Mike Kravetz > Reviewed-by: Oscar Salvador > Reviewed-by: Muchun Song > Reviewed-by: Miaohe Lin Acked-by: Michal

Re: [PATCH v3 1/5] hugetlb: use page.private for hugetlb specific page flags

2021-01-27 Thread Michal Hocko
Or would that be too confusing? > Conversion of other state information will happen in subsequent patches. > > Signed-off-by: Mike Kravetz I do not see any problems with the patch. I hope I haven't overlooked anything... Acked-by: Michal Hocko > --- > fs/hugetlbfs/inode.c

Re: [PATCH] procfs/dmabuf: Add /proc//task//dmabuf_fds

2021-01-27 Thread Michal Hocko
_lseek, > + .release = proc_dmabuf_fds_release, > +}; > + > diff --git a/fs/proc/internal.h b/fs/proc/internal.h > index f60b379dcdc7..4ca74220db9c 100644 > --- a/fs/proc/internal.h > +++ b/fs/proc/internal.h > @@ -303,6 +303,7 @@ extern const struct file_operations > proc_pid_smaps_operations; > extern const struct file_operations proc_pid_smaps_rollup_operations; > extern const struct file_operations proc_clear_refs_operations; > extern const struct file_operations proc_pagemap_operations; > +extern const struct file_operations proc_tid_dmabuf_fds_operations; > > extern unsigned long task_vsize(struct mm_struct *); > extern unsigned long task_statm(struct mm_struct *, > diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h > index cf72699cb2bc..087e11f7f193 100644 > --- a/include/linux/dma-buf.h > +++ b/include/linux/dma-buf.h > @@ -27,6 +27,11 @@ struct device; > struct dma_buf; > struct dma_buf_attachment; > > +/** > + * Check if struct file* is associated with dma_buf. > + */ > +int is_dma_buf_file(struct file *file); > + > /** > * struct dma_buf_ops - operations possible on struct dma_buf > * @vmap: [optional] creates a virtual mapping for the buffer into kernel > -- > 2.30.0.280.ga3ce27912f-goog -- Michal Hocko SUSE Labs

Re: [PATCH v4 4/4] dma-buf: heaps: add chunk heap to dmabuf heaps

2021-01-27 Thread Michal Hocko
On Tue 26-01-21 11:25:36, Minchan Kim wrote: > On Tue, Jan 26, 2021 at 08:46:05AM +0100, Michal Hocko wrote: > > On Thu 21-01-21 09:55:02, Minchan Kim wrote: > > > From: Hyesoo Yu > > > > > > This patch supports chunk heap that allocates the buffers that >

Re: [PATCH v4 2/4] mm: failfast mode with __GFP_NORETRY in alloc_contig_range

2021-01-27 Thread Michal Hocko
On Tue 26-01-21 11:10:18, Minchan Kim wrote: > On Tue, Jan 26, 2021 at 08:44:49AM +0100, Michal Hocko wrote: > > On Mon 25-01-21 11:33:36, Minchan Kim wrote: > > > On Mon, Jan 25, 2021 at 02:12:00PM +0100, Michal Hocko wrote: > > > > On Thu 21-01-

Re: [PATCH] mm/filemap: Adding missing mem_cgroup_uncharge() to __add_to_page_cache_locked()

2021-01-26 Thread Michal Hocko
gt;mm, gfp); > if (error) > goto error; > + charged = true; > } > > gfp &= GFP_RECLAIM_MASK; > @@ -896,6 +898,8 @@ noinline int __add_to_page_cache_locked(struct page *page, > > if (xas_error(&xas)) { > error = xas_error(&xas); > + if (charged) > + mem_cgroup_uncharge(page); > goto error; > } > > -- > 2.18.1 > -- Michal Hocko SUSE Labs

Re: [PATCH v16 06/11] mm: introduce memfd_secret system call to create "secret" memory areas

2021-01-26 Thread Michal Hocko
On Tue 26-01-21 10:33:11, Mike Rapoport wrote: > On Tue, Jan 26, 2021 at 08:16:14AM +0100, Michal Hocko wrote: > > On Mon 25-01-21 23:36:18, Mike Rapoport wrote: > > > On Mon, Jan 25, 2021 at 06:01:22PM +0100, Michal Hocko wrote: > > > > On Thu 21-01-21

Re: [PATCH] mm/filemap: Adding missing mem_cgroup_uncharge() to __add_to_page_cache_locked()

2021-01-26 Thread Michal Hocko
On Mon 25-01-21 13:57:18, Waiman Long wrote: > On 1/25/21 1:52 PM, Johannes Weiner wrote: > > On Mon, Jan 25, 2021 at 01:23:58PM -0500, Waiman Long wrote: > > > On 1/25/21 1:14 PM, Michal Hocko wrote: [...] > > > > With the proposed simplification by Willy

Re: [PATCH v4 4/4] dma-buf: heaps: add chunk heap to dmabuf heaps

2021-01-26 Thread Michal Hocko
llowing Rob Herring's suggestion in [1]. > > [1] > https://lore.kernel.org/lkml/20191025225009.50305-2-john.stu...@linaro.org/T/#m3dc63acd33fea269a584f43bb799a876f0b2b45d Who is using this allocator in the kernel? -- Michal Hocko SUSE Labs

Re: [PATCH v4 2/4] mm: failfast mode with __GFP_NORETRY in alloc_contig_range

2021-01-26 Thread Michal Hocko
On Mon 25-01-21 11:33:36, Minchan Kim wrote: > On Mon, Jan 25, 2021 at 02:12:00PM +0100, Michal Hocko wrote: > > On Thu 21-01-21 09:55:00, Minchan Kim wrote: > > > Contiguous memory allocation can be stalled due to waiting > > > on page writeback and/or page lock

Re: [PATCH v4 1/4] mm: cma: introduce gfp flag in cma_alloc instead of no_warn

2021-01-26 Thread Michal Hocko
On Mon 25-01-21 11:42:34, Minchan Kim wrote: > On Mon, Jan 25, 2021 at 02:07:01PM +0100, Michal Hocko wrote: > > On Thu 21-01-21 09:54:59, Minchan Kim wrote: > > > The upcoming patch will introduce __GFP_NORETRY semantic > > > in alloc_contig_range which is

Re: [PATCH v16 08/11] secretmem: add memcg accounting

2021-01-26 Thread Michal Hocko
On Mon 25-01-21 23:38:17, Mike Rapoport wrote: > On Mon, Jan 25, 2021 at 05:54:51PM +0100, Michal Hocko wrote: > > On Thu 21-01-21 14:27:20, Mike Rapoport wrote: > > > From: Mike Rapoport > > > > > > Account memory consumed by secretmem to memcg. The account

Re: [PATCH v16 08/11] secretmem: add memcg accounting

2021-01-26 Thread Michal Hocko
fusion. Maybe they shouldn't be so special and they should live on unevistable LRU and get their stats automagically. I definitely do agree that this would be a better fit than NR_SLAB abuse. But considering that this is somehow even more special than mlock then a dedicated counter sounds as even better fit. -- Michal Hocko SUSE Labs

Re: Test report for kernel direct mapping performance

2021-01-26 Thread Michal Hocko
resentative workloads. Some of them tend to be rather narrow in executed code paths or data structures used AFAIU. Is it possible they simply didn't generate sufficient TLB pressure? Have you tried to look closer on profiles of respective configurations where the overhead comes from? -- Michal Hocko SUSE Labs

Re: [RFC PATCH v0] mm/slub: Let number of online CPUs determine the slub page order

2021-01-26 Thread Michal Hocko
On Tue 26-01-21 14:38:14, Vincent Guittot wrote: > On Tue, 26 Jan 2021 at 09:52, Michal Hocko wrote: > > > > On Thu 21-01-21 19:19:21, Vlastimil Babka wrote: > > [...] > > > We could also start questioning the very assumption that number of cpus > > > sh

Re: [PATCH v2 1/1] mm/madvise: replace ptrace attach requirement for process_madvise

2021-01-26 Thread Michal Hocko
On Wed 20-01-21 14:17:39, Jann Horn wrote: > On Wed, Jan 13, 2021 at 3:22 PM Michal Hocko wrote: > > On Tue 12-01-21 09:51:24, Suren Baghdasaryan wrote: > > > On Tue, Jan 12, 2021 at 9:45 AM Oleg Nesterov wrote: > > > > > > > > On 01/12, Michal Hocko wro

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-01-26 Thread Michal Hocko
On Tue 26-01-21 12:56:48, David Hildenbrand wrote: > On 26.01.21 12:46, Michal Hocko wrote: > > On Thu 21-01-21 14:27:19, Mike Rapoport wrote: > > > From: Mike Rapoport > > > > > > Removing a PAGE_SIZE page from the direct map every time such page is > &g

Re: [PATCH v16 07/11] secretmem: use PMD-size pages to amortize direct map fragmentation

2021-01-26 Thread Michal Hocko
panic the system. Now you could be less drastic and only make SIGBUS on fault but that would be still quite terrible. There is a very good reason why hugetlb implements is non-trivial reservation system to avoid exactly these problems. So unless I am really misreading the code Nacked-by: Michal Ho

Re: [PATCH v16 06/11] mm: introduce memfd_secret system call to create "secret" memory areas

2021-01-26 Thread Michal Hocko
On Tue 26-01-21 10:53:08, David Hildenbrand wrote: [...] > I assume you've seen the benchmark results provided by Xing Zhengjun > > https://lore.kernel.org/linux-mm/213b4567-46ce-f116-9cdf-bbd0c884e...@linux.intel.com/ I was not. Thanks for the pointer. I will have a look. -- Mic

Re: [PATCH v16 06/11] mm: introduce memfd_secret system call to create "secret" memory areas

2021-01-26 Thread Michal Hocko
On Tue 26-01-21 11:20:11, Mike Rapoport wrote: > On Tue, Jan 26, 2021 at 10:00:13AM +0100, Michal Hocko wrote: > > On Tue 26-01-21 10:33:11, Mike Rapoport wrote: > > > On Tue, Jan 26, 2021 at 08:16:14AM +0100, Michal Hocko wrote: > > > > On Mon 25-01-21 23:36:18, Mik

Re: [PATCH v16 06/11] mm: introduce memfd_secret system call to create "secret" memory areas

2021-01-26 Thread Michal Hocko
On Tue 26-01-21 10:00:14, Michal Hocko wrote: > On Tue 26-01-21 10:33:11, Mike Rapoport wrote: > > On Tue, Jan 26, 2021 at 08:16:14AM +0100, Michal Hocko wrote: > > > On Mon 25-01-21 23:36:18, Mike Rapoport wrote: > > > > On Mon, Jan 25, 2021 at 06:01:22PM +0100, Mi

Re: [RFC PATCH v0] mm/slub: Let number of online CPUs determine the slub page order

2021-01-26 Thread Michal Hocko
ould be focusing on the actual reasons for the performance impact than playing with some fancy math and tuning for a benchmark on a particular machine which doesn't work for others due to subtle initialization timing issues. Fundamentally why should higher number of CPUs imply the size of slab in the first place? -- Michal Hocko SUSE Labs

Re: [PATCH v16 08/11] secretmem: add memcg accounting

2021-01-26 Thread Michal Hocko
On Tue 26-01-21 10:56:54, Mike Rapoport wrote: > On Tue, Jan 26, 2021 at 08:31:42AM +0100, Michal Hocko wrote: > > On Mon 25-01-21 23:38:17, Mike Rapoport wrote: > > > On Mon, Jan 25, 2021 at 05:54:51PM +0100, Michal Hocko wrote: > > > > On Thu 21-01-21

Re: [PATCH v16 06/11] mm: introduce memfd_secret system call to create "secret" memory areas

2021-01-26 Thread Michal Hocko
On Mon 25-01-21 23:36:18, Mike Rapoport wrote: > On Mon, Jan 25, 2021 at 06:01:22PM +0100, Michal Hocko wrote: > > On Thu 21-01-21 14:27:18, Mike Rapoport wrote: > > > From: Mike Rapoport > > > > > > Introduce "memfd_secret" system call with the abi

<    1   2   3   4   5   6   7   8   9   10   >