Re: [PATCH v2 6/6] mm: export dump_mm()

2023-01-25 Thread Michal Hocko
; > Signed-off-by: Suren Baghdasaryan Acked-by: Michal Hocko > --- > mm/debug.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/mm/debug.c b/mm/debug.c > index 9d3d893dc7f4..96d594e16292 100644 > --- a/mm/debug.c > +++ b/mm/debug.c > @@ -215,6 +215

Re: [PATCH v2 2/6] mm: replace VM_LOCKED_CLEAR_MASK with VM_LOCKED_MASK

2023-01-25 Thread Michal Hocko
On Wed 25-01-23 00:38:47, Suren Baghdasaryan wrote: > To simplify the usage of VM_LOCKED_CLEAR_MASK in clear_vm_flags(), > replace it with VM_LOCKED_MASK bitmask and convert all users. > > Signed-off-by: Suren Baghdasaryan Acked-by: Michal Hocko > --- > include/linux/mm.h

Re: [PATCH v2 4/6] mm: replace vma->vm_flags indirect modification in ksm_madvise

2023-01-25 Thread Michal Hocko
cation attempts. Those BUG_ONs scream to much IMHO. KSM is an MM internal code so I gueess we should be willing to trust it. > Signed-off-by: Suren Baghdasaryan Acked-by: Michal Hocko -- Michal Hocko SUSE Labs

Re: [PATCH v2 1/6] mm: introduce vma->vm_flags modifier functions

2023-01-25 Thread Michal Hocko
; operations. Introduce modifier functions for vm_flags to be used whenever > flags are updated. This way we can better check and control correct > locking behavior during these updates. > > Signed-off-by: Suren Baghdasaryan Acked-by: Michal Hocko > --- >

Re: [PATCH v2 3/6] mm: replace vma->vm_flags direct modifications with modifier calls

2023-01-25 Thread Michal Hocko
sors which would also prevent any future direct setting of those flags in uncontrolled way as well. Anyway Acked-by: Michal Hocko -- Michal Hocko SUSE Labs

Re: [PATCH v2 5/6] mm: introduce mod_vm_flags_nolock and use it in untrack_pfn

2023-01-25 Thread Michal Hocko
tistics and freeing VMAs */ > mas_set(_detach, start); > remove_mt(mm, _detach); > @@ -2704,7 +2708,7 @@ unsigned long mmap_region(struct file *file, unsigned > long addr, > > /* Undo any partial mapping done by a device driver. */ > unmap_region(mm, >mm_mt, vma, prev, next, vma->vm_start, > - vma->vm_end); > + vma->vm_end, true); > } > if (file && (vm_flags & VM_SHARED)) > mapping_unmap_writable(file->f_mapping); > @@ -3031,7 +3035,7 @@ void exit_mmap(struct mm_struct *mm) > tlb_gather_mmu_fullmm(, mm); > /* update_hiwater_rss(mm) here? but nobody should be looking */ > /* Use ULONG_MAX here to ensure all VMAs in the mm are unmapped */ > - unmap_vmas(, >mm_mt, vma, 0, ULONG_MAX); > + unmap_vmas(, >mm_mt, vma, 0, ULONG_MAX, false); > mmap_read_unlock(mm); > > /* > -- > 2.39.1 -- Michal Hocko SUSE Labs

Re: [PATCH v2 4/6] mm: replace vma->vm_flags indirect modification in ksm_madvise

2023-01-25 Thread Michal Hocko
On Wed 25-01-23 08:57:48, Suren Baghdasaryan wrote: > On Wed, Jan 25, 2023 at 1:38 AM 'Michal Hocko' via kernel-team > wrote: > > > > On Wed 25-01-23 00:38:49, Suren Baghdasaryan wrote: > > > Replace indirect modifications to vma->vm_flags with calls to modif

Re: [PATCH v2 1/4] memcg: Track exported dma-buffers

2023-01-25 Thread Michal Hocko
On Tue 24-01-23 10:55:21, T.J. Mercier wrote: > On Tue, Jan 24, 2023 at 7:00 AM Michal Hocko wrote: > > > > On Mon 23-01-23 19:17:23, T.J. Mercier wrote: > > > When a buffer is exported to userspace, use memcg to attribute the > > > buffer to the allocating c

Re: [PATCH v2 1/4] memcg: Track exported dma-buffers

2023-01-25 Thread Michal Hocko
On Tue 24-01-23 19:46:28, Shakeel Butt wrote: > On Tue, Jan 24, 2023 at 03:59:58PM +0100, Michal Hocko wrote: > > On Mon 23-01-23 19:17:23, T.J. Mercier wrote: > > > When a buffer is exported to userspace, use memcg to attribute the > > > buffer to the allocating cgroup

Re: [PATCH v2 1/4] memcg: Track exported dma-buffers

2023-01-24 Thread Michal Hocko
o reclaim the memory and even trigger the memcg oom killer if the request size is <= 8 pages. Is this a desirable behavior? -- Michal Hocko SUSE Labs

Re: [PATCH 0/4] Track exported dma-buffers with memcg

2023-01-12 Thread Michal Hocko
reclaimable. With a dedicated counter an excessive dmabuf usage would be visible in the oom report because we do print memcg stats. It is definitely preferable to have a shrinker mechanism but if that is to be done in a follow up step then this is acceptable. But leaving out charging from early on sounds like a bad choice to me. -- Michal Hocko SUSE Labs

Re: [PATCH 1/4] memcg: Track exported dma-buffers

2023-01-10 Thread Michal Hocko
any memory (at least not for non-root memcgs). -- Michal Hocko SUSE Labs

Re: [PATCH v5] dma-buf: Add DmaBufTotal counter in meminfo

2021-04-21 Thread Michal Hocko
hen it essentially makes it impossible to know whether an excessive usage contribues to RAM or device memory depletion - hence this is completely bogus for the OOM report. -- Michal Hocko SUSE Labs ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v4] dma-buf: Add DmaBufTotal counter in meminfo

2021-04-20 Thread Michal Hocko
On Tue 20-04-21 09:25:51, peter.enderb...@sony.com wrote: > On 4/20/21 11:12 AM, Michal Hocko wrote: > > On Tue 20-04-21 09:02:57, peter.enderb...@sony.com wrote: > >>>> But that isn't really system memory at all, it's just allocated device > >>>> memory. &g

Re: [PATCH v4] dma-buf: Add DmaBufTotal counter in meminfo

2021-04-20 Thread Michal Hocko
a single counter without a wider context cannot be put into any reasonable context. There is no notion of the total amount of device memory usable for dma-buf. As Christian explained some of it can be RAM based. So a single number is rather pointless on its own in many cases. Or let me just ask. What can

Re: [PATCH 0/2 V6]Add dma-buf counter

2021-04-20 Thread Michal Hocko
now replaced with dma-buf. ION had some overview metrics that was > similar. The discussion around the previous version is still not over and as it seems your proposed approach is not really viable. So please do not send new versions until that is sorted out. Thanks! -- Michal

Re: [PATCH v4] dma-buf: Add DmaBufTotal counter in meminfo

2021-04-20 Thread Michal Hocko
On Tue 20-04-21 10:00:07, Christian König wrote: > Am 20.04.21 um 09:46 schrieb Michal Hocko: > > On Tue 20-04-21 09:32:14, Christian König wrote: > > > Am 20.04.21 um 09:04 schrieb Michal Hocko: > > > > On Mon 19-04-21 18:37:13, Christian König wrote: > > >

Re: [PATCH v4] dma-buf: Add DmaBufTotal counter in meminfo

2021-04-20 Thread Michal Hocko
On Tue 20-04-21 10:20:43, Mike Rapoport wrote: > On Tue, Apr 20, 2021 at 09:04:51AM +0200, Michal Hocko wrote: > > On Mon 19-04-21 18:37:13, Christian König wrote: > > > Am 19.04.21 um 18:11 schrieb Michal Hocko: > > [...] > > > > The question is not

Re: [PATCH v4] dma-buf: Add DmaBufTotal counter in meminfo

2021-04-20 Thread Michal Hocko
On Tue 20-04-21 09:32:14, Christian König wrote: > Am 20.04.21 um 09:04 schrieb Michal Hocko: > > On Mon 19-04-21 18:37:13, Christian König wrote: > > > Am 19.04.21 um 18:11 schrieb Michal Hocko: [...] > > What I am trying to bring up with NUMA side is that the same probl

Re: [PATCH v4] dma-buf: Add DmaBufTotal counter in meminfo

2021-04-20 Thread Michal Hocko
On Mon 19-04-21 18:37:13, Christian König wrote: > Am 19.04.21 um 18:11 schrieb Michal Hocko: [...] > > The question is not whether it is NUMA aware but whether it is useful to > > know per-numa data for the purpose the counter is supposed to serve. > > No, not at all. The

Re: [PATCH v4] dma-buf: Add DmaBufTotal counter in meminfo

2021-04-19 Thread Michal Hocko
On Mon 19-04-21 17:44:13, Christian König wrote: > Am 19.04.21 um 17:19 schrieb peter.enderb...@sony.com: > > On 4/19/21 5:00 PM, Michal Hocko wrote: > > > On Mon 19-04-21 12:41:58, peter.enderb...@sony.com wrote: > > > > On 4/19/21 2:16 PM, Michal Hocko wrote: >

Re: [PATCH v4] dma-buf: Add DmaBufTotal counter in meminfo

2021-04-19 Thread Michal Hocko
On Mon 19-04-21 12:41:58, peter.enderb...@sony.com wrote: > On 4/19/21 2:16 PM, Michal Hocko wrote: > > On Sat 17-04-21 12:40:32, Peter Enderborg wrote: > >> This adds a total used dma-buf memory. Details > >> can be found in debugfs, however it is not for everyone &g

Re: [PATCH v4] dma-buf: Add DmaBufTotal counter in meminfo

2021-04-19 Thread Michal Hocko
ter explanation and secondly is this information useful for OOM situations analysis? If yes then show_mem should dump the value as well. >From the implementation point of view, is there any reason why this hasn't used the existing global_node_page_state infrastructure? -- Mich

Re: [PATCH] drm/ttm: stop warning on TT shrinker failure

2021-03-23 Thread Michal Hocko
On Tue 23-03-21 14:56:54, Christian König wrote: > Am 23.03.21 um 14:41 schrieb Michal Hocko: [...] > > Anyway, I am wondering whether the overall approach is sound. Why don't > > you simply use shmem as your backing storage from the beginning and pin > > those pages if they a

Re: [PATCH] drm/ttm: stop warning on TT shrinker failure

2021-03-23 Thread Michal Hocko
On Tue 23-03-21 14:15:05, Daniel Vetter wrote: > On Tue, Mar 23, 2021 at 01:04:03PM +0100, Michal Hocko wrote: > > On Tue 23-03-21 12:48:58, Christian König wrote: > > > Am 23.03.21 um 12:28 schrieb Daniel Vetter: > > > > On Tue, Mar 23, 2021 at 08:38:33AM +0100, M

Re: [PATCH] drm/ttm: stop warning on TT shrinker failure

2021-03-23 Thread Michal Hocko
On Tue 23-03-21 14:06:25, Christian König wrote: > Am 23.03.21 um 13:37 schrieb Michal Hocko: > > On Tue 23-03-21 13:21:32, Christian König wrote: [...] > > > Ideally I would like to be able to trigger swapping out the shmem page I > > > allocated immediately after do

Re: [PATCH] drm/ttm: stop warning on TT shrinker failure

2021-03-23 Thread Michal Hocko
On Tue 23-03-21 13:21:32, Christian König wrote: > Am 23.03.21 um 13:04 schrieb Michal Hocko: > > On Tue 23-03-21 12:48:58, Christian König wrote: > > > Am 23.03.21 um 12:28 schrieb Daniel Vetter: > > > > On Tue, Mar 23, 2021 at 08:38:33AM +0100, Michal Hocko wrote: &

Re: [PATCH] drm/ttm: stop warning on TT shrinker failure

2021-03-23 Thread Michal Hocko
On Tue 23-03-21 12:51:13, Christian König wrote: > > > Am 23.03.21 um 12:46 schrieb Michal Hocko: > > On Tue 23-03-21 12:28:20, Daniel Vetter wrote: > > > On Tue, Mar 23, 2021 at 08:38:33AM +0100, Michal Hocko wrote: > > [...] > > > > > >

Re: [PATCH] drm/ttm: stop warning on TT shrinker failure

2021-03-23 Thread Michal Hocko
On Tue 23-03-21 12:48:58, Christian König wrote: > Am 23.03.21 um 12:28 schrieb Daniel Vetter: > > On Tue, Mar 23, 2021 at 08:38:33AM +0100, Michal Hocko wrote: > > > On Mon 22-03-21 20:34:25, Christian König wrote: [...] > > > > My only concern is that if I could r

Re: [PATCH] drm/ttm: stop warning on TT shrinker failure

2021-03-23 Thread Michal Hocko
On Tue 23-03-21 12:28:20, Daniel Vetter wrote: > On Tue, Mar 23, 2021 at 08:38:33AM +0100, Michal Hocko wrote: [...] > > > > fs_reclaim_acquire is there to make sure lockdep understands that this > > > > is a shrinker and that it checks all the dependencies for us like

Re: [PATCH] drm/ttm: stop warning on TT shrinker failure

2021-03-23 Thread Michal Hocko
On Mon 22-03-21 20:34:25, Christian König wrote: > Am 22.03.21 um 18:02 schrieb Daniel Vetter: > > On Mon, Mar 22, 2021 at 5:06 PM Michal Hocko wrote: > > > On Mon 22-03-21 14:05:48, Matthew Wilcox wrote: > > > > On Mon, Mar 22, 2021 at 02:49:27PM +0100, Daniel Vette

Re: [PATCH] drm/ttm: stop warning on TT shrinker failure

2021-03-22 Thread Michal Hocko
since iirc Willy or Dave told me the opposite, and > > the memalloc_no* stuff is for e.g. nfs calling into network layer (needs > > GFP_NOFS) or swap on top of a filesystems (even needs GFP_NOIO I think). > > > > Adding them, maybe I got confused. > >

Re: [PATCH] procfs/dmabuf: Add /proc//task//dmabuf_fds

2021-01-28 Thread Michal Hocko
overhead as being the main concern as oom is a real cold path. Anyway, if you want to revamp those patches then feel free to CC me and we can discuss further. I do not want to hijack this thread by an unrelated topic. -- Michal Hocko SUSE Labs ___ dri-deve

Re: [PATCH] procfs/dmabuf: Add /proc//task//dmabuf_fds

2021-01-28 Thread Michal Hocko
On Wed 27-01-21 12:08:50, Christian König wrote: > Am 27.01.21 um 12:02 schrieb Michal Hocko: > > On Wed 27-01-21 11:53:55, Christian König wrote: > > [...] > > > In general processes are currently not held accountable for memory they > > > reference through their

Re: [PATCH] procfs/dmabuf: Add /proc//task//dmabuf_fds

2021-01-28 Thread Michal Hocko
se= proc_dmabuf_fds_release, > +}; > + > diff --git a/fs/proc/internal.h b/fs/proc/internal.h > index f60b379dcdc7..4ca74220db9c 100644 > --- a/fs/proc/internal.h > +++ b/fs/proc/internal.h > @@ -303,6 +303,7 @@ extern const struct file_operations > proc_pid_smaps_operations; > extern const struct file_operations proc_pid_smaps_rollup_operations; > extern const struct file_operations proc_clear_refs_operations; > extern const struct file_operations proc_pagemap_operations; > +extern const struct file_operations proc_tid_dmabuf_fds_operations; > > extern unsigned long task_vsize(struct mm_struct *); > extern unsigned long task_statm(struct mm_struct *, > diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h > index cf72699cb2bc..087e11f7f193 100644 > --- a/include/linux/dma-buf.h > +++ b/include/linux/dma-buf.h > @@ -27,6 +27,11 @@ struct device; > struct dma_buf; > struct dma_buf_attachment; > > +/** > + * Check if struct file* is associated with dma_buf. > + */ > +int is_dma_buf_file(struct file *file); > + > /** > * struct dma_buf_ops - operations possible on struct dma_buf > * @vmap: [optional] creates a virtual mapping for the buffer into kernel > -- > 2.30.0.280.ga3ce27912f-goog -- Michal Hocko SUSE Labs ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH] procfs/dmabuf: Add /proc//task//dmabuf_fds

2021-01-28 Thread Michal Hocko
consider shmem in badness so this wouldn't go out of line. Kernel oom killer could be more clever with these special fds though and query for buffer size directly. -- Michal Hocko SUSE Labs ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH] procfs/dmabuf: Add /proc//task//dmabuf_fds

2021-01-28 Thread Michal Hocko
vel and/or oom_badness doesn't take them into consideration. -- Michal Hocko SUSE Labs ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [patch 00/13] preempt: Make preempt count unconditional

2020-09-30 Thread Michal Hocko
as to carefuly consider failure. This is not a random allocation mode. -- Michal Hocko SUSE Labs ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [patch 00/13] preempt: Make preempt count unconditional

2020-09-30 Thread Michal Hocko
On Tue 29-09-20 11:00:03, Daniel Vetter wrote: > On Tue, Sep 29, 2020 at 10:19:38AM +0200, Michal Hocko wrote: > > On Wed 16-09-20 23:43:02, Daniel Vetter wrote: > > > I can > > > then figure out whether it's better to risk not spotting issues with > > > call_

Re: next-20200515: Xorg killed due to "OOM"

2020-06-01 Thread Michal Hocko
On Sun 31-05-20 14:16:01, Pavel Machek wrote: > On Thu 2020-05-28 14:07:50, Michal Hocko wrote: > > On Thu 28-05-20 14:03:54, Pavel Machek wrote: > > > On Thu 2020-05-28 11:05:17, Michal Hocko wrote: > > > > On Tue 26-05-20 11:10:54, Pavel Machek wrote: > &g

Re: next-20200515: Xorg killed due to "OOM"

2020-05-28 Thread Michal Hocko
On Thu 28-05-20 14:03:54, Pavel Machek wrote: > On Thu 2020-05-28 11:05:17, Michal Hocko wrote: > > On Tue 26-05-20 11:10:54, Pavel Machek wrote: > > [...] > > > [38617.276517] oom_reaper: reaped process 31769 (chromium), now > > > anon-rss:0kB, file-rss:0kB, shm

Re: next-20200515: Xorg killed due to "OOM"

2020-05-28 Thread Michal Hocko
ler has been invoked is that the page fault handler has returned VM_FAULT_OOM. So this is not a result of the page allocator struggling to allocate a memory. It would be interesting to check which code path has returned this. -- Michal Hocko SUSE Labs _

Re: [PATCH v4 0/9] Huge page-table entries for TTM

2020-03-03 Thread Michal Hocko
On Fri 28-02-20 14:08:04, Thomas Hellström (VMware) wrote: > Andrew, Michal > > I'm wondering what's the best way here to get the patches touching mm > reviewed and accepted? I am sorry, but I am busy with other stuff and unlikely to find time to review this series. -- Michal Hoc

Re: [PATCH v4 1/2] mm: Add a vmf_insert_mixed_prot() function

2019-12-20 Thread Michal Hocko
m functionality as vmf_insert_mixed_prot(). > > Cc: Andrew Morton > Cc: Michal Hocko > Cc: "Matthew Wilcox (Oracle)" > Cc: "Kirill A. Shutemov" > Cc: Ralph Campbell > Cc: "Jérôme Glisse" > Cc: "Christian König" > Signed-off-by: Thomas Hellstrom

Re: [PATCH v3 2/2] mm, drm/ttm: Fix vm page protection handling

2019-12-06 Thread Michal Hocko
On Fri 06-12-19 14:16:10, Thomas Hellstrom wrote: > Hi Michal, > > On Fri, 2019-12-06 at 11:30 +0100, Michal Hocko wrote: > > On Fri 06-12-19 09:24:26, Thomas Hellström (VMware) wrote: > > [...] > > > @@ -283,11 +282,26 @@ vm_fault_t ttm_bo_vm_fault_reser

Re: [PATCH v3 2/2] mm, drm/ttm: Fix vm page protection handling

2019-12-06 Thread Michal Hocko
ery valuable piece of information I believe we need to document this in the generic code where everybody will find it. vmf_insert_mixed_prot sounds like a good place to me. So being explicit about VM_MIXEDMAP. Also a reference from vm_page_prot to this function would be really helpeful. Thanks! -- Michal Hock

Re: [PATCH v2 2/2] drm/ttm: Fix vm page protection handling

2019-12-04 Thread Michal Hocko
On Wed 04-12-19 16:19:27, Thomas Hellström (VMware) wrote: > On 12/4/19 3:42 PM, Michal Hocko wrote: > > On Wed 04-12-19 15:36:58, Thomas Hellström (VMware) wrote: > > > On 12/4/19 3:35 PM, Michal Hocko wrote: > > > > On Wed 04-12-19 15:16:09, Thomas Hellström (VMwar

Re: [PATCH v2 2/2] drm/ttm: Fix vm page protection handling

2019-12-04 Thread Michal Hocko
On Wed 04-12-19 15:36:58, Thomas Hellström (VMware) wrote: > On 12/4/19 3:35 PM, Michal Hocko wrote: > > On Wed 04-12-19 15:16:09, Thomas Hellström (VMware) wrote: > > > On 12/4/19 2:52 PM, Michal Hocko wrote: > > > > On Tue 03-12-19 11:48:53, Thomas Hellström

Re: [PATCH v2 2/2] drm/ttm: Fix vm page protection handling

2019-12-04 Thread Michal Hocko
On Wed 04-12-19 15:16:09, Thomas Hellström (VMware) wrote: > On 12/4/19 2:52 PM, Michal Hocko wrote: > > On Tue 03-12-19 11:48:53, Thomas Hellström (VMware) wrote: > > > From: Thomas Hellstrom > > > > > > TTM graphics buffer objects may, transparently t

Re: [PATCH v2 2/2] drm/ttm: Fix vm page protection handling

2019-12-04 Thread Michal Hocko
essentially this should have any new side effect on functionality it is just making a hacky/ugly code less so? In other words what are the consequences of having page protection inconsistent from vma's? > Cc: Andrew Morton > Cc: Michal Hocko > Cc: "Matthew Wilcox (Oracle)" >

Re: [PATCH v2 1/2] mm: Add and export vmf_insert_mixed_prot()

2019-12-04 Thread Michal Hocko
_prot(). A new symbol should be added in a patch that adds a new user which is not the case here. -- Michal Hocko SUSE Labs ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH RFC v4 00/16] new cgroup controller for gpu/drm subsystem

2019-09-10 Thread Michal Hocko
On Tue 10-09-19 09:03:29, Tejun Heo wrote: > Hello, Michal. > > On Tue, Sep 10, 2019 at 01:54:48PM +0200, Michal Hocko wrote: > > > So, while it'd great to have shrinkers in the longer term, it's not a > > > strict requirement to be accounted in memcg. It already acc

Re: [PATCH RFC v4 00/16] new cgroup controller for gpu/drm subsystem

2019-09-10 Thread Michal Hocko
f the amount of memory charged like that is negligible to the overall size. But from the discussion it seems that these buffers are really large. -- Michal Hocko SUSE Labs ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 3/5] kernel.h: Add non_block_start/end()

2019-08-28 Thread Michal Hocko
s on top, but Michal > said those are less of a problem because spinlocks can't have an > indirect dependency upon the page allocator and hence close the loop > with the oom reaper. > > Suggested by Michal Hocko. > > v2: > - Improve commit message (Michal) > - Also check

Re: [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-20 Thread Michal Hocko
On Fri 16-08-19 11:31:45, Jason Gunthorpe wrote: > On Fri, Aug 16, 2019 at 02:26:25PM +0200, Michal Hocko wrote: [...] > > I believe I have given some examples when introducing __GFP_NOLOCKDEP. > > Okay, I think that is 7e7844226f10 ("lockdep: allow to disable reclaim >

Re: [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-16 Thread Michal Hocko
On Fri 16-08-19 09:19:06, Jason Gunthorpe wrote: > On Fri, Aug 16, 2019 at 10:10:29AM +0200, Michal Hocko wrote: > > On Thu 15-08-19 17:13:23, Jason Gunthorpe wrote: > > > On Thu, Aug 15, 2019 at 09:35:26PM +0200, Michal Hocko wrote: > > > > > > > > The l

Re: [Intel-gfx] [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-16 Thread Michal Hocko
On Thu 15-08-19 22:16:43, Daniel Vetter wrote: > On Thu, Aug 15, 2019 at 9:35 PM Michal Hocko wrote: [...] > > > The last detail is I'm still unclear what a GFP flags a blockable > > > invalidate_range_start() should use. Is GFP_KERNEL OK? > > > > I hope

Re: [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-16 Thread Michal Hocko
On Thu 15-08-19 15:15:09, Andrew Morton wrote: > On Thu, 15 Aug 2019 10:44:29 +0200 Michal Hocko wrote: > > > > I continue to struggle with this. It introduces a new kernel state > > > "running preemptibly but must not call schedule()". How does this make >

Re: [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-16 Thread Michal Hocko
On Thu 15-08-19 17:13:23, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 09:35:26PM +0200, Michal Hocko wrote: > > > > The last detail is I'm still unclear what a GFP flags a blockable > > > invalidate_range_start() should use. Is GFP_KERNEL OK? > > > > I h

Re: [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Michal Hocko
On Thu 15-08-19 16:18:10, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 09:05:25PM +0200, Michal Hocko wrote: > > > This is what you claim and I am saying that fs_reclaim is about a > > restricted reclaim context and it is an ugly hack. It has proven to > > report false

Re: [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Michal Hocko
On Thu 15-08-19 15:24:48, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 07:42:07PM +0200, Michal Hocko wrote: > > On Thu 15-08-19 13:56:31, Jason Gunthorpe wrote: > > > On Thu, Aug 15, 2019 at 06:00:41PM +0200, Michal Hocko wrote: > > > > > > > > AFAIK

Re: [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Michal Hocko
On Thu 15-08-19 13:56:31, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 06:00:41PM +0200, Michal Hocko wrote: > > > > AFAIK 'GFP_NOWAIT' is characterized by the lack of __GFP_FS and > > > __GFP_DIRECT_RECLAIM.. > > > > > > This matches the existi

Re: [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Michal Hocko
On Thu 15-08-19 11:12:19, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 03:21:27PM +0200, Michal Hocko wrote: > > On Thu 15-08-19 09:23:44, Jason Gunthorpe wrote: > > > On Thu, Aug 15, 2019 at 08:58:29AM +0200, Daniel Vetter wrote: > > > > On Wed, Aug 14, 2

Re: [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Michal Hocko
On Thu 15-08-19 10:04:15, Jason Gunthorpe wrote: > On Thu, Aug 15, 2019 at 10:44:29AM +0200, Michal Hocko wrote: > > > As the oom reaper is the primary guarantee of the oom handling forward > > progress it cannot be blocked on anything that might depend on blockable >

Re: [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Michal Hocko
ckable had to do with not recursing into reclaim and deadlocking, > and didn't seem to have much to do with blocking. > > I'm asking if *non-blocking* is really the requirement or if this is > just the usual 'do not deadlock on the allocator' thing reclaim paths > alread have? No, non-blocking is a very coarse approximation of what we really need. But it should give us even a stronger condition. Essentially any sleep other than a preemption shouldn't be allowed in that context. -- Michal Hocko SUSE Labs

Re: [PATCH 2/5] kernel.h: Add non_block_start/end()

2019-08-15 Thread Michal Hocko
t we can end up in a silent hang with an unusable machine. Now we hope for reasonable implementations of mmu notifiers (strong words I know ;) and this should be relatively simple and effective catch all tool to detect something suspicious is going on. Does that make the situation more clear? -- Michal Hocko SUSE Labs

Re: [PATCH 00/34] put_user_pages(): miscellaneous call sites

2019-08-07 Thread Michal Hocko
On Wed 07-08-19 10:37:26, Jan Kara wrote: > On Fri 02-08-19 12:14:09, John Hubbard wrote: > > On 8/2/19 7:52 AM, Jan Kara wrote: > > > On Fri 02-08-19 07:24:43, Matthew Wilcox wrote: > > > > On Fri, Aug 02, 2019 at 02:41:46PM +0200, Jan Kara wrote: > > > >

Re: [PATCH 00/34] put_user_pages(): miscellaneous call sites

2019-08-02 Thread Michal Hocko
e changes that would break the balance though. -- Michal Hocko SUSE Labs ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH] mm/hmm: replace hmm_update with mmu_notifier_range

2019-07-24 Thread Michal Hocko
On Wed 24-07-19 20:56:17, Michal Hocko wrote: > On Wed 24-07-19 15:08:37, Jason Gunthorpe wrote: > > On Wed, Jul 24, 2019 at 07:58:58PM +0200, Michal Hocko wrote: > [...] > > > Maybe new users have started relying on a new semantic in the meantime, > > > back then,

Re: [PATCH] mm/hmm: replace hmm_update with mmu_notifier_range

2019-07-24 Thread Michal Hocko
On Wed 24-07-19 15:08:37, Jason Gunthorpe wrote: > On Wed, Jul 24, 2019 at 07:58:58PM +0200, Michal Hocko wrote: [...] > > Maybe new users have started relying on a new semantic in the meantime, > > back then, none of the notifier has even started any action in blocking >

Re: [PATCH] mm/hmm: replace hmm_update with mmu_notifier_range

2019-07-24 Thread Michal Hocko
ing events? The only place > where mmu_notifier_invalidate_range_start_nonblock is called is the oom > killer, which means the process is about to die and the pagetable will > get torn down ASAP. Should we just skip them unconditionally? How do you tell whether they are going to block or

Re: [PATCH] mm/hmm: replace hmm_update with mmu_notifier_range

2019-07-24 Thread Michal Hocko
r becomes unable to get to 0. > > Below is the best idea I've had so far.. > > Michal, what do you think? IIRC we have discussed this with Jerome back then when I've introduced this code and unless I misremember he said the current code was OK. Maybe new users have started relying on

Re: [PATCH 05/22] mm: export alloc_pages_vma

2019-06-27 Thread Michal Hocko
On Wed 26-06-19 09:14:32, Dan Williams wrote: > On Tue, Jun 25, 2019 at 10:46 PM Michal Hocko wrote: > > > > On Tue 25-06-19 12:52:18, Dan Williams wrote: > > [...] > > > > Documentation/process/stable-api-nonsense.rst > > > > > > That d

Re: [PATCH 06/25] mm: export alloc_pages_vma

2019-06-26 Thread Michal Hocko
On Wed 26-06-19 14:27:05, Christoph Hellwig wrote: > nouveau is currently using this through an odd hmm wrapper, and I plan > to switch it to the real thing later in this series. > > Signed-off-by: Christoph Hellwig > Reviewed-by: John Hubbard Acked-by: Michal Hocko Thank

Re: [PATCH 05/22] mm: export alloc_pages_vma

2019-06-25 Thread Michal Hocko
xported regardless of that document. Can you point me to any specific example where this would be the case for the core kernel symbols please? -- Michal Hocko SUSE Labs ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.or

Re: [PATCH 18/22] mm: mark DEVICE_PUBLIC as broken

2019-06-25 Thread Michal Hocko
to use it. So it really > does seem safe to remove, although of course it's good to start with > BROKEN and see if anyone pops up and complains. Well, I do not really see much of a difference. Preserving an unused code which doesn't have any user in sight just adds a maintenance burden wh

Re: [PATCH 05/22] mm: export alloc_pages_vma

2019-06-25 Thread Michal Hocko
On Tue 25-06-19 11:03:53, Dan Williams wrote: > On Tue, Jun 25, 2019 at 8:01 AM Michal Hocko wrote: > > > > On Tue 25-06-19 09:23:17, Christoph Hellwig wrote: > > > On Mon, Jun 24, 2019 at 11:24:48AM -0700, Dan Williams wrote: > > > > I asked for thi

Re: [PATCH 05/22] mm: export alloc_pages_vma

2019-06-25 Thread Michal Hocko
uld go with consistency and export the same way we do with other functions. Also we do not want people to reinvent this API and screw that like we have seen in other cases when external modules try reimplement core functionality themselves. Thanks! -- Michal Hocko SUSE Labs __

Re: [PATCH 04/22] mm: don't clear ->mapping in hmm_devmem_free

2019-06-20 Thread Michal Hocko
data; > > - page->mapping = NULL; > - > devmem->ops->free(devmem, page); > } > > -- > 2.20.1 -- Michal Hocko SUSE Labs ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 03/22] mm: remove hmm_devmem_add_resource

2019-06-20 Thread Michal Hocko
directly now that we've simplified the API for it. > > Signed-off-by: Christoph Hellwig Acked-by: Michal Hocko > --- > include/linux/hmm.h | 3 --- > mm/hmm.c| 54 - > 2 files changed, 57 deletions(-) > > diff --git a

Re: [PATCH 18/22] mm: mark DEVICE_PUBLIC as broken

2019-06-20 Thread Michal Hocko
emove all the DEVICE_PUBLIC code. > Signed-off-by: Christoph Hellwig Anyway Acked-by: Michal Hocko > --- > mm/Kconfig | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/mm/Kconfig b/mm/Kconfig > index 0d2ba7e1f43e..406fa45e9ecc 100644 > --- a/mm/Kconfig > +++ b/mm/

Re: [PATCH 05/22] mm: export alloc_pages_vma

2019-06-20 Thread Michal Hocko
> } > +EXPORT_SYMBOL_GPL(alloc_pages_vma); All allocator exported symbols are EXPORT_SYMBOL, what is a reason to have this one special? -- Michal Hocko SUSE Labs ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH] kernel.h: Add non_block_start/end()

2019-05-21 Thread Michal Hocko
leep() debug checks. Add a non_block_start/end() > > pair to annotate these. > > Just putting preempt on/off around these is not sufficient? It is not a critical section. It is a _debugging_ facility to help discover blocking contexts. -- Michal Hocko SUSE Labs __

Re: [PATCH] kernel.h: Add non_block_start/end()

2019-05-21 Thread Michal Hocko
s on top, but Michal > said those are less of a problem because spinlocks can't have an > indirect dependency upon the page allocator and hence close the loop > with the oom reaper. > > Suggested by Michal Hocko. > > v2: > - Improve commit message (Michal) > - Also check

Re: [PATCH] mm: Don't let userspace spam allocations warnings

2019-02-20 Thread Michal Hocko
the OOM killer report will be printed regardless of __GFP_NOWARN. > It also means more filtering for our CI, because our testsuite > exercises these corner cases and so hits these a lot. > > Signed-off-by: Daniel Vetter > Cc: Andrew Morton > Cc: Mike Rapoport > Cc: Mich

Re: [PATCH 2/4] kernel.h: Add non_block_start/end()

2018-12-10 Thread Michal Hocko
On Mon 10-12-18 16:22:53, Peter Zijlstra wrote: > On Mon, Dec 10, 2018 at 04:01:59PM +0100, Michal Hocko wrote: > > On Mon 10-12-18 15:47:11, Peter Zijlstra wrote: > > > On Mon, Dec 10, 2018 at 03:13:37PM +0100, Michal Hocko wrote: > > > > I do not see any sc

Re: [PATCH 2/4] kernel.h: Add non_block_start/end()

2018-12-10 Thread Michal Hocko
On Mon 10-12-18 15:47:11, Peter Zijlstra wrote: > On Mon, Dec 10, 2018 at 03:13:37PM +0100, Michal Hocko wrote: > > I do not see any scheduler guys Cced and it would be really great to get > > their opinion here. > > > > On Mon 10-12-18 11:36:39, Daniel Vetter wrote:

Re: [PATCH 2/4] kernel.h: Add non_block_start/end()

2018-12-10 Thread Michal Hocko
, and that really has a different semantic, I think this makes some sense. The cotext is preemptible but we do not want notifier to sleep on any locks, WQ etc. > Suggested by Michal Hocko. > > Cc: Andrew Morton > Cc: Michal Hocko > Cc: David Rientjes > Cc: "Christian König

Re: [PATCH 1/4] mm: Check if mmu notifier callbacks are allowed to fail

2018-12-10 Thread Michal Hocko
going to help with testing then I do not have any objections of course. > v2: Drop the full WARN_ON backtrace in favour of just a pr_warn for > the problematic case (Michal Hocko). Thanks! > Cc: Andrew Morton > Cc: Michal Hocko > Cc: "Christian König" > Cc: David Rientjes

Re: [PATCH 1/3] mm: Check if mmu notifier callbacks are allowed to fail

2018-11-23 Thread Michal Hocko
On Fri 23-11-18 14:15:11, Daniel Vetter wrote: > On Fri, Nov 23, 2018 at 1:43 PM Michal Hocko wrote: > > On Fri 23-11-18 13:30:57, Daniel Vetter wrote: > > > On Fri, Nov 23, 2018 at 12:15:57PM +0100, Michal Hocko wrote: > > > > On Thu 22-11-18 17:51:04, Daniel Vett

Re: [PATCH 2/3] mm, notifier: Catch sleeping/blocking for !blockable

2018-11-23 Thread Michal Hocko
On Fri 23-11-18 13:38:38, Daniel Vetter wrote: > On Fri, Nov 23, 2018 at 12:12:37PM +0100, Michal Hocko wrote: > > On Thu 22-11-18 17:51:05, Daniel Vetter wrote: > > > We need to make sure implementations don't cheat and don't have a > > > possible schedule/blocking

Re: [PATCH 1/3] mm: Check if mmu notifier callbacks are allowed to fail

2018-11-23 Thread Michal Hocko
On Fri 23-11-18 13:30:57, Daniel Vetter wrote: > On Fri, Nov 23, 2018 at 12:15:57PM +0100, Michal Hocko wrote: > > On Thu 22-11-18 17:51:04, Daniel Vetter wrote: > > > Just a bit of paranoia, since if we start pushing this deep into > > > callchains it's hard to s

Re: [PATCH 1/3] mm: Check if mmu notifier callbacks are allowed to fail

2018-11-23 Thread Michal Hocko
Is really backtrace that interesting? > Cc: Andrew Morton > Cc: Michal Hocko > Cc: "Christian König" > Cc: David Rientjes > Cc: Daniel Vetter > Cc: "Jérôme Glisse" > Cc: linux...@kvack.org > Cc: Paolo Bonzini > Signed-off-by: Daniel Vetter

Re: [Intel-gfx] [PATCH 1/3] mm: Check if mmu notifier callbacks are allowed to fail

2018-11-23 Thread Michal Hocko
of memory. And where the process is > gone already, so semantics of what exactly happens don't matter that much > anymore. Yes this was indeed the case. There is still the exit path which would do the rest of the work so we are not leaving anything behind. -- Michal Hocko SUSE Labs __

Re: [PATCH 2/3] mm, notifier: Catch sleeping/blocking for !blockable

2018-11-23 Thread Michal Hocko
callsites trigger, and it's a bit ugly in the code flow. > But it gets the job done. Yeah, it is quite ugly. Especially because it makes DEBUG config bahavior much different. So is this really worth it? Has this already discovered any existing bug? > Cc: Andrew Morton > Cc: Michal Hocko

Re: [PATCH v8 1/7] mm, devm_memremap_pages: Mark devm_memremap_pages() EXPORT_SYMBOL_GPL

2018-11-23 Thread Michal Hocko
On Thu 22-11-18 17:38:58, Christoph Hellwig wrote: > On Thu, Nov 22, 2018 at 02:30:13PM +0100, Michal Hocko wrote: > > Whoever needs a wrapper around arch_add_memory can do so because this > > symbol has no restriction for the usage. > > arch_add_memory is not exported,

Re: [PATCH v8 1/7] mm, devm_memremap_pages: Mark devm_memremap_pages() EXPORT_SYMBOL_GPL

2018-11-22 Thread Michal Hocko
other areas in the past. And out-of-tree modules? Who cares. Those are on their own completely and have their ways to go around. > Cc: Michal Hocko > Cc: "Jérôme Glisse" > Reviewed-by: Christoph Hellwig > Signed-off-by: Dan Williams That being said Acked-by: Michal Hocko

Re: [PATCH] mm: convert totalram_pages, totalhigh_pages and managed_pages to atomic.

2018-10-23 Thread Michal Hocko
s hard to review manually. -- Michal Hocko SUSE Labs ___ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
about? From f7ac75277d526dccd011f343818dc6af627af2af Mon Sep 17 00:00:00 2001 From: Michal Hocko Date: Fri, 24 Aug 2018 15:32:24 +0200 Subject: [PATCH] mm, mmu_notifier: be explicit about range invalition non-blocking mode If invalidate_range_start is called for !blocking mode then all callbacks have

Re: [PATCH] mm, oom: distinguish blockable mode for mmu notifiers

2018-08-24 Thread Michal Hocko
On Fri 24-08-18 23:52:25, Tetsuo Handa wrote: > On 2018/08/24 22:32, Michal Hocko wrote: > > On Fri 24-08-18 22:02:23, Tetsuo Handa wrote: > >> I worry that (currently > >> out-of-tree) users of this API are involving work / recursion. > > > > I do not giv

  1   2   3   >