Re: [PATCH RFC v3 4/9] mm: Export alloc_contig_range() / free_contig_range()

On 16.10.19 13:20, Michal Hocko wrote: On Thu 19-09-19 16:22:23, David Hildenbrand wrote: A virtio-mem device wants to allocate memory from the memory region it manages in order to unplug it in the hypervisor - similar to a balloon driver. Also, it might want to plug previously unplugged

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

On 16.10.19 13:43, Michal Hocko wrote: On Thu 19-09-19 16:22:25, David Hildenbrand wrote: virtio-mem wants to allow to offline memory blocks of which some parts were unplugged, especially, to later offline and remove completely unplugged memory blocks. The important part is that PageOffline

Re: [PATCH RFC v3 8/9] mm/memory_hotplug: Introduce offline_and_remove_memory()

On 16.10.19 13:47, Michal Hocko wrote: On Thu 19-09-19 16:22:27, David Hildenbrand wrote: virtio-mem wants to offline and remove a memory block once it unplugged all subblocks (e.g., using alloc_contig_range()). Let's provide an interface to do that from a driver. virtio-mem already suppor

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

On 16.10.19 13:43, Michal Hocko wrote: On Thu 19-09-19 16:22:25, David Hildenbrand wrote: virtio-mem wants to allow to offline memory blocks of which some parts were unplugged, especially, to later offline and remove completely unplugged memory blocks. The important part is that PageOffline

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

On 16.10.19 15:45, Michal Hocko wrote: On Wed 16-10-19 14:50:30, David Hildenbrand wrote: On 16.10.19 13:43, Michal Hocko wrote: On Thu 19-09-19 16:22:25, David Hildenbrand wrote: virtio-mem wants to allow to offline memory blocks of which some parts were unplugged, especially, to later

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

On 16.10.19 15:45, Michal Hocko wrote: On Wed 16-10-19 14:50:30, David Hildenbrand wrote: On 16.10.19 13:43, Michal Hocko wrote: On Thu 19-09-19 16:22:25, David Hildenbrand wrote: virtio-mem wants to allow to offline memory blocks of which some parts were unplugged, especially, to later

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

On 16.10.19 16:03, Michal Hocko wrote: On Wed 16-10-19 15:45:06, David Hildenbrand wrote: On 16.10.19 13:43, Michal Hocko wrote: On Thu 19-09-19 16:22:25, David Hildenbrand wrote: virtio-mem wants to allow to offline memory blocks of which some parts were unplugged, especially, to later

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

On 16.10.19 16:09, Michal Hocko wrote: On Wed 16-10-19 15:55:00, David Hildenbrand wrote: On 16.10.19 15:45, Michal Hocko wrote: [...] There is state stored in the struct page. In other words this shouldn't be really different from HWPoison pages. I cannot find the code that is doing tha

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

On 18.10.19 10:15, Michal Hocko wrote: On Wed 16-10-19 16:14:52, David Hildenbrand wrote: On 16.10.19 16:03, Michal Hocko wrote: [...] But why cannot you keep the reference count at 1 (do get_page when offlining the page)? In other words as long as the driver knows the page has been returned

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

On 18.10.19 13:20, Michal Hocko wrote: On Fri 18-10-19 10:50:24, David Hildenbrand wrote: On 18.10.19 10:15, Michal Hocko wrote: On Wed 16-10-19 16:14:52, David Hildenbrand wrote: On 16.10.19 16:03, Michal Hocko wrote: [...] But why cannot you keep the reference count at 1 (do get_page when

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

>> Please note that we have other users that use PG_offline + refcount >= 1 >> (HyperV balloon, XEN). We should not affect these users (IOW, >> has_unmovable_pages() has to stop right there if we see one of these pages). > > OK, this is exactly what I was worried about. I can see why you might > w

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

On 23.10.19 11:43, Michal Hocko wrote: On Tue 22-10-19 16:02:09, David Hildenbrand wrote: [...] MEM_CANCEL_OFFLINE could gain the reference back to balance the MEM_GOING_OFFLINE step. The pages are already unisolated and could be used by the buddy. But again, I think you have an idea that

Re: [PATCH RFC v3 6/9] mm: Allow to offline PageOffline() pages with a reference count of 0

On 24.10.19 10:42, Michal Hocko wrote: On Wed 23-10-19 12:03:51, David Hildenbrand wrote: Do you see any downsides? The only downside I see is that we get more false negatives on has_unmovable_pages(), eventually resulting in the offlining stage after isolation to loop forever (as some

[PATCH RFC] mm: Allow to offline unmovable PageOffline() pages if the driver agrees

a Cc: Johannes Weiner Cc: Anthony Yznaga Cc: Michal Hocko Cc: Oscar Salvador Cc: Mel Gorman Cc: Mike Rapoport Cc: Dan Williams Cc: Anshuman Khandual Cc: Qian Cai Cc: Pingfan Liu Signed-off-by: David Hildenbrand --- Michal, this is the approach where we allow has_unmovable_pages() to su

[PATCH] virtio-balloon: fix managed page counts when migrating pages between zones

ges") Cc: # v3.11+ Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Jiang Liu Cc: Andrew Morton Cc: virtualization@lists.linux-foundation.org Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_balloon.c | 6 ++ 1 file changed, 6 insertions(+) diff --git a/drivers/virtio/vir

Re: [PATCH] virtio-balloon: fix managed page counts when migrating pages between zones

On 04.12.19 21:48, David Hildenbrand wrote: > In case we have to migrate a ballon page to a newpage of another zone, the > managed page count of both zones is wrong. Paired with memory offlining > (which will adjust the managed page count), we can trigger kernel crashes > and

[PATCH v2] virtio-balloon: fix managed page counts when migrating pages between zones

Cc: Jiang Liu Cc: Andrew Morton Cc: virtualization@lists.linux-foundation.org Signed-off-by: David Hildenbrand --- v1 -> v2: - Adjust count before enquing newpage (and it possibly gets free form the balloon) - Check if the zone changed --- drivers/virtio/virtio_balloon.c | 7 +++ 1

Re: [PATCH] virtio-balloon: fix managed page counts when migrating pages between zones

On 05.12.19 11:08, Igor Mammedov wrote: > On Wed, 4 Dec 2019 21:48:07 +0100 > David Hildenbrand wrote: > >> In case we have to migrate a ballon page to a newpage of another zone, the >> managed page count of both zones is wrong. Paired with memory offlining >> (wh

Re: [PATCH] virtio-balloon: fix managed page counts when migrating pages between zones

On 10.12.19 14:31, Michael S. Tsirkin wrote: > On Wed, Dec 04, 2019 at 09:48:07PM +0100, David Hildenbrand wrote: >> In case we have to migrate a ballon page to a newpage of another zone, the >> managed page count of both zones is wrong. Paired with memory offlining >> (

Re: [PATCH v2] virtio-balloon: fix managed page counts when migrating pages between zones

On 05.12.19 10:24, David Hildenbrand wrote: > In case we have to migrate a ballon page to a newpage of another zone, the > managed page count of both zones is wrong. Paired with memory offlining > (which will adjust the managed page count), we can trigger kernel crashes > and

Re: [PATCH v2] virtio-balloon: fix managed page counts when migrating pages between zones

On 10.12.19 16:24, Michael S. Tsirkin wrote: > On Tue, Dec 10, 2019 at 02:44:38PM +0100, David Hildenbrand wrote: >> On 05.12.19 10:24, David Hildenbrand wrote: >>> In case we have to migrate a ballon page to a newpage of another zone, the >>> managed page count of both

[PATCH v3] virtio-balloon: fix managed page counts when migrating pages between zones

: Andrew Morton Cc: Igor Mammedov Cc: virtualization@lists.linux-foundation.org Signed-off-by: David Hildenbrand --- v2 -> v3: - Refine comment - s/only/online/ in description - Clarify why VIRTIO_BALLOON_F_DEFLATE_ON_OOM has to be checked v1 -> v2: - Adjust count before enquing newpage (and it possi

[PATCH RFC v4 01/13] ACPI: NUMA: export pxm_to_node

Will be needed by virtio-mem to identify the node from a pxm. Cc: "Rafael J. Wysocki" Cc: Len Brown Cc: linux-a...@vger.kernel.org Signed-off-by: David Hildenbrand --- drivers/acpi/numa/srat.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/acpi/numa/srat.c b/drivers

[PATCH RFC v4 03/13] virtio-mem: Paravirtualized memory hotunplug part 1

Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 109 +++- 1 file changed, 107 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 8f

[PATCH RFC v4 00/13] virtio-mem: paravirtualized memory

VM be? A: virtio-mem memory will not go to the DMA zones. So to avoid running out of DMA memory, I suggest something like 2-3GB on x86-64. But many VMs can most probably deal with less DMA memory - depends on the use case. [1] https://events.linuxfoundation.org/wp-content/uploads/2017/12/

[PATCH RFC v4 06/13] mm: Allow to offline unmovable PageOffline() pages via MEM_GOING_OFFLINE

n Cc: Alexander Duyck Cc: Vlastimil Babka Cc: Johannes Weiner Cc: Anthony Yznaga Cc: Michal Hocko Cc: Oscar Salvador Cc: Mel Gorman Cc: Mike Rapoport Cc: Dan Williams Cc: Anshuman Khandual Cc: Qian Cai Cc: Pingfan Liu Signed-off-by: David Hildenbrand --- include/linux/page-flags.h

[PATCH RFC v4 07/13] virtio-mem: Allow to offline partially unplugged memory blocks

ichal Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 64 - 1 file changed, 63 insertions(+), 1 de

[PATCH RFC v4 02/13] virtio-mem: Paravirtualized memory hotplug

l S. Tsirkin" Cc: Jason Wang Cc: Oscar Salvador Cc: Michal Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Cc: "Rafael J. Wysocki" Cc: Len Brown Cc: linux-a...@vger.kernel.org Signed

[PATCH RFC v4 04/13] mm: Export alloc_contig_range() / free_contig_range()

o Cc: Vlastimil Babka Cc: Oscar Salvador Cc: Mel Gorman Cc: Mike Rapoport Cc: Dan Williams Cc: Alexander Duyck Cc: Pavel Tatashin Cc: Alexander Potapenko Acked-by: Michal Hocko # to export contig range allocator API Signed-off-by: David Hildenbrand --- mm/page_alloc.c | 2 ++ 1 file chang

[PATCH RFC v4 09/13] virtio-mem: Offline and remove completely unplugged memory blocks

nother notifier NACKs the offlining request). Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Oscar Salvador Cc: Michal Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenb

[PATCH RFC v4 05/13] virtio-mem: Paravirtualized memory hotunplug part 2

Cc: Michal Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/Kconfig | 1 + drivers/virtio/virtio_mem.c | 156

[PATCH RFC v4 10/13] virtio-mem: Better retry handling

Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virt

[PATCH RFC v4 11/13] mm/vmscan: Move count_vm_event(DROP_SLAB) into drop_slab()

Let's count within the function itself, so every invocation (of future users) will be counted. Cc: Alexander Viro Cc: Andrew Morton Cc: linux-fsde...@vger.kernel.org Signed-off-by: David Hildenbrand --- fs/drop_caches.c | 4 +--- mm/vmscan.c | 1 + 2 files changed, 2 insertions(

[PATCH RFC v4 13/13] virtio-mem: Drop slab objects when unplug continues to fail

Young Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Cc: linux-fsde...@vger.kernel.org Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 30 ++ 1 file changed, 30 insertions(+) diff --git a/drivers/virtio/virtio_mem.c b

[PATCH RFC v4 12/13] mm/vmscan: Export drop_slab() and drop_slab_node()

d-off-by: David Hildenbrand --- include/linux/mm.h | 4 ++-- mm/vmscan.c| 2 ++ 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 64799c5cb39f..483300f58be8 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2706,8 +2706,

[PATCH RFC v4 08/13] mm/memory_hotplug: Introduce offline_and_remove_memory()

onlined by user space again before it gets removed. To keep things simple, allow to only work on a single memory block. Cc: Andrew Morton Cc: David Hildenbrand Cc: Oscar Salvador Cc: Michal Hocko Cc: Pavel Tatashin Cc: Wei Yang Cc: Dan Williams Cc: Qian Cai Signed-off-by: David Hildenbran

Re: [PATCH RFC v4 01/13] ACPI: NUMA: export pxm_to_node

On 12.12.19 22:43, Rafael J. Wysocki wrote: > On Thursday, December 12, 2019 6:11:25 PM CET David Hildenbrand wrote: >> Will be needed by virtio-mem to identify the node from a pxm. >> >> Cc: "Rafael J. Wysocki" >> Cc: Len Brown >> Cc: linux-a

Re: [PATCH RFC v4 00/13] virtio-mem: paravirtualized memory

On 13.12.19 21:15, Konrad Rzeszutek Wilk wrote: > On Thu, Dec 12, 2019 at 06:11:24PM +0100, David Hildenbrand wrote: >> This series is based on latest linux-next. The patches are located at: >> https://github.com/davidhildenbrand/linux.git virtio-mem-rfc-v4 > Heya! Hi K

Re: [PATCH RFC v4 00/13] virtio-mem: paravirtualized memory

> Am 24.12.2019 um 08:04 schrieb teawater : > > Hi David, > > Thanks for your work. > > I Got following build fail if X86_64_ACPI_NUMA is n with rfc3 and rfc4: > make -j8 bzImage > GEN Makefile > DESCEND objtool > CALL/home/teawater/kernel/linux-upstream3/scripts/atomic/check-atom

Re: [PATCH RFC v4 00/13] virtio-mem: paravirtualized memory

On 12.12.19 18:11, David Hildenbrand wrote: > This series is based on latest linux-next. The patches are located at: > https://github.com/davidhildenbrand/linux.git virtio-mem-rfc-v4 > > The basic idea of virtio-mem is to provide a flexible, > cross-architecture memory hot(u

Re: [PATCH RFC v4 00/13] virtio-mem: paravirtualized memory

On 09.01.20 14:48, David Hildenbrand wrote: > On 12.12.19 18:11, David Hildenbrand wrote: >> This series is based on latest linux-next. The patches are located at: >> https://github.com/davidhildenbrand/linux.git virtio-mem-rfc-v4 >> >> The basic idea of virtio-m

Re: Balloon pressuring page cache

On 29.01.20 01:22, Tyler Sanderson via Virtualization wrote: > A primary advantage of virtio balloon over other memory reclaim > mechanisms is that it can pressure the guest's page cache into shrinking. > > However, since the balloon driver changed to using the shrinker API >

Re: Balloon pressuring page cache

On 29.01.20 20:11, Tyler Sanderson wrote: > > > On Wed, Jan 29, 2020 at 2:31 AM David Hildenbrand <mailto:da...@redhat.com>> wrote: > > On 29.01.20 01:22, Tyler Sanderson via Virtualization wrote: > > A primary advantage of virtio balloon over other memory

Re: Balloon pressuring page cache

On 30.01.20 16:20, Michael S. Tsirkin wrote: > On Thu, Jan 30, 2020 at 04:02:34PM +0100, David Hildenbrand wrote: >> On 29.01.20 20:11, Tyler Sanderson wrote: >>> >>> >>> On Wed, Jan 29, 2020 at 2:31 AM David Hildenbrand >> <mailto:da...@redhat.com&

Re: Balloon pressuring page cache

On Thursday, January 30, 2020 11:03 PM, David Hildenbrand wrote: >>> > On 29.01.20 20:11, Tyler Sanderson wrote: >>> > > >>> > > >>> > > On Wed, Jan 29, 2020 at 2:31 AM David Hildenbrand >> > > <mai

Re: Balloon pressuring page cache

On 03.02.20 21:32, Tyler Sanderson wrote: > There were apparently good reasons for moving away from OOM notifier > callback: > https://lkml.org/lkml/2018/7/12/314 > https://lkml.org/lkml/2018/8/2/322 > > In particular the OOM notifier is worse than the shrinker because: The issue is that DEFLATE_

Re: Balloon pressuring page cache

>>> I would say reverting probably makes sense. I'm not sure there is much >>> value to having a shrinker running deflation when you are actively trying >>> to increase the balloon. It would make more sense to wait until you are >>> actually about to start hitting oom. >> >> I think the shrinker ma

Re: Balloon pressuring page cache

On 04.02.20 09:40, Michael S. Tsirkin wrote: > On Tue, Feb 04, 2020 at 09:35:21AM +0100, David Hildenbrand wrote: >>>>> I would say reverting probably makes sense. I'm not sure there is much >>>>> value to having a shrinker running deflation when you are

Re: Balloon pressuring page cache

On 04.02.20 09:40, Michael S. Tsirkin wrote: > On Tue, Feb 04, 2020 at 09:35:21AM +0100, David Hildenbrand wrote: >>>>> I would say reverting probably makes sense. I'm not sure there is much >>>>> value to having a shrinker running deflation when you are

Re: Balloon pressuring page cache

[...] >> >> Issue 2: When called via the shrinker, (but also to fix Issue 1), it could be >> that we do have VIRTIO_BALLOON_F_MUST_TELL_HOST. I assume this means >> (-ENOCLUE) that we have to wait until the hypervisor notifies us via the >> STOP? Or >> for which event do we have to wait? Because

Re: Balloon pressuring page cache

On 04.02.20 19:52, Tyler Sanderson wrote: > > > On Tue, Feb 4, 2020 at 12:29 AM David Hildenbrand <mailto:da...@redhat.com>> wrote: > > On 03.02.20 21:32, Tyler Sanderson wrote: > > There were apparently good reasons for moving away from OOM notifier &

Re: Balloon pressuring page cache

>> Issue 2: When called via the shrinker, (but also to fix Issue 1), it could >> be that >> we do have VIRTIO_BALLOON_F_MUST_TELL_HOST. > > I don't think it is an issue here. > MUST_TELL_HOST is for the ballooning pages, where pages are offered to host > to _USE_. > For free page hint, as the n

Re: Balloon pressuring page cache

On 05.02.20 08:35, Nadav Amit wrote: >> On Feb 3, 2020, at 2:50 PM, Nadav Amit wrote: >> >>> On Feb 3, 2020, at 8:34 AM, David Hildenbrand wrote: >>> >>> On 03.02.20 17:18, Alexander Duyck wrote: >>>> On Mon, 2020-02-03 at 08:11 -0500, Michael

Re: Balloon pressuring page cache

On 04.02.20 21:33, Michael S. Tsirkin wrote: > On Tue, Feb 04, 2020 at 05:56:22PM +0100, David Hildenbrand wrote: >> [...] >> >>>> >>>> Issue 2: When called via the shrinker, (but also to fix Issue 1), it could >>>> be >>>> that

Re: Balloon pressuring page cache

On 05.02.20 09:54, Wang, Wei W wrote: > On Wednesday, February 5, 2020 4:19 PM, David Hildenbrand wrote: >> Yes, I agree with you. Yet, I am thinking about one >> (unlikely?impossible?) scenario. Can you refresh my brain why that cannot >> happen (IOW, why we don't ha

Re: Balloon pressuring page cache

On 05.02.20 10:00, Wang, Wei W wrote: > On Wednesday, February 5, 2020 4:57 PM, David Hildenbrand wrote: >>>> Yes, I agree with you. Yet, I am thinking about one >>>> (unlikely?impossible?) scenario. Can you refresh my brain why that >>>> cannot happen (IOW,

Re: Balloon pressuring page cache

>> 1. Guest allocates a page and sends it to the host. >> 2. Shrinker gets active and releases that page again. >> 3. Some user in the guest allocates and modifies that page. The dirty bit is >> set in the hypervisor. > > The bit will be set in KVM's bitmap, and will be synced to QEMU's bitmap whe

Re: Balloon pressuring page cache

On 05.02.20 10:35, Wang, Wei W wrote: > On Wednesday, February 5, 2020 5:23 PM, David Hildenbrand wrote: >> So, if you run a TCG guest and use it with free page reporting, the race is >> possible? So the correctness depends on two dirty bitmaps in the hypervisor >> and how th

Re: Balloon pressuring page cache

On 05.02.20 10:49, Wang, Wei W wrote: > On Wednesday, February 5, 2020 5:37 PM, David Hildenbrand wrote: >>> >>> Not sure how TCG tracks the dirty bits. But In whatever >>> implementation, the hypervisor should have >> >> There is only a single bitmap

Re: Balloon pressuring page cache

On 05.02.20 11:25, Michael S. Tsirkin wrote: > On Wed, Feb 05, 2020 at 10:58:14AM +0100, David Hildenbrand wrote: >> On 05.02.20 10:49, Wang, Wei W wrote: >>> On Wednesday, February 5, 2020 5:37 PM, David Hildenbrand wrote: >>>>> >>>>> Not sur

Re: Balloon pressuring page cache

On 05.02.20 11:27, Michael S. Tsirkin wrote: > On Wed, Feb 05, 2020 at 09:19:58AM +0100, David Hildenbrand wrote: >> On 05.02.20 08:35, Nadav Amit wrote: >>>> On Feb 3, 2020, at 2:50 PM, Nadav Amit wrote: >>>> >>>>> On Feb 3, 2020, at 8:34 AM, Davi

[PATCH v1 1/3] virtio-balloon: Fix memory leak when unloading while hinting is in progress

ng Li Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_balloon.c | 4 1 file changed, 4 insertions(+) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c index 8e400ece9273..abef2306c899 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio

[PATCH v1 2/3] virtio_balloon: Fix memory leaks on errors in virtballoon_probe()

We forget to put the inode and unmount the kernfs used for compaction. Fixes: 71994620bb25 ("virtio_balloon: replace oom notifier with shrinker") Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Wei Wang Cc: Liang Li Signed-off-by: David Hildenbrand --- drivers/virtio/v

[PATCH v1 3/3] virtio-balloon: Switch back to OOM handler for VIRTIO_BALLOON_F_DEFLATE_ON_OOM

pdating the balloon. If the communication with the host ever becomes a problem on this call path. [1] https://www.spinics.net/lists/linux-virtualization/msg40863.html Reported-by: Tyler Sanderson Cc: Michael S. Tsirkin Cc: Wei Wang Cc: Alexander Duyck Cc: David Rientjes Cc: Nadav Amit

[PATCH v1 0/3] virtio-balloon: Fixes + switch back to OOM handler

n/msg40863.html David Hildenbrand (3): virtio-balloon: Fix memory leak when unloading while hinting is in progress virtio_balloon: Fix memory leaks on errors in virtballoon_probe() virtio-balloon: Switch back to OOM handler for VIRTIO_BALLOON_F_DEFLATE_ON_OOM drivers/virtio/virtio_

Re: [PATCH v1 3/3] virtio-balloon: Switch back to OOM handler for VIRTIO_BALLOON_F_DEFLATE_ON_OOM

> Am 05.02.2020 um 23:37 schrieb Tyler Sanderson : > >  > > >> On Wed, Feb 5, 2020 at 8:34 AM David Hildenbrand wrote: >> Commit 71994620bb25 ("virtio_balloon: replace oom notifier with shrinker") >> changed the behavior when deflation happens

Re: [PATCH] virtio_balloon: prevent pfn array overflow

page. >* Note that the first pfn points at start of the page. > Reviewed-by: David Hildenbrand -- Thanks, David / dhildenb ___ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v1 3/3] virtio-balloon: Switch back to OOM handler for VIRTIO_BALLOON_F_DEFLATE_ON_OOM

On 06.02.20 08:40, Michael S. Tsirkin wrote: > On Wed, Feb 05, 2020 at 05:34:02PM +0100, David Hildenbrand wrote: >> Commit 71994620bb25 ("virtio_balloon: replace oom notifier with shrinker") >> changed the behavior when deflation happens automatically. Instead of >&g

Re: [PATCH v1 3/3] virtio-balloon: Switch back to OOM handler for VIRTIO_BALLOON_F_DEFLATE_ON_OOM

>> commit bf50e69f63d21091e525185c3ae761412be0ba72 >> Author: Dave Hansen >> Date: Thu Apr 7 10:43:25 2011 -0700 >> >> virtio balloon: kill tell-host-first logic >> >> The virtio balloon driver has a VIRTIO_BALLOON_F_MUST_TELL_HOST >> feature bit. Whenever the bit is set, the guest

Re: [PATCH RFC] virtio_balloon: conservative balloon page shrinking

On 06.02.20 09:01, Wei Wang wrote: > There are cases that users want to shrink balloon pages after the > pagecache depleted. The conservative_shrinker lets the shrinker > shrink balloon pages when all the pagecache has been reclaimed. > > Signed-off-by: Wei Wang > --- > drivers/virtio/virtio_bal

Re: [PATCH v1 3/3] virtio-balloon: Switch back to OOM handler for VIRTIO_BALLOON_F_DEFLATE_ON_OOM

On 06.02.20 10:12, Michael S. Tsirkin wrote: > On Wed, Feb 05, 2020 at 05:34:02PM +0100, David Hildenbrand wrote: >> Commit 71994620bb25 ("virtio_balloon: replace oom notifier with shrinker") >> changed the behavior when deflation happens automatically. Instead of >&g

Re: [PATCH RFC] virtio_balloon: conservative balloon page shrinking

On 06.02.20 10:28, Wang, Wei W wrote: > On Thursday, February 6, 2020 5:10 PM, David Hildenbrand wrote: >> so dropping caches (echo 3 > /proc/sys/vm/drop_caches) will no longer >> deflate the balloon when conservative_shrinker=true? >> > > Should be. Need Tyler'

Re: [PATCH RFC] virtio_balloon: conservative balloon page shrinking

On 06.02.20 10:44, Wang, Wei W wrote: > On Thursday, February 6, 2020 5:32 PM, David Hildenbrand wrote: >> >> If the page cache is empty, a drop_slab() will deflate the whole balloon if I >> am not wrong. >> >> Especially, a echo 3 > /proc/sys/vm/drop_caches

Re: Balloon pressuring page cache

>>> It's not about dangers as such. It's just that when linux hits OOM >>> all kind of error paths are being hit, latent bugs start triggering, >>> latency goes up drastically. >> Doesn't this suggest that the shrinker is preferable to the OOM >> notifier in the case that we're

Re: [PATCH] virtio_balloon: Fix unused label warning

t; 963 | out_del_vqs: > | ^~ > > The CONFIG_BALLOON_COMPACTION ifdeffery should enclose it too. > > Signed-off-by: Borislav Petkov > Cc: David Hildenbrand > --- > drivers/virtio/virtio_balloon.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff

Re: [PATCH v1 3/3] virtio-balloon: Switch back to OOM handler for VIRTIO_BALLOON_F_DEFLATE_ON_OOM

On 05.02.20 17:34, David Hildenbrand wrote: > Commit 71994620bb25 ("virtio_balloon: replace oom notifier with shrinker") > changed the behavior when deflation happens automatically. Instead of > deflating when called by the OOM handler, the shrinker is used. > > However,

Re: [PATCH v1 3/3] virtio-balloon: Switch back to OOM handler for VIRTIO_BALLOON_F_DEFLATE_ON_OOM

>> There was a report that this results in undesired side effects when >> inflating the balloon to shrink the page cache. [1] >> "When inflating the balloon against page cache (i.e. no free memory >> remains) vmscan.c will both shrink page cache, but also invoke the >> shrinkers --

Re: [PATCH v1 3/3] virtio-balloon: Switch back to OOM handler for VIRTIO_BALLOON_F_DEFLATE_ON_OOM

> Am 14.02.2020 um 21:49 schrieb Tyler Sanderson : > >  > Regarding Wei's patch that modifies the shrinker implementation, versus this > patch which reverts to OOM notifier: > I am in favor of both patches. But I do want to make sure a fix gets back > ported to 4.19 where the performance regr

Re: [PATCH] virtio_balloon: Adjust label in virtballoon_probe

> Am 16.02.2020 um 01:41 schrieb Nathan Chancellor : > > Clang warns when CONFIG_BALLOON_COMPACTION is unset: > > ../drivers/virtio/virtio_balloon.c:963:1: warning: unused label > 'out_del_vqs' [-Wunused-label] > out_del_vqs: > ^~~~ > 1 warning generated. > Thanks, there is already „

Re: [PATCH] virtio_balloon: Fix unused label warning

virtio/virtio_balloon.c:963:1: warning: label ‘out_del_vqs’ defined >> but not used [-Wunused-label] >>963 | out_del_vqs: >>| ^~ >> >> The CONFIG_BALLOON_COMPACTION ifdeffery should enclose it too. >> >> Signed-off-by: Borislav Petkov >>

Re: [PATCH] virtio_balloon: Adjust label in virtballoon_probe

> Am 16.02.2020 um 08:47 schrieb Nathan Chancellor : > > On Sun, Feb 16, 2020 at 08:36:45AM +0100, David Hildenbrand wrote: >> >> >>>> Am 16.02.2020 um 01:41 schrieb Nathan Chancellor >>>> : >>> >>> Clang warns when

Re: [PATCH] virtio_balloon: Adjust label in virtballoon_probe

tio_balloon.c > +++ b/drivers/virtio/virtio_balloon.c > @@ -959,8 +959,8 @@ static int virtballoon_probe(struct virtio_device *vdev) > iput(vb->vb_dev_info.inode); > out_kern_unmount: > kern_unmount(balloon_mnt); > -#endif > out_del_vqs: > +#endif >

Re: [PATCH] virtio_balloon: Fix build error seen with CONFIG_BALLOON_COMPACTION=n

rror=unused-label] > > This is seen with CONFIG_BALLOON_COMPACTION=n. > > Reported-by: kbuild test robot > Fixes: 1ad6f58ea936 ("virtio_balloon: Fix memory leaks on errors in > virtballoon_probe()") > Cc: David Hildenbrand > Cc: Michael S. Tsirkin > Signed-off-by: Guente

Re: [PATCH RFC v4 00/13] virtio-mem: paravirtualized memory

On 29.01.20 10:41, David Hildenbrand wrote: > On 09.01.20 14:48, David Hildenbrand wrote: >> On 12.12.19 18:11, David Hildenbrand wrote: >>> This series is based on latest linux-next. The patches are located at: >>> https://github.com/davidhildenbrand/linux.git vi

Re: [PATCH RFC v4 08/13] mm/memory_hotplug: Introduce offline_and_remove_memory()

On 25.02.20 15:11, Michal Hocko wrote: > On Thu 12-12-19 18:11:32, David Hildenbrand wrote: >> virtio-mem wants to offline and remove a memory block once it unplugged >> all subblocks (e.g., using alloc_contig_range()). Let's provide >> an interface to do that from a

Re: [PATCH RFC v4 12/13] mm/vmscan: Export drop_slab() and drop_slab_node()

On 25.02.20 15:58, Michal Hocko wrote: > On Thu 12-12-19 18:11:36, David Hildenbrand wrote: >> We already have a way to trigger reclaiming of all reclaimable slab objects >> from user space (echo 2 > /proc/sys/vm/drop_caches). Let's allow drivers >> to also trigger

Re: [PATCH RFC v4 12/13] mm/vmscan: Export drop_slab() and drop_slab_node()

On 25.02.20 18:06, Michal Hocko wrote: > On Tue 25-02-20 16:09:29, David Hildenbrand wrote: >> On 25.02.20 15:58, Michal Hocko wrote: >>> On Thu 12-12-19 18:11:36, David Hildenbrand wrote: >>>> We already have a way to trigger reclaiming of all reclaimable slab object

Re: [PATCH RFC v4 06/13] mm: Allow to offline unmovable PageOffline() pages via MEM_GOING_OFFLINE

>> /* >> * Scan pfn range [start,end) to find movable/migratable pages (LRU pages, >> - * non-lru movable pages and hugepages). We scan pfn because it's much >> - * easier than scanning over linked list. This function returns the pfn >> - * of the first found movable page if it's found, otherwis

Re: [PATCH RFC v4 06/13] mm: Allow to offline unmovable PageOffline() pages via MEM_GOING_OFFLINE

On 25.02.20 22:46, Alexander Duyck wrote: > On Tue, 2020-02-25 at 19:49 +0100, David Hildenbrand wrote: >>>> /* >>>> * Scan pfn range [start,end) to find movable/migratable pages (LRU pages, >>>> - * non-lru movable pages and hugepages). We scan pfn

Re: [PATCH RFC v4 08/13] mm/memory_hotplug: Introduce offline_and_remove_memory()

On 02.03.20 13:48, Michal Hocko wrote: > On Tue 25-02-20 15:27:28, David Hildenbrand wrote: >> On 25.02.20 15:11, Michal Hocko wrote: >>> On Thu 12-12-19 18:11:32, David Hildenbrand wrote: >>>> virtio-mem wants to offline and remove a memory block once it unplugged &

[PATCH v1 01/11] ACPI: NUMA: export pxm_to_node

Will be needed by virtio-mem to identify the node from a pxm. Acked-by: "Rafael J. Wysocki" Cc: Len Brown Cc: linux-a...@vger.kernel.org Signed-off-by: David Hildenbrand --- drivers/acpi/numa/srat.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/acpi/numa/srat.c b/dr

[PATCH v1 00/11] virtio-mem: paravirtualized memory

tio-mem added memory can be onlined to the ZONE_MOVABLE, but subblocks will not get unplugged from it. Q: How big should the initial (!virtio-mem) memory of a VM be? A: virtio-mem memory will not go to the DMA zones. So to avoid running out of DMA memory, I suggest something like 2-3GB on x86

[PATCH v1 02/11] virtio-mem: Paravirtualized memory hotplug

/virtio_mem.c b/drivers/virtio/virtio_mem.c new file mode 100644 index ..d8da656c9145 --- /dev/null +++ b/drivers/virtio/virtio_mem.c @@ -0,0 +1,1559 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Virtio-mem device driver. + * + * Copyright Red Hat, Inc. 2020 + * + * Author(s): David Hil

[PATCH v1 03/11] virtio-mem: Paravirtualized memory hotunplug part 1

Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 116 +++- 1 file changed, 114 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index d8

[PATCH v1 05/11] virtio-mem: Paravirtualized memory hotunplug part 2

el Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/Kconfig | 1 + drivers/virtio/virtio_mem.c | 157 2 files changed, 144 insertions(+), 14 deletions(-) diff --git a/drivers/virtio/Kconfig b/drivers

[PATCH v1 04/11] mm: Export alloc_contig_range() / free_contig_range()

o Cc: Vlastimil Babka Cc: Oscar Salvador Cc: Mel Gorman Cc: Mike Rapoport Cc: Dan Williams Cc: Alexander Duyck Cc: Pavel Tatashin Cc: Alexander Potapenko Acked-by: Michal Hocko # to export contig range allocator API Signed-off-by: David Hildenbrand --- mm/page_alloc.c | 2 ++ 1 file chang

[PATCH v1 06/11] mm: Allow to offline unmovable PageOffline() pages via MEM_GOING_OFFLINE

n Cc: Alexander Duyck Cc: Vlastimil Babka Cc: Johannes Weiner Cc: Anthony Yznaga Cc: Michal Hocko Cc: Oscar Salvador Cc: Mel Gorman Cc: Mike Rapoport Cc: Dan Williams Cc: Anshuman Khandual Cc: Qian Cai Cc: Pingfan Liu Signed-off-by: David Hildenbrand --- include/linux/page-flags.h

[PATCH v1 08/11] mm/memory_hotplug: Introduce offline_and_remove_memory()

vices (esp. find_memory_block() and device_offline()) and the device hotplug lock. To keep things simple, allow to only work on a single memory block. Cc: Andrew Morton Cc: David Hildenbrand Cc: Oscar Salvador Cc: Michal Hocko Cc: Pavel Tatashin Cc: Wei Yang Cc: Dan Williams Cc: Qian Cai Sig

[PATCH v1 07/11] virtio-mem: Allow to offline partially unplugged memory blocks

ichal Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 64 - 1 file changed, 63 insertions(+), 1 de

[PATCH v1 09/11] virtio-mem: Offline and remove completely unplugged memory blocks

nother notifier NACKs the offlining request). Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Oscar Salvador Cc: Michal Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenb

<    1   2   3   4   5   6   7   8   9   >