[virtio-dev] [RFC][Patch v12 1/2] mm: page_reporting: core infrastructure

2019-08-12 Thread Nitesh Narayan Lal
This patch introduces the core infrastructure for free page reporting in virtual environments. It enables the kernel to track the free pages which can be reported to its hypervisor so that the hypervisor could free and reuse that memory as per its requirement. While the pages are getting processed

[virtio-dev] [RFC][PATCH v12 0/2] mm: Support for page reporting

2019-08-12 Thread Nitesh Narayan Lal
This patch series proposes an efficient mechanism for reporting free memory from a guest to its hypervisor. It especially enables guests with no page cache (e.g., nvdimm, virtio-pmem) or with small page caches (e.g., ram > disk) to rapidly hand back free memory to the hypervisor. This approach has

[virtio-dev] [RFC][Patch v12 2/2] virtio-balloon: interface to support free page reporting

2019-08-12 Thread Nitesh Narayan Lal
Enables the kernel to negotiate VIRTIO_BALLOON_F_REPORTING feature with the host. If it is available and page_reporting_flag is set to true, page_reporting is enabled and its callback is configured along with the max_pages count which indicates the maximum number of pages that can be isolated and r

[virtio-dev] [QEMU Patch 1/2] virtio-balloon: adding bit for page reporting support

2019-08-12 Thread Nitesh Narayan Lal
This patch will be replaced once the feature is merged into the Linux kernel. Signed-off-by: Nitesh Narayan Lal --- include/standard-headers/linux/virtio_balloon.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/standard-headers/linux/virtio_balloon.h b/include/standard-headers/linu

[virtio-dev] [QEMU Patch 2/2] virtio-balloon: support for handling page reporting

2019-08-12 Thread Nitesh Narayan Lal
Page reporting is a feature which enables the virtual machine to report chunk of free pages to the hypervisor. This patch enables QEMU to process these reports from the VM and discard the unused memory range. Signed-off-by: Nitesh Narayan Lal --- hw/virtio/virtio-balloon.c | 41 +

[virtio-dev] Re: [PATCH v5 2/2] virtio-fs: add DAX window

2019-08-12 Thread Stefan Hajnoczi
On Fri, Jul 26, 2019 at 07:41:57AM -0400, Michael S. Tsirkin wrote: > On Fri, Jul 26, 2019 at 10:53:38AM +0100, Stefan Hajnoczi wrote: > > Describe how shared memory region ID 0 is the DAX window and how > > FUSE_SETUPMAPPING maps file ranges into the window. > > > > Signed-off-by: Stefan Hajnoczi

[virtio-dev] Re: [PATCH v5 0/2] virtio-fs: add virtio file system device

2019-08-12 Thread Stefan Hajnoczi
On Fri, Jul 26, 2019 at 01:34:16PM +0100, Dr. David Alan Gilbert wrote: > * Michael S. Tsirkin (m...@redhat.com) wrote: > > Didn't look at the spec yet, but I have a question: > > > > On Fri, Jul 26, 2019 at 10:53:36AM +0100, Stefan Hajnoczi wrote: > > > v5: > > > * Explain multiqueue semantics:

[virtio-dev] Re: [QEMU Patch 2/2] virtio-balloon: support for handling page reporting

2019-08-12 Thread Alexander Duyck
On Mon, Aug 12, 2019 at 6:14 AM Nitesh Narayan Lal wrote: > > Page reporting is a feature which enables the virtual machine to report > chunk of free pages to the hypervisor. > This patch enables QEMU to process these reports from the VM and discard the > unused memory range. > > Signed-off-by: Ni

[virtio-dev] Re: [QEMU Patch 2/2] virtio-balloon: support for handling page reporting

2019-08-12 Thread Nitesh Narayan Lal
On 8/12/19 11:18 AM, Alexander Duyck wrote: > On Mon, Aug 12, 2019 at 6:14 AM Nitesh Narayan Lal wrote: >> Page reporting is a feature which enables the virtual machine to report >> chunk of free pages to the hypervisor. >> This patch enables QEMU to process these reports from the VM and discard

[virtio-dev] Re: [RFC][Patch v12 1/2] mm: page_reporting: core infrastructure

2019-08-12 Thread Alexander Duyck
On Mon, Aug 12, 2019 at 6:13 AM Nitesh Narayan Lal wrote: > > This patch introduces the core infrastructure for free page reporting in > virtual environments. It enables the kernel to track the free pages which > can be reported to its hypervisor so that the hypervisor could > free and reuse that

[virtio-dev] Re: [RFC][Patch v12 1/2] mm: page_reporting: core infrastructure

2019-08-12 Thread David Hildenbrand
>> --- >> include/linux/mmzone.h | 11 ++ >> include/linux/page_reporting.h | 63 +++ >> mm/Kconfig | 6 + >> mm/Makefile| 1 + >> mm/page_alloc.c| 42 - >> mm/page_reporting.c| 332 +

[virtio-dev] Re: [RFC][Patch v12 1/2] mm: page_reporting: core infrastructure

2019-08-12 Thread Nitesh Narayan Lal
On 8/12/19 2:47 PM, Alexander Duyck wrote: > On Mon, Aug 12, 2019 at 6:13 AM Nitesh Narayan Lal wrote: >> This patch introduces the core infrastructure for free page reporting in >> virtual environments. It enables the kernel to track the free pages which >> can be reported to its hypervisor so

[virtio-dev] [PATCH v5 2/6] mm: Move set/get_pcppage_migratetype to mmzone.h

2019-08-12 Thread Alexander Duyck
From: Alexander Duyck In order to support page reporting it will be necessary to store and retrieve the migratetype of a page. To enable that I am moving the set and get operations for pcppage_migratetype into the mm/internal.h header so that they can be used outside of the page_alloc.c file. Si

[virtio-dev] [PATCH v5 1/6] mm: Adjust shuffle code to allow for future coalescing

2019-08-12 Thread Alexander Duyck
From: Alexander Duyck This patch is meant to move the head/tail adding logic out of the shuffle code and into the __free_one_page function since ultimately that is where it is really needed anyway. By doing this we should be able to reduce the overhead and can consolidate all of the list addition

[virtio-dev] [PATCH v5 0/6] mm / virtio: Provide support for unused page reporting

2019-08-12 Thread Alexander Duyck
This series provides an asynchronous means of reporting to a hypervisor that a guest page is no longer in use and can have the data associated with it dropped. To do this I have implemented functionality that allows for what I am referring to as unused page reporting The functionality for this is

[virtio-dev] [PATCH v5 6/6] virtio-balloon: Add support for providing unused page reports to host

2019-08-12 Thread Alexander Duyck
From: Alexander Duyck Add support for the page reporting feature provided by virtio-balloon. Reporting differs from the regular balloon functionality in that is is much less durable than a standard memory balloon. Instead of creating a list of pages that cannot be accessed the pages are only inac

[virtio-dev] [PATCH v5 QEMU 1/3] virtio-ballon: Implement support for page poison tracking feature

2019-08-12 Thread Alexander Duyck
From: Alexander Duyck We need to make certain to advertise support for page poison tracking if we want to actually get data on if the guest will be poisoning pages. So if free page hinting is active we should add page poisoning support and let the guest disable it if it isn't using it. Page pois

[virtio-dev] [PATCH v5 5/6] virtio-balloon: Pull page poisoning config out of free page hinting

2019-08-12 Thread Alexander Duyck
From: Alexander Duyck Currently the page poisoning setting wasn't being enabled unless free page hinting was enabled. However we will need the page poisoning tracking logic as well for unused page reporting. As such pull it out and make it a separate bit of config in the probe function. In addit

[virtio-dev] [PATCH v5 QEMU 2/3] virtio-balloon: Add bit to notify guest of unused page reporting

2019-08-12 Thread Alexander Duyck
From: Alexander Duyck Add a bit for the page reporting feature provided by virtio-balloon. This patch should be replaced once the feature is added to the Linux kernel and the bit is backported into this exported kernel header. Signed-off-by: Alexander Duyck --- include/standard-headers/linux/

[virtio-dev] [PATCH v5 3/6] mm: Use zone and order instead of free area in free_list manipulators

2019-08-12 Thread Alexander Duyck
From: Alexander Duyck In order to enable the use of the zone from the list manipulator functions I will need access to the zone pointer. As it turns out most of the accessors were always just being directly passed &zone->free_area[order] anyway so it would make sense to just fold that into the fu

[virtio-dev] [PATCH v5 QEMU 3/3] virtio-balloon: Provide a interface for unused page reporting

2019-08-12 Thread Alexander Duyck
From: Alexander Duyck Add support for what I am referring to as "unused page reporting". Basically the idea is to function very similar to how the balloon works in that we basically end up madvising the page as not being used. However we don't really need to bother with any deflate type logic sin

[virtio-dev] [PATCH v5 4/6] mm: Introduce Reported pages

2019-08-12 Thread Alexander Duyck
From: Alexander Duyck In order to pave the way for free page reporting in virtualized environments we will need a way to get pages out of the free lists and identify those pages after they have been returned. To accomplish this, this patch adds the concept of a Reported Buddy, which is essentiall

[virtio-dev] Re: [PATCH v5 1/6] mm: Adjust shuffle code to allow for future coalescing

2019-08-12 Thread Alexander Duyck
On Mon, Aug 12, 2019 at 3:24 PM Dan Williams wrote: > > On Mon, Aug 12, 2019 at 2:33 PM Alexander Duyck > wrote: > > > > From: Alexander Duyck > > > > This patch is meant to move the head/tail adding logic out of the shuffle > > s/This patch is meant to move/Move/ I'll update that on next submi