[virtio-dev] Re: [RFC][Patch v12 1/2] mm: page_reporting: core infrastructure

2019-08-14 Thread David Hildenbrand
On 14.08.19 01:14, Alexander Duyck wrote: > On Tue, Aug 13, 2019 at 3:34 AM David Hildenbrand wrote: >> >>>>>> +static int process_free_page(struct page *page, >>>>>> +struct page_reporting_config *phconf, int >>>

[virtio-dev] [PATCH v5 4/6] mm: Introduce Reported pages

2019-08-14 Thread David Hildenbrand
>>> +struct page *get_unreported_page(struct zone *zone, unsigned int order, >>> int mt) >>> +{ >>> + struct list_head *tail = get_unreported_tail(zone, order, mt); >>> + struct free_area *area = &(zone->free_area[order]); >>> + struct list_head *list = >free_list[mt]; >>> + struct page

[virtio-dev] Re: [PATCH v5 4/6] mm: Introduce Reported pages

2019-08-13 Thread David Hildenbrand
On 12.08.19 23:33, Alexander Duyck wrote: > From: Alexander Duyck > > In order to pave the way for free page reporting in virtualized > environments we will need a way to get pages out of the free lists and > identify those pages after they have been returned. To accomplish this, > this patch

[virtio-dev] Re: [RFC][Patch v12 1/2] mm: page_reporting: core infrastructure

2019-08-12 Thread David Hildenbrand
>> --- >> include/linux/mmzone.h | 11 ++ >> include/linux/page_reporting.h | 63 +++ >> mm/Kconfig | 6 + >> mm/Makefile| 1 + >> mm/page_alloc.c| 42 - >> mm/page_reporting.c| 332

[virtio-dev] Re: [RFC][Patch v12 1/2] mm: page_reporting: core infrastructure

2019-08-13 Thread David Hildenbrand
+static int process_free_page(struct page *page, +struct page_reporting_config *phconf, int count) +{ + int mt, order, ret = 0; + + mt = get_pageblock_migratetype(page); + order = page_private(page); +

[virtio-dev] Re: [RFC][Patch v12 1/2] mm: page_reporting: core infrastructure

2019-08-13 Thread David Hildenbrand
On 13.08.19 12:42, Nitesh Narayan Lal wrote: > > On 8/13/19 6:34 AM, David Hildenbrand wrote: >>>>>> +static int process_free_page(struct page *page, >>>>>> +struct page_reporting_config *phconf, int >>>>>

Re: [virtio-dev] Re: [RFC][Patch v12 1/2] mm: page_reporting: core infrastructure

2019-09-04 Thread David Hildenbrand
>>> For some reason, I am not seeing this work as I would have expected >>> but I don't have solid reasoning to share yet. It could be simply >>> because I am putting my hook at the wrong place. I will continue >>> investigating this. >>> >>> In any case, I may be over complicating things here, so

[virtio-dev] Re: [PATCH v9 0/8] stg mail -e --version=v9 \

2019-09-12 Thread David Hildenbrand
On 12.09.19 09:16, Michal Hocko wrote: > On Wed 11-09-19 18:09:18, David Hildenbrand wrote: >> On 11.09.19 15:51, Michal Hocko wrote: >>> On Wed 11-09-19 15:20:02, Michal Hocko wrote: >>> [...] >>>>> 4. Continuously report, not the "one time rep

[virtio-dev] Re: [PATCH v9 2/8] mm: Adjust shuffle code to allow for future coalescing

2019-09-09 Thread David Hildenbrand
il(void); > static inline void shuffle_free_memory(pg_data_t *pgdat) > { > if (!static_branch_unlikely(_alloc_shuffle_key)) > @@ -43,6 +45,11 @@ static inline bool is_shuffle_order(int order) > return false; > return order >= SHUFFLE_ORDER; > } > + > +static inline bool shuffle_pick_tail(void) > +{ > + return __shuffle_pick_tail(); > +} > #else > static inline void shuffle_free_memory(pg_data_t *pgdat) > { > @@ -60,5 +67,10 @@ static inline bool is_shuffle_order(int order) > { > return false; > } > + > +static inline bool shuffle_pick_tail(void) > +{ > + return false; > +} > #endif > #endif /* _MM_SHUFFLE_H */ > > Acked-by: David Hildenbrand -- Thanks, David / dhildenb - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org

[virtio-dev] Re: [PATCH v9 3/8] mm: Move set/get_pcppage_migratetype to mmzone.h

2019-09-09 Thread David Hildenbrand
> migratetype) > -{ > - page->index = migratetype; > -} > - > #ifdef CONFIG_PM_SLEEP > /* > * The following functions are used by the suspend/hibernate code to > temporarily > > Still have to understand in detail how this will be used, but the change

[virtio-dev] Re: [PATCH v9 5/8] arm64: Move hugetlb related definitions out of pgtable.h to page-defs.h

2019-09-09 Thread David Hildenbrand
On 07.09.19 19:25, Alexander Duyck wrote: > From: Alexander Duyck > > Move the static definition for things such as HUGETLB_PAGE_ORDER out of > asm/pgtable.h and place it in page-defs.h. By doing this the includes > become much easier to deal with as currently arm64 is the only architecture >

[virtio-dev] Re: [PATCH v9 1/8] mm: Add per-cpu logic to page shuffling

2019-09-09 Thread David Hildenbrand
On 07.09.19 19:25, Alexander Duyck wrote: > From: Alexander Duyck > > Change the logic used to generate randomness in the suffle path so that we > can avoid cache line bouncing. The previous logic was sharing the offset > and entropy word between all CPUs. As such this can result in cache line >

[virtio-dev] Re: [PATCH v9 7/8] virtio-balloon: Pull page poisoning config out of free page hinting

2019-09-09 Thread David Hildenbrand
- if (!page_poisoning_enabled()) > + /* Notify host if we care about poison value */ "Tell the host whether we care about poisoned pages." ? > + if (IS_ENABLED(CONFIG_PAGE_POISONING_NO_SANITY) || > + !page

Re: [virtio-dev] Re: [PATCH v9 0/8] stg mail -e --version=v9 \

2019-09-11 Thread David Hildenbrand
On 11.09.19 11:23, Michael S. Tsirkin wrote: > On Tue, Sep 10, 2019 at 06:22:37PM +0200, David Hildenbrand wrote: >> On 10.09.19 18:18, Dr. David Alan Gilbert wrote: >>> * Alexander Duyck (alexander.du...@gmail.com) wrote: >>>> On Tue, Sep 10, 2019

Re: [virtio-dev] Re: [PATCH v9 0/8] stg mail -e --version=v9 \

2019-09-10 Thread David Hildenbrand
On 10.09.19 18:18, Dr. David Alan Gilbert wrote: > * Alexander Duyck (alexander.du...@gmail.com) wrote: >> On Tue, Sep 10, 2019 at 7:47 AM Michal Hocko wrote: >>> >>> On Tue 10-09-19 07:42:43, Alexander Duyck wrote: On Tue, Sep 10, 2019 at 5:42 AM Michal Hocko wrote: > > I wanted to

[virtio-dev] Re: [PATCH v9 0/8] stg mail -e --version=v9 \

2019-09-11 Thread David Hildenbrand
On 11.09.19 14:25, Michal Hocko wrote: > On Wed 11-09-19 14:19:41, Michal Hocko wrote: >> On Wed 11-09-19 08:08:38, Michael S. Tsirkin wrote: >>> On Wed, Sep 11, 2019 at 01:36:19PM +0200, Michal Hocko wrote: On Tue 10-09-19 14:23:40, Alexander Duyck wrote: [...] > We don't put any

[virtio-dev] Re: [PATCH v9 0/8] stg mail -e --version=v9 \

2019-09-11 Thread David Hildenbrand
On 11.09.19 14:54, Michal Hocko wrote: > On Wed 11-09-19 14:42:41, David Hildenbrand wrote: >> On 11.09.19 14:25, Michal Hocko wrote: >>> On Wed 11-09-19 14:19:41, Michal Hocko wrote: >>>> On Wed 11-09-19 08:08:38, Michael S. Tsirkin wrote: >>>>> On

[virtio-dev] Re: [PATCH v9 0/8] stg mail -e --version=v9 \

2019-09-11 Thread David Hildenbrand
On 11.09.19 13:36, Michal Hocko wrote: > On Tue 10-09-19 14:23:40, Alexander Duyck wrote: > [...] >> We don't put any limitations on the allocator other then that it needs to >> clean up the metadata on allocation, and that it cannot allocate a page >> that is in the process of being reported

[virtio-dev] Re: [RFC][PATCH v12 0/2] mm: Support for page reporting

2019-09-11 Thread David Hildenbrand
On 12.08.19 15:12, Nitesh Narayan Lal wrote: > This patch series proposes an efficient mechanism for reporting free memory > from a guest to its hypervisor. It especially enables guests with no page > cache > (e.g., nvdimm, virtio-pmem) or with small page caches (e.g., ram > disk) to > rapidly

[virtio-dev] Re: [PATCH v7 3/6] mm: Use zone and order instead of free area in free_list manipulators

2019-09-06 Thread David Hildenbrand
gned long watermark; > struct zone *zone; > int mt; > @@ -3139,7 +3131,7 @@ int __isolate_free_page(struct page *page, unsigned int > order) > > /* Remove page from free list */ > > - del_page_from_free_area(page, area); > + del_page_from_free_list(page, zone, order); > > /* >* Set the pageblock if the isolated page is at least half of a > @@ -8560,7 +8552,7 @@ void zone_pcp_reset(struct zone *zone) > pr_info("remove from free list %lx %d %lx\n", > pfn, 1 << order, end_pfn); > #endif > - del_page_from_free_area(page, >free_area[order]); > + del_page_from_free_list(page, zone, order); > for (i = 0; i < (1 << order); i++) > SetPageReserved((page+i)); > pfn += (1 << order); > Looks like a nice cleanup to me. Reviewed-by: David Hildenbrand -- Thanks, David / dhildenb - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail: virtio-dev-h...@lists.oasis-open.org

[virtio-dev] Re: [PATCH v9 0/8] stg mail -e --version=v9 \

2019-09-11 Thread David Hildenbrand
On 11.09.19 15:51, Michal Hocko wrote: > On Wed 11-09-19 15:20:02, Michal Hocko wrote: > [...] >>> 4. Continuously report, not the "one time report everything" approach. >> >> So you mean the allocator reporting this rather than an external code to >> poll right? I do not know, how much this is

[virtio-dev] Re: [PATCH v9 0/8] stg mail -e --version=v9 \

2019-09-11 Thread David Hildenbrand
>> Something slightly similar is being performed by Nitesh's patch set. On >> every free of a certain granularity, he records it in the bitmap. These >> bits are "hints of free pages". >> >> A thread then walks over the bitmap and tries to allocate the "hints". >> If the pages were already reused,

[virtio-dev] Re: [PATCH v11 0/6] mm / virtio: Provide support for unused page reporting

2019-10-01 Thread David Hildenbrand
On 01.10.19 17:29, Alexander Duyck wrote: > This series provides an asynchronous means of reporting to a hypervisor > that a guest page is no longer in use and can have the data associated > with it dropped. To do this I have implemented functionality that allows > for what I am referring to as

[virtio-dev] Re: [PATCH v10 3/6] mm: Introduce Reported pages

2019-09-23 Thread David Hildenbrand
On 23.09.19 17:50, Michael S. Tsirkin wrote: > On Mon, Sep 23, 2019 at 05:47:24PM +0200, David Hildenbrand wrote: >> On 23.09.19 17:45, David Hildenbrand wrote: >>> On 23.09.19 17:37, Michael S. Tsirkin wrote: >>>> On Mon, Sep 23, 2019 at 08:28:00AM -0700, Alexander

[virtio-dev] Re: [PATCH v10 3/6] mm: Introduce Reported pages

2019-09-23 Thread David Hildenbrand
On 23.09.19 17:37, Michael S. Tsirkin wrote: > On Mon, Sep 23, 2019 at 08:28:00AM -0700, Alexander Duyck wrote: >> On Mon, Sep 23, 2019 at 8:00 AM Michael S. Tsirkin wrote: >>> >>> On Mon, Sep 23, 2019 at 07:50:15AM -0700, Alexander Duyck wrote: >> +static inline void >>

[virtio-dev] Re: [PATCH v10 3/6] mm: Introduce Reported pages

2019-09-23 Thread David Hildenbrand
On 23.09.19 17:45, David Hildenbrand wrote: > On 23.09.19 17:37, Michael S. Tsirkin wrote: >> On Mon, Sep 23, 2019 at 08:28:00AM -0700, Alexander Duyck wrote: >>> On Mon, Sep 23, 2019 at 8:00 AM Michael S. Tsirkin wrote: >>>> >>>> On Mon, Sep 23, 2019

[virtio-dev] Re: [PATCH v10 0/6] mm / virtio: Provide support for unused page reporting

2019-09-24 Thread David Hildenbrand
On 24.09.19 16:23, Michal Hocko wrote: > On Wed 18-09-19 10:52:25, Alexander Duyck wrote: > [...] >> In order to try and keep the time needed to find a non-reported page to >> a minimum we maintain a "reported_boundary" pointer. This pointer is used >> by the get_unreported_pages iterator to

[virtio-dev] Re: [PATCH v11 0/6] mm / virtio: Provide support for unused page reporting

2019-10-02 Thread David Hildenbrand
On 02.10.19 02:55, Alexander Duyck wrote: > On Tue, Oct 1, 2019 at 12:16 PM Nitesh Narayan Lal wrote: >> >> >> On 10/1/19 12:21 PM, Alexander Duyck wrote: >>> On Tue, 2019-10-01 at 17:35 +0200, David Hildenbrand wrote: >>>> On 01.10.19 17:29, Alexand

[virtio-dev] Re: [PATCH v10 0/6] mm / virtio: Provide support for unused page reporting

2019-09-24 Thread David Hildenbrand
On 24.09.19 19:07, Alexander Duyck wrote: > On Tue, Sep 24, 2019 at 8:32 AM David Hildenbrand wrote: >> >> On 24.09.19 16:23, Michal Hocko wrote: >>> On Wed 18-09-19 10:52:25, Alexander Duyck wrote: >>> [...] >>>> In order to try and kee

[virtio-dev] Re: [PATCH v11 0/6] mm / virtio: Provide support for unused page reporting

2019-10-01 Thread David Hildenbrand
>> I think Michal asked for a performance comparison against Nitesh's >> approach, to evaluate if keeping the reported state + tracking inside >> the buddy is really worth it. Do you have any such numbers already? (or >> did my tired eyes miss them in this cover letter? :/) >> > > I thought what

[virtio-dev] Re: [PATCH RFC v4 01/13] ACPI: NUMA: export pxm_to_node

2019-12-13 Thread David Hildenbrand
On 12.12.19 22:43, Rafael J. Wysocki wrote: > On Thursday, December 12, 2019 6:11:25 PM CET David Hildenbrand wrote: >> Will be needed by virtio-mem to identify the node from a pxm. >> >> Cc: "Rafael J. Wysocki" >> Cc: Len Brown >> Cc: linux-a

[virtio-dev] [PATCH RFC v4 02/13] virtio-mem: Paravirtualized memory hotplug

2019-12-12 Thread David Hildenbrand
Tsirkin" Cc: Jason Wang Cc: Oscar Salvador Cc: Michal Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Cc: "Rafael J. Wysocki" Cc: Len Brown Cc: linux-a...@vger.kernel.org Signed-off-

[virtio-dev] [PATCH RFC v4 03/13] virtio-mem: Paravirtualized memory hotunplug part 1

2019-12-12 Thread David Hildenbrand
Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 109 +++- 1 file changed, 107 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 8f

[virtio-dev] [PATCH RFC v4 01/13] ACPI: NUMA: export pxm_to_node

2019-12-12 Thread David Hildenbrand
Will be needed by virtio-mem to identify the node from a pxm. Cc: "Rafael J. Wysocki" Cc: Len Brown Cc: linux-a...@vger.kernel.org Signed-off-by: David Hildenbrand --- drivers/acpi/numa/srat.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/acpi/numa/srat.c b/drivers

[virtio-dev] [PATCH RFC v4 00/13] virtio-mem: paravirtualized memory

2019-12-12 Thread David Hildenbrand
es. So to avoid running out of DMA memory, I suggest something like 2-3GB on x86-64. But many VMs can most probably deal with less DMA memory - depends on the use case. [1] https://events.linuxfoundation.org/wp-content/uploads/2017/12/virtio-mem-Paravirtualized-Memory-David-Hildenbrand-Red

[virtio-dev] [PATCH RFC v4 06/13] mm: Allow to offline unmovable PageOffline() pages via MEM_GOING_OFFLINE

2019-12-12 Thread David Hildenbrand
: Alexander Duyck Cc: Vlastimil Babka Cc: Johannes Weiner Cc: Anthony Yznaga Cc: Michal Hocko Cc: Oscar Salvador Cc: Mel Gorman Cc: Mike Rapoport Cc: Dan Williams Cc: Anshuman Khandual Cc: Qian Cai Cc: Pingfan Liu Signed-off-by: David Hildenbrand --- include/linux/page-flags.h | 10

[virtio-dev] [PATCH RFC v4 07/13] virtio-mem: Allow to offline partially unplugged memory blocks

2019-12-12 Thread David Hildenbrand
Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 64 - 1 file changed, 63 insertions(+), 1 deletio

[virtio-dev] [PATCH RFC v4 04/13] mm: Export alloc_contig_range() / free_contig_range()

2019-12-12 Thread David Hildenbrand
: Vlastimil Babka Cc: Oscar Salvador Cc: Mel Gorman Cc: Mike Rapoport Cc: Dan Williams Cc: Alexander Duyck Cc: Pavel Tatashin Cc: Alexander Potapenko Acked-by: Michal Hocko # to export contig range allocator API Signed-off-by: David Hildenbrand --- mm/page_alloc.c | 2 ++ 1 file changed, 2

[virtio-dev] [PATCH RFC v4 05/13] virtio-mem: Paravirtualized memory hotunplug part 2

2019-12-12 Thread David Hildenbrand
Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/Kconfig | 1 + drivers/virtio/virtio_mem.c | 156 2 fi

[virtio-dev] [PATCH RFC v4 13/13] virtio-mem: Drop slab objects when unplug continues to fail

2019-12-12 Thread David Hildenbrand
g Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Cc: linux-fsde...@vger.kernel.org Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 30 ++ 1 file changed, 30 insertions(+) diff --git a/drivers/virtio/virtio_mem.c b/driv

[virtio-dev] [PATCH RFC v4 11/13] mm/vmscan: Move count_vm_event(DROP_SLAB) into drop_slab()

2019-12-12 Thread David Hildenbrand
Let's count within the function itself, so every invocation (of future users) will be counted. Cc: Alexander Viro Cc: Andrew Morton Cc: linux-fsde...@vger.kernel.org Signed-off-by: David Hildenbrand --- fs/drop_caches.c | 4 +--- mm/vmscan.c | 1 + 2 files changed, 2 insertions(+), 3

[virtio-dev] [PATCH RFC v4 09/13] virtio-mem: Offline and remove completely unplugged memory blocks

2019-12-12 Thread David Hildenbrand
NACKs the offlining request). Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Oscar Salvador Cc: Michal Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- driv

[virtio-dev] [PATCH RFC v4 10/13] virtio-mem: Better retry handling

2019-12-12 Thread David Hildenbrand
Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/vi

[virtio-dev] [PATCH RFC v4 12/13] mm/vmscan: Export drop_slab() and drop_slab_node()

2019-12-12 Thread David Hildenbrand
off-by: David Hildenbrand --- include/linux/mm.h | 4 ++-- mm/vmscan.c| 2 ++ 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 64799c5cb39f..483300f58be8 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2706,8 +2706,8 @@

[virtio-dev] Re: [PATCH RFC v4 00/13] virtio-mem: paravirtualized memory

2019-12-16 Thread David Hildenbrand
On 13.12.19 21:15, Konrad Rzeszutek Wilk wrote: > On Thu, Dec 12, 2019 at 06:11:24PM +0100, David Hildenbrand wrote: >> This series is based on latest linux-next. The patches are located at: >> https://github.com/davidhildenbrand/linux.git virtio-mem-rfc-v4 > Heya! Hi K

[virtio-dev] Re: [PATCH RFC v4 12/13] mm/vmscan: Export drop_slab() and drop_slab_node()

2020-02-25 Thread David Hildenbrand
On 25.02.20 18:06, Michal Hocko wrote: > On Tue 25-02-20 16:09:29, David Hildenbrand wrote: >> On 25.02.20 15:58, Michal Hocko wrote: >>> On Thu 12-12-19 18:11:36, David Hildenbrand wrote: >>>> We already have a way to trigger reclaiming of all reclaimable slab object

[virtio-dev] Re: [PATCH RFC v4 06/13] mm: Allow to offline unmovable PageOffline() pages via MEM_GOING_OFFLINE

2020-02-25 Thread David Hildenbrand
>> /* >> * Scan pfn range [start,end) to find movable/migratable pages (LRU pages, >> - * non-lru movable pages and hugepages). We scan pfn because it's much >> - * easier than scanning over linked list. This function returns the pfn >> - * of the first found movable page if it's found,

[virtio-dev] Re: [PATCH RFC v4 06/13] mm: Allow to offline unmovable PageOffline() pages via MEM_GOING_OFFLINE

2020-02-25 Thread David Hildenbrand
On 25.02.20 22:46, Alexander Duyck wrote: > On Tue, 2020-02-25 at 19:49 +0100, David Hildenbrand wrote: >>>> /* >>>> * Scan pfn range [start,end) to find movable/migratable pages (LRU pages, >>>> - * non-lru movable pages and hugepages). We scan pfn

[virtio-dev] [PATCH v1 07/11] virtio-mem: Allow to offline partially unplugged memory blocks

2020-03-02 Thread David Hildenbrand
Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 64 - 1 file changed, 63 insertions(+), 1 deletio

[virtio-dev] [PATCH v1 09/11] virtio-mem: Offline and remove completely unplugged memory blocks

2020-03-02 Thread David Hildenbrand
NACKs the offlining request). Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Oscar Salvador Cc: Michal Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- driv

[virtio-dev] [PATCH v1 06/11] mm: Allow to offline unmovable PageOffline() pages via MEM_GOING_OFFLINE

2020-03-02 Thread David Hildenbrand
: Alexander Duyck Cc: Vlastimil Babka Cc: Johannes Weiner Cc: Anthony Yznaga Cc: Michal Hocko Cc: Oscar Salvador Cc: Mel Gorman Cc: Mike Rapoport Cc: Dan Williams Cc: Anshuman Khandual Cc: Qian Cai Cc: Pingfan Liu Signed-off-by: David Hildenbrand --- include/linux/page-flags.h | 10

[virtio-dev] [PATCH v1 11/11] MAINTAINERS: Add myself as virtio-mem maintainer

2020-03-02 Thread David Hildenbrand
Let's make sure patches/bug reports find the right person. Cc: "Michael S. Tsirkin" Signed-off-by: David Hildenbrand --- MAINTAINERS | 7 +++ 1 file changed, 7 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 1636ce4613e3..fc7371c4b7eb 100644 --- a/MAINTAINERS +++ b/M

[virtio-dev] [PATCH v1 08/11] mm/memory_hotplug: Introduce offline_and_remove_memory()

2020-03-02 Thread David Hildenbrand
. find_memory_block() and device_offline()) and the device hotplug lock. To keep things simple, allow to only work on a single memory block. Cc: Andrew Morton Cc: David Hildenbrand Cc: Oscar Salvador Cc: Michal Hocko Cc: Pavel Tatashin Cc: Wei Yang Cc: Dan Williams Cc: Qian Cai Signed-off

[virtio-dev] [PATCH v1 10/11] virtio-mem: Better retry handling

2020-03-02 Thread David Hildenbrand
Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/vi

[virtio-dev] Re: [PATCH v1 04/11] mm: Export alloc_contig_range() / free_contig_range()

2020-03-02 Thread David Hildenbrand
On 02.03.20 15:05, Michal Hocko wrote: > On Mon 02-03-20 14:49:34, David Hildenbrand wrote: >> A virtio-mem device wants to allocate memory from the memory region it >> manages in order to unplug it in the hypervisor - similar to >> a balloon driver. Also, it might wa

[virtio-dev] [PATCH v1 02/11] virtio-mem: Paravirtualized memory hotplug

2020-03-02 Thread David Hildenbrand
/virtio_mem.c b/drivers/virtio/virtio_mem.c new file mode 100644 index ..d8da656c9145 --- /dev/null +++ b/drivers/virtio/virtio_mem.c @@ -0,0 +1,1559 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Virtio-mem device driver. + * + * Copyright Red Hat, Inc. 2020 + * + * Author(s): David Hil

[virtio-dev] [PATCH v1 01/11] ACPI: NUMA: export pxm_to_node

2020-03-02 Thread David Hildenbrand
Will be needed by virtio-mem to identify the node from a pxm. Acked-by: "Rafael J. Wysocki" Cc: Len Brown Cc: linux-a...@vger.kernel.org Signed-off-by: David Hildenbrand --- drivers/acpi/numa/srat.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/acpi/numa/srat.c b/dr

[virtio-dev] [PATCH v1 05/11] virtio-mem: Paravirtualized memory hotunplug part 2

2020-03-02 Thread David Hildenbrand
tashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/Kconfig | 1 + drivers/virtio/virtio_mem.c | 157 2 files changed, 144 insertions(+), 14 deletions(-) diff --git a/drivers/virtio/Kconfig b/drivers/virt

[virtio-dev] [PATCH v1 03/11] virtio-mem: Paravirtualized memory hotunplug part 1

2020-03-02 Thread David Hildenbrand
Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 116 +++- 1 file changed, 114 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index d8

[virtio-dev] [PATCH v1 04/11] mm: Export alloc_contig_range() / free_contig_range()

2020-03-02 Thread David Hildenbrand
: Vlastimil Babka Cc: Oscar Salvador Cc: Mel Gorman Cc: Mike Rapoport Cc: Dan Williams Cc: Alexander Duyck Cc: Pavel Tatashin Cc: Alexander Potapenko Acked-by: Michal Hocko # to export contig range allocator API Signed-off-by: David Hildenbrand --- mm/page_alloc.c | 2 ++ 1 file changed, 2

[virtio-dev] Re: [PATCH v1 01/11] ACPI: NUMA: export pxm_to_node

2020-03-02 Thread David Hildenbrand
On 02.03.20 15:03, Michal Hocko wrote: > On Mon 02-03-20 14:49:31, David Hildenbrand wrote: >> Will be needed by virtio-mem to identify the node from a pxm. > > No objection to export the symbol. But it is almost always better to add > the export in the patch that actually uses

[virtio-dev] [PATCH v1 00/11] virtio-mem: paravirtualized memory

2020-03-02 Thread David Hildenbrand
ubblocks will not get unplugged from it. Q: How big should the initial (!virtio-mem) memory of a VM be? A: virtio-mem memory will not go to the DMA zones. So to avoid running out of DMA memory, I suggest something like 2-3GB on x86-64. But many VMs can most probably deal with less DMA memory

[virtio-dev] Re: [PATCH RFC v4 08/13] mm/memory_hotplug: Introduce offline_and_remove_memory()

2020-03-02 Thread David Hildenbrand
On 02.03.20 13:48, Michal Hocko wrote: > On Tue 25-02-20 15:27:28, David Hildenbrand wrote: >> On 25.02.20 15:11, Michal Hocko wrote: >>> On Thu 12-12-19 18:11:32, David Hildenbrand wrote: >>>> virtio-mem wants to offline and remove a memory block once it unplugged &

[virtio-dev] Re: [PATCH v1 00/11] virtio-mem: paravirtualized memory

2020-03-02 Thread David Hildenbrand
On 02.03.20 14:49, David Hildenbrand wrote: > This series is based on latest linux-next. The patches are located at: > https://github.com/davidhildenbrand/linux.git virtio-mem-v1 > > The basic idea of virtio-mem is to provide a flexible, > cross-architecture memory hot(u

[virtio-dev] Re: [PATCH v1 00/11] virtio-mem: paravirtualized memory

2020-03-02 Thread David Hildenbrand
On 02.03.20 19:29, Michal Hocko wrote: > On Mon 02-03-20 19:15:09, David Hildenbrand wrote: > [...] >> As requested by Michal, I will squash some patches. > > Just to clarify. If I am the only one to care then do not bother. Oh, I do bother about your review comments a lot

[virtio-dev] Re: [PATCH v1 06/11] mm: Allow to offline unmovable PageOffline() pages via MEM_GOING_OFFLINE

2020-03-02 Thread David Hildenbrand
On 02.03.20 18:40, Alexander Duyck wrote: > Reviewed-by: Alexander Duyck Thanks a lot Alex! -- Thanks, David / dhildenb - To unsubscribe, e-mail: virtio-dev-unsubscr...@lists.oasis-open.org For additional commands, e-mail:

[virtio-dev] Re: [PATCH RFC v4 12/13] mm/vmscan: Export drop_slab() and drop_slab_node()

2020-02-25 Thread David Hildenbrand
On 25.02.20 15:58, Michal Hocko wrote: > On Thu 12-12-19 18:11:36, David Hildenbrand wrote: >> We already have a way to trigger reclaiming of all reclaimable slab objects >> from user space (echo 2 > /proc/sys/vm/drop_caches). Let's allow drivers >> to also trigger th

[virtio-dev] Re: [PATCH RFC v4 00/13] virtio-mem: paravirtualized memory

2020-02-25 Thread David Hildenbrand
On 29.01.20 10:41, David Hildenbrand wrote: > On 09.01.20 14:48, David Hildenbrand wrote: >> On 12.12.19 18:11, David Hildenbrand wrote: >>> This series is based on latest linux-next. The patches are located at: >>> https://github.com/davidhildenbrand/

[virtio-dev] Re: [PATCH RFC v4 08/13] mm/memory_hotplug: Introduce offline_and_remove_memory()

2020-02-25 Thread David Hildenbrand
On 25.02.20 15:11, Michal Hocko wrote: > On Thu 12-12-19 18:11:32, David Hildenbrand wrote: >> virtio-mem wants to offline and remove a memory block once it unplugged >> all subblocks (e.g., using alloc_contig_range()). Let's provide >> an interface to do that from a drive

[virtio-dev] Re: [PATCH v1 02/11] virtio-mem: Paravirtualized memory hotplug

2020-03-03 Thread David Hildenbrand
if your patch is applied to the wrong git tree, please drop us a note to help > improve the system. BTW, we also suggest to use '--base' option to specify the > base tree in git format-patch, please see > https://stackoverflow.com/a/37406982] > > url: > https://github.com/0day-ci/

[virtio-dev] Re: [PATCH RFC v4 00/13] virtio-mem: paravirtualized memory

2020-01-29 Thread David Hildenbrand
On 09.01.20 14:48, David Hildenbrand wrote: > On 12.12.19 18:11, David Hildenbrand wrote: >> This series is based on latest linux-next. The patches are located at: >> https://github.com/davidhildenbrand/linux.git virtio-mem-rfc-v4 >> >> The basic idea of virtio-m

[virtio-dev] Re: Balloon pressuring page cache

2020-02-05 Thread David Hildenbrand
On 04.02.20 21:33, Michael S. Tsirkin wrote: > On Tue, Feb 04, 2020 at 05:56:22PM +0100, David Hildenbrand wrote: >> [...] >> >>>> >>>> Issue 2: When called via the shrinker, (but also to fix Issue 1), it could >>>> be >>>> that

[virtio-dev] Re: [PATCH RFC v4 00/13] virtio-mem: paravirtualized memory

2020-01-09 Thread David Hildenbrand
On 12.12.19 18:11, David Hildenbrand wrote: > This series is based on latest linux-next. The patches are located at: > https://github.com/davidhildenbrand/linux.git virtio-mem-rfc-v4 > > The basic idea of virtio-mem is to provide a flexible, > cross-architecture memory hot(u

[virtio-dev] Re: [PATCH RFC v4 00/13] virtio-mem: paravirtualized memory

2019-12-24 Thread David Hildenbrand
> Am 24.12.2019 um 08:04 schrieb teawater : > > Hi David, > > Thanks for your work. > > I Got following build fail if X86_64_ACPI_NUMA is n with rfc3 and rfc4: > make -j8 bzImage > GEN Makefile > DESCEND objtool > CALL

[virtio-dev] Re: [PATCH v1 07/11] virtio-mem: Allow to offline partially unplugged memory blocks

2020-03-10 Thread David Hildenbrand
On 10.03.20 12:59, Michal Hocko wrote: > On Tue 10-03-20 12:46:05, David Hildenbrand wrote: >> On 10.03.20 12:43, Michal Hocko wrote: >>> On Mon 02-03-20 14:49:37, David Hildenbrand wrote: >>> [...] >>>> +static void virtio_mem_not

[virtio-dev] Re: [PATCH v1 06/11] mm: Allow to offline unmovable PageOffline() pages via MEM_GOING_OFFLINE

2020-03-10 Thread David Hildenbrand
On 10.03.20 12:47, Michal Hocko wrote: > On Mon 02-03-20 14:49:36, David Hildenbrand wrote: >> virtio-mem wants to allow to offline memory blocks of which some parts >> were unplugged (allocated via alloc_contig_range()), especially, to later >> offline and remove comple

[virtio-dev] Re: [PATCH v1 07/11] virtio-mem: Allow to offline partially unplugged memory blocks

2020-03-10 Thread David Hildenbrand
On 10.03.20 12:43, Michal Hocko wrote: > On Mon 02-03-20 14:49:37, David Hildenbrand wrote: > [...] >> +static void virtio_mem_notify_going_offline(struct virtio_mem *vm, >> +unsigned long mb_id) >> +{ >> +const unsign

[virtio-dev] Re: [PATCH v2 07/10] mm/memory_hotplug: Introduce offline_and_remove_memory()

2020-03-11 Thread David Hildenbrand
On 11.03.20 18:14, David Hildenbrand wrote: > virtio-mem wants to offline and remove a memory block once it unplugged > all subblocks (e.g., using alloc_contig_range()). Let's provide > an interface to do that from a driver. virtio-mem already supports to > offline partially unpl

[virtio-dev] [PATCH v2 06/10] virtio-mem: Allow to offline partially unplugged memory blocks

2020-03-11 Thread David Hildenbrand
Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 68 - 1 file changed, 67 insertions(+), 1 deletion(-) diff --git a/drivers/virtio/virtio_mem.c

[virtio-dev] [PATCH v2 08/10] virtio-mem: Offline and remove completely unplugged memory blocks

2020-03-11 Thread David Hildenbrand
NACKs the offlining request). Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Oscar Salvador Cc: Michal Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- driv

[virtio-dev] [PATCH v2 10/10] MAINTAINERS: Add myself as virtio-mem maintainer

2020-03-11 Thread David Hildenbrand
Let's make sure patches/bug reports find the right person. Cc: "Michael S. Tsirkin" Signed-off-by: David Hildenbrand --- MAINTAINERS | 7 +++ 1 file changed, 7 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index f6dcd1233935..ee0e7cbca5c6 100644 --- a/MAINTAINERS +++ b/M

[virtio-dev] [PATCH v2 09/10] virtio-mem: Better retry handling

2020-03-11 Thread David Hildenbrand
n Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index aa322e7732a4..48e96702d4ce 100644 --- a/driv

[virtio-dev] [PATCH v2 05/10] mm: Allow to offline unmovable PageOffline() pages via MEM_GOING_OFFLINE

2020-03-11 Thread David Hildenbrand
: Konrad Rzeszutek Wilk Cc: Pavel Tatashin Cc: Alexander Duyck Cc: Vlastimil Babka Cc: Johannes Weiner Cc: Anthony Yznaga Cc: Michal Hocko Cc: Oscar Salvador Cc: Mel Gorman Cc: Mike Rapoport Cc: Dan Williams Cc: Anshuman Khandual Cc: Qian Cai Cc: Pingfan Liu Signed-off-by: David

[virtio-dev] [PATCH v2 07/10] mm/memory_hotplug: Introduce offline_and_remove_memory()

2020-03-11 Thread David Hildenbrand
. find_memory_block() and device_offline()) and the device hotplug lock. To keep things simple, allow to only work on a single memory block. Cc: Andrew Morton Cc: David Hildenbrand Cc: Oscar Salvador Cc: Michal Hocko Cc: Pavel Tatashin Cc: Wei Yang Cc: Dan Williams Cc: Qian Cai Signed-off

[virtio-dev] [PATCH v2 02/10] virtio-mem: Allow to specify an ACPI PXM as nid

2020-03-11 Thread David Hildenbrand
socki" # for the export Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Oscar Salvador Cc: Michal Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Cc: Len Brown Cc: linux-a...@vger.kernel.org

[virtio-dev] [PATCH v2 04/10] virtio-mem: Paravirtualized memory hotunplug part 2

2020-03-11 Thread David Hildenbrand
Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Cc: Mel Gorman Cc: Mike Rapoport Cc: Alexander Duyck Cc: Alexander Potapenko Signed-off-by: David Hildenbrand --- drivers/virtio/Kconfig | 1 + drivers/virtio/virtio_m

[virtio-dev] [PATCH v2 03/10] virtio-mem: Paravirtualized memory hotunplug part 1

2020-03-11 Thread David Hildenbrand
Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 116 +++- 1 file changed, 114 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index d8

[virtio-dev] [PATCH v2 01/10] virtio-mem: Paravirtualized memory hotplug

2020-03-11 Thread David Hildenbrand
Vlastimil Babka Cc: "Rafael J. Wysocki" Cc: Len Brown Cc: linux-a...@vger.kernel.org Signed-off-by: David Hildenbrand --- drivers/virtio/Kconfig | 17 + drivers/virtio/Makefile |1 + drivers/virtio/virtio_mem.c | 1526 +++ incl

[virtio-dev] [PATCH v2 00/10] virtio-mem: paravirtualized memory

2020-03-11 Thread David Hildenbrand
io-mem memory will not go to the DMA zones. So to avoid running out of DMA memory, I suggest something like 2-3GB on x86-64. But many VMs can most probably deal with less DMA memory - depends on the use case. [1] https://events.linuxfoundation.org/wp-content/uploads/2017/12/virtio-mem-Parav

[virtio-dev] Re: [PATCH v18 QEMU 3/3] virtio-balloon: Provide a interface for free page reporting

2020-04-09 Thread David Hildenbrand
On 09.04.20 19:34, Alexander Duyck wrote: > On Thu, Apr 9, 2020 at 12:44 AM David Hildenbrand wrote: >> >> On 09.04.20 00:55, Alexander Duyck wrote: >>> From: Alexander Duyck >>> >>> Add support for what I am referring to as "free page reportin

[virtio-dev] Re: [PATCH v18 QEMU 3/3] virtio-balloon: Provide a interface for free page reporting

2020-04-09 Thread David Hildenbrand
On 09.04.20 00:55, Alexander Duyck wrote: > From: Alexander Duyck > > Add support for what I am referring to as "free page reporting". "Add support for "free page reporting". > Basically the idea is to function very similar to how the balloon works > in that we basically end up madvising the

[virtio-dev] Re: [PATCH v18 QEMU 2/3] virtio-balloon: Add support for providing free page reports to host

2020-04-09 Thread David Hildenbrand
On 09.04.20 00:55, Alexander Duyck wrote: > From: Alexander Duyck > > Add support for the page reporting feature provided by virtio-balloon. > Reporting differs from the regular balloon functionality in that is is > much less durable than a standard memory balloon. Instead of creating a > list

[virtio-dev] Re: [PATCH v2 00/10] virtio-mem: paravirtualized memory

2020-04-14 Thread David Hildenbrand
On 14.04.20 18:28, Michael S. Tsirkin wrote: > On Tue, Apr 14, 2020 at 11:15:18AM +0200, David Hildenbrand wrote: >> On 29.03.20 14:42, Michael S. Tsirkin wrote: >>> On Wed, Mar 11, 2020 at 06:14:12PM +0100, David Hildenbrand wrote: >>>> This series is based on

[virtio-dev] Re: [PATCH v19 QEMU 1/4] virtio-balloon: Implement support for page poison tracking feature

2020-04-15 Thread David Hildenbrand
> > The comment above explains the "why". Basically poisoning a page will > dirty it. So why hint a page as free when that will drop it back into > the guest and result in it being dirtied again. What you end up with > is all the pages that were temporarily placed in the balloon are dirty > after

[virtio-dev] Re: [PATCH v2] virtio-balloon: Avoid using the word 'report' when referring to free page hinting

2020-04-15 Thread David Hildenbrand
to carry that name for legacy support. > + */ > + union { > + __u32 free_page_hint_cmd_id; > + __u32 free_page_report_cmd_id; /* deprecated */ > + }; > /* Stores PAGE_POISON if page poisoning is in use */ > __u32 poison_val; >

[virtio-dev] Re: [PATCH v19 QEMU 3/4] virtio-balloon: Provide an interface for free page reporting

2020-04-15 Thread David Hildenbrand
On 10.04.20 05:41, Alexander Duyck wrote: > From: Alexander Duyck > > Add support for free page reporting. The idea is to function very similar > to how the balloon works in that we basically end up madvising the page as > not being used. However we don't really need to bother with any deflate >

[virtio-dev] Re: [PATCH v19 QEMU 1/4] virtio-balloon: Implement support for page poison tracking feature

2020-04-15 Thread David Hildenbrand
On 10.04.20 05:41, Alexander Duyck wrote: > From: Alexander Duyck > > We need to make certain to advertise support for page poison tracking if > we want to actually get data on if the guest will be poisoning pages. So > if free page hinting is active we should add page poisoning support and >

[virtio-dev] Re: [PATCH v19 QEMU 3/4] virtio-balloon: Provide an interface for free page reporting

2020-04-15 Thread David Hildenbrand
On 15.04.20 10:17, David Hildenbrand wrote: > On 10.04.20 05:41, Alexander Duyck wrote: >> From: Alexander Duyck >> >> Add support for free page reporting. The idea is to function very similar >> to how the balloon works in that we basically end up madvising the page as

[virtio-dev] Re: [PATCH v19 QEMU 1/4] virtio-balloon: Implement support for page poison tracking feature

2020-04-16 Thread David Hildenbrand
> The other thing to keep in mind is that the poison value only really > comes into play with hinting/reporting. In the case of the standard > balloon the pages are considered allocated from the guest's Currently just as free page hinting IMHO. They are temporarily considered allocated. >

[virtio-dev] Re: [PATCH v19 QEMU 1/4] virtio-balloon: Implement support for page poison tracking feature

2020-04-16 Thread David Hildenbrand
>> We should document our result of page poisoning, free page hinting, and >> free page reporting there as well. I hope you'll have time for the latter. >> >> - >> Semantics of VIRTIO_BALLOON_F_PAGE_POISON >>

  1   2   3   >