Re: [PATCH v2 27/29] mm/memory_hotplug: extend offline_and_remove_memory() to handle more than one memory block

2020-11-18 Thread David Hildenbrand
On 18.11.20 05:53, Andrew Morton wrote: On Thu, 12 Nov 2020 14:38:13 +0100 David Hildenbrand wrote: virtio-mem soon wants to use offline_and_remove_memory() memory that exceeds a single Linux memory block (memory_block_size_bytes()). Let's remove that restriction. Let's remember the old

[PATCH v2 29/29] virtio-mem: Big Block Mode (BBM) - safe memory hotunplug

2020-11-12 Thread David Hildenbrand
Wei Yang Cc: Andrew Morton Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 97 - 1 file changed, 95 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index f1696cdb7b0c..9fc9ec4a25f5 100644

[PATCH v2 25/29] virtio-mem: Big Block Mode (BBM) memory hotplug

2020-11-12 Thread David Hildenbrand
that we might see in the future. So we won't care how many Linux memory blocks a big block actually spans, and how the memory notifier is called. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Cc: Michal Hocko Cc: Oscar Salvador Cc: Wei Yang Cc: Andrew Morton Signed-off

[PATCH v2 28/29] virtio-mem: Big Block Mode (BBM) - basic memory hotunplug

2020-11-12 Thread David Hildenbrand
block sizes. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Cc: Michal Hocko Cc: Oscar Salvador Cc: Wei Yang Cc: Andrew Morton Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 156 +++- 1 file changed, 155 insert

[PATCH v2 26/29] virtio-mem: allow to force Big Block Mode (BBM) and set the big block size

2020-11-12 Thread David Hildenbrand
ael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Cc: Michal Hocko Cc: Oscar Salvador Cc: Wei Yang Cc: Andrew Morton Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 31 --- 1 file changed, 28 insertions(+), 3 deletions(-) diff --git a/driv

[PATCH v2 27/29] mm/memory_hotplug: extend offline_and_remove_memory() to handle more than one memory block

2020-11-12 Thread David Hildenbrand
-by: Wei Yang Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Cc: Michal Hocko Cc: Oscar Salvador Cc: Wei Yang Cc: Andrew Morton Signed-off-by: David Hildenbrand --- mm/memory_hotplug.c | 105 +--- 1 file changed, 89 inserti

[PATCH v2 22/29] virito-mem: existing (un)plug functions are specific to Sub Block Mode (SBM)

2020-11-12 Thread David Hildenbrand
Let's rename them accordingly. virtio_mem_plug_request() and virtio_mem_unplug_request() will be handled separately. Reviewed-by: Wei Yang Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_

[PATCH v2 21/29] virtio-mem: memory block ids are specific to Sub Block Mode (SBM)

2020-11-12 Thread David Hildenbrand
Let's move first_mb_id/next_mb_id/last_usable_mb_id accordingly. Reviewed-by: Wei Yang Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 46 ++--- 1 file changed, 23 inserti

[PATCH v2 20/29] virtio-mem: nb_sb_per_mb and subblock_size are specific to Sub Block Mode (SBM)

2020-11-12 Thread David Hildenbrand
Let's rename to "sbs_per_mb" and "sb_size" and move accordingly. Reviewed-by: Wei Yang Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 96 ++--- 1 file

[PATCH v2 19/29] virito-mem: subblock states are specific to Sub Block Mode (SBM)

2020-11-12 Thread David Hildenbrand
Let's rename and move accordingly. While at it, rename sb_bitmap to "sb_states". Reviewed-by: Wei Yang Reviewed-by: Pankaj Gupta Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virti

[PATCH v2 24/29] virtio-mem: factor out adding/removing memory from Linux

2020-11-12 Thread David Hildenbrand
that in racy context soon (and we do have proper BUG_ON() statements in the current cases where it must never happen). Reviewed-by: Wei Yang Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_m

[PATCH v2 23/29] virtio-mem: memory notifier callbacks are specific to Sub Block Mode (SBM)

2020-11-12 Thread David Hildenbrand
Let's rename accordingly. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 29 +++-- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/driv

[PATCH v2 16/29] virtio-mem: generalize handling when memory is getting onlined deferred

2020-11-12 Thread David Hildenbrand
relevant for Big Block Mode (BBM), whereby we might re-online memory in case offlining of another memory block failed. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 95 ---

[PATCH v2 13/29] virtio-mem: generalize virtio_mem_overlaps_range()

2020-11-12 Thread David Hildenbrand
Avoid using memory block ids. While at it, use uint64_t for address/size. This is a preparation for Big Block Mode (BBM). Reviewed-by: Pankaj Gupta Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 10 +++

[PATCH v2 15/29] virtio-mem: don't always trigger the workqueue when offlining memory

2020-11-12 Thread David Hildenbrand
to 1. The result are endless retries. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 40 ++--- 1 file changed, 28 insertions(+), 12 deletions(-) diff --git a/drivers/virtio/vi

[PATCH v2 17/29] virito-mem: document Sub Block Mode (SBM)

2020-11-12 Thread David Hildenbrand
: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 15 +++ 1 file changed, 15 insertions(+) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index cbd0aa5eb95c..4234bfc0cf52 100644 ---

[PATCH v2 18/29] virtio-mem: memory block states are specific to Sub Block Mode (SBM)

2020-11-12 Thread David Hildenbrand
ot; - "set_mb_state" / "get_mb_state" vs. "mb_set_state" / "mb_get_state" - Don't use lengthy "enum virtio_mem_smb_mb_state", simply use "uint8_t" Reviewed-by: Wei Yang Reviewed-by: Pankaj Gupta Cc: "Michael S. Tsirkin&quo

[PATCH v2 14/29] virtio-mem: drop last_mb_id

2020-11-12 Thread David Hildenbrand
No longer used, let's drop it. Reviewed-by: Wei Yang Reviewed-by: Pankaj Gupta Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 4 1 file changed, 4 deletions(-) diff --git a/drivers/virtio/virt

[PATCH v2 11/29] virtio-mem: generalize check for added memory

2020-11-12 Thread David Hildenbrand
block states. Reviewed-by: Wei Yang Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 19 +++ 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/driv

[PATCH v2 07/29] virtio-mem: print debug messages from virtio_mem_send_*_request()

2020-11-12 Thread David Hildenbrand
Let's move the existing dev_dbg() into the functions, print if something went wrong, and also print for virtio_mem_send_unplug_all_request(). Reviewed-by: Wei Yang Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_

[PATCH v2 01/29] virtio-mem: determine nid only once using memory_add_physaddr_to_nid()

2020-11-12 Thread David Hildenbrand
Let's determine the target nid only once in case we have none specified - usually, we'll end up with node 0 either way. Reviewed-by: Wei Yang Reviewed-by: Pankaj Gupta Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/vi

[PATCH v2 10/29] virtio-mem: retry fake-offlining via alloc_contig_range() on ZONE_MOVABLE

2020-11-12 Thread David Hildenbrand
ael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 37 ++--- 1 file changed, 26 insertions(+), 11 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index c2

[PATCH v2 06/29] virtio-mem: factor out calculation of the bit number within the subblock bitmap

2020-11-12 Thread David Hildenbrand
The calculation is already complicated enough, let's limit it to one location. Reviewed-by: Wei Yang Reviewed-by: Pankaj Gupta Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 20 +++- 1 fi

[PATCH v2 00/29] virtio-mem: Big Block Mode (BBM)

2020-11-12 Thread David Hildenbrand
ake reviweing easier, and to have cleanups+optimizations before all refactorings - "virtio-mem: more precise calculation in virtio_mem_mb_state_prepare_next_mb()" -- Changed subject -- Avoid two local variables David Hildenbrand (29): virtio-mem: determine nid on

[PATCH v2 09/29] virtio-mem: factor out handling of fake-offline pages in memory notifier

2020-11-12 Thread David Hildenbrand
Let's factor out the core pieces and place the implementation next to virtio_mem_fake_offline(). We'll reuse this functionality soon. Reviewed-by: Wei Yang Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_

[PATCH v2 12/29] virtio-mem: generalize virtio_mem_owned_mb()

2020-11-12 Thread David Hildenbrand
Avoid using memory block ids. Rename it to virtio_mem_contains_range(). This is a preparation for Big Block Mode (BBM). Reviewed-by: Pankaj Gupta Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 9 +--

[PATCH v2 08/29] virtio-mem: factor out fake-offlining into virtio_mem_fake_offline()

2020-11-12 Thread David Hildenbrand
... which now matches virtio_mem_fake_online(). We'll reuse this functionality soon. Reviewed-by: Wei Yang Reviewed-by: Pankaj Gupta Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_

[PATCH v2 03/29] virtio-mem: simplify MAX_ORDER - 1 / pageblock_order handling

2020-11-12 Thread David Hildenbrand
Let's use pageblock_nr_pages and MAX_ORDER_NR_PAGES instead where possible to simplify. Add a comment why we have that restriction for now. Reviewed-by: Wei Yang Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_

[PATCH v2 04/29] virtio-mem: drop rc2 in virtio_mem_mb_plug_and_add()

2020-11-12 Thread David Hildenbrand
We can drop rc2, we don't actually need the value. Reviewed-by: Pankaj Gupta Reviewed-by: Wei Yang Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletion

[PATCH v2 05/29] virtio-mem: use "unsigned long" for nr_pages when fake onlining/offlining

2020-11-12 Thread David Hildenbrand
No harm done, but let's be consistent. Reviewed-by: Pankaj Gupta Reviewed-by: Wei Yang Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 8 1 file changed, 4 insertions(+), 4 deletions(-)

[PATCH v2 02/29] virtio-mem: more precise calculation in virtio_mem_mb_state_prepare_next_mb()

2020-11-12 Thread David Hildenbrand
We actually need one byte less (next_mb_id is exclusive, first_mb_id is inclusive). While at it, compact the code. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 6 ++ 1 file changed, 2 insert

Re: [PATCH v1 09/29] virtio-mem: don't always trigger the workqueue when offlining memory

2020-10-20 Thread David Hildenbrand
>> So, you'll keep retrying forever. Found while debugging that exact issue :) >> > > If this is the case, my suggestion is to record it in the changelog. > Otherwise, we may lose this corner case which is important to this change. Yes, already added it - thanks! -- Thanks, David /

Re: [PATCH v1 20/29] virtio-mem: nb_sb_per_mb and subblock_size are specific to Sub Block Mode (SBM)

2020-10-19 Thread David Hildenbrand
On 18.10.20 14:41, Wei Yang wrote: > On Fri, Oct 16, 2020 at 03:17:06PM +0200, David Hildenbrand wrote: >> On 16.10.20 10:53, Wei Yang wrote: >>> On Mon, Oct 12, 2020 at 02:53:14PM +0200, David Hildenbrand wrote: >>>> Let's rename to "sbs_per_mb" and &quo

Re: [PATCH v1 21/29] virtio-mem: memory notifier callbacks are specific to Sub Block Mode (SBM)

2020-10-19 Thread David Hildenbrand
On 19.10.20 03:57, Wei Yang wrote: > On Mon, Oct 12, 2020 at 02:53:15PM +0200, David Hildenbrand wrote: >> Let's rename accordingly. >> >> Cc: "Michael S. Tsirkin" >> Cc: Jason Wang >> Cc: Pankaj Gupta >> Signed-off-by: David Hilden

Re: [PATCH v1 25/29] virtio-mem: Big Block Mode (BBM) memory hotplug

2020-10-19 Thread David Hildenbrand
On 19.10.20 04:26, Wei Yang wrote: > On Mon, Oct 12, 2020 at 02:53:19PM +0200, David Hildenbrand wrote: >> Currently, we do not support device block sizes that exceed the Linux >> memory block size. For example, having a device block size of 1 GiB (e.g., >> gigantic pages in

Re: [PATCH v1 28/29] virtio-mem: Big Block Mode (BBM) - basic memory hotunplug

2020-10-19 Thread David Hildenbrand
On 19.10.20 05:48, Wei Yang wrote: > On Mon, Oct 12, 2020 at 02:53:22PM +0200, David Hildenbrand wrote: >> Let's try to unplug completely offline big blocks first. Then, (if >> enabled via unplug_offline) try to offline and remove whole big blocks. >> >> No lock

Re: [PATCH v1 09/29] virtio-mem: don't always trigger the workqueue when offlining memory

2020-10-19 Thread David Hildenbrand
On 18.10.20 05:57, Wei Yang wrote: > On Fri, Oct 16, 2020 at 11:18:39AM +0200, David Hildenbrand wrote: >> On 16.10.20 06:03, Wei Yang wrote: >>> On Mon, Oct 12, 2020 at 02:53:03PM +0200, David Hildenbrand wrote: >>>> Let's trigger from offlining code when we'

Re: [PATCH v1 29/29] virtio-mem: Big Block Mode (BBM) - safe memory hotunplug

2020-10-19 Thread David Hildenbrand
On 19.10.20 09:54, Wei Yang wrote: > On Mon, Oct 12, 2020 at 02:53:23PM +0200, David Hildenbrand wrote: >> Let's add a safe mechanism to unplug memory, avoiding long/endless loops >> when trying to offline memory - similar to in SBM. >> >> Fake-offline all memory (via a

Re: [PATCH v1 00/29] virtio-mem: Big Block Mode (BBM)

2020-10-18 Thread David Hildenbrand
> Am 18.10.2020 um 14:50 schrieb Wei Yang : > > On Mon, Oct 12, 2020 at 02:52:54PM +0200, David Hildenbrand wrote: >> virtio-mem currently only supports device block sizes that span at most >> a single Linux memory block. For example, gigantic pages in the hyperviso

Re: [PATCH v1 00/29] virtio-mem: Big Block Mode (BBM)

2020-10-18 Thread David Hildenbrand
> Am 18.10.2020 um 17:29 schrieb Michael S. Tsirkin : > > On Mon, Oct 12, 2020 at 02:52:54PM +0200, David Hildenbrand wrote: >> virtio-mem currently only supports device block sizes that span at most >> a single Linux memory block. For example, gigantic pages in the hyperv

Re: [PATCH v1 05/29] virtio-mem: generalize check for added memory

2020-10-17 Thread David Hildenbrand
> Am 17.10.2020 um 00:38 schrieb Wei Yang : > > On Fri, Oct 16, 2020 at 12:32:50PM +0200, David Hildenbrand wrote: >>>>> Ok, I seems to understand the logic now. >>>>> >>>>> But how we prevent ONLINE_PARTIAL memory

Re: [PATCH v1 20/29] virtio-mem: nb_sb_per_mb and subblock_size are specific to Sub Block Mode (SBM)

2020-10-16 Thread David Hildenbrand
On 16.10.20 10:53, Wei Yang wrote: > On Mon, Oct 12, 2020 at 02:53:14PM +0200, David Hildenbrand wrote: >> Let's rename to "sbs_per_mb" and "sb_size" and move accordingly. >> >> Cc: "Michael S. Tsirkin" >> Cc: Jason Wang >> Cc: Pank

Re: [PATCH v1 25/29] virtio-mem: Big Block Mode (BBM) memory hotplug

2020-10-16 Thread David Hildenbrand
On 16.10.20 11:38, Wei Yang wrote: > On Mon, Oct 12, 2020 at 02:53:19PM +0200, David Hildenbrand wrote: >> Currently, we do not support device block sizes that exceed the Linux >> memory block size. For example, having a device block size of 1 GiB (e.g., >> gigantic pages in

Re: [PATCH v1 05/29] virtio-mem: generalize check for added memory

2020-10-16 Thread David Hildenbrand
>>> Ok, I seems to understand the logic now. >>> >>> But how we prevent ONLINE_PARTIAL memory block get offlined? There are three >>> calls in virtio_mem_set_fake_offline(), while all of them adjust page's >>> flag. >>> How they hold reference to struct page? >> >> Sorry, I should have given you

Re: [PATCH v1 09/29] virtio-mem: don't always trigger the workqueue when offlining memory

2020-10-16 Thread David Hildenbrand
On 16.10.20 06:03, Wei Yang wrote: > On Mon, Oct 12, 2020 at 02:53:03PM +0200, David Hildenbrand wrote: >> Let's trigger from offlining code when we're not allowed to touch online >> memory. > > This describes the change in virtio_mem_memory_notifier_cb()? Ah, yes, can try

Re: [PATCH v1 05/29] virtio-mem: generalize check for added memory

2020-10-16 Thread David Hildenbrand
>> That's an interesting corner case. Assume you have a 128MB memory block >> but only 64MB are plugged. > > Since we just plug a part of memory block, this state is OFFLINE_PARTIAL > first. But then we would add these memory and online it. This means the state > of this memory block is

Re: [PATCH v1 02/29] virtio-mem: simplify calculation in virtio_mem_mb_state_prepare_next_mb()

2020-10-16 Thread David Hildenbrand
On 15.10.20 22:24, Pankaj Gupta wrote: >> We actually need one byte less (next_mb_id is exclusive, first_mb_id is >> inclusive). Simplify. >> >> Cc: "Michael S. Tsirkin" >> Cc: Jason Wang >> Cc: Pankaj Gupta >> Signed-off-by: David Hil

Re: [PATCH v1 13/29] virtio-mem: factor out handling of fake-offline pages in memory notifier

2020-10-16 Thread David Hildenbrand
>> Do we adjust the count twice? >> > > Ah, I got the reason why we need to adjust count for *unplugged* sub-blocks. Exactly. > >>> - for (i = 0; i < nr_pages; i++) { >>> - page = pfn_to_page(pfn + i); >>> - if

Re: [PATCH v1 02/29] virtio-mem: simplify calculation in virtio_mem_mb_state_prepare_next_mb()

2020-10-15 Thread David Hildenbrand
On 15.10.20 12:00, Wei Yang wrote: > On Thu, Oct 15, 2020 at 10:00:15AM +0200, David Hildenbrand wrote: >> On 15.10.20 06:02, Wei Yang wrote: >>> On Mon, Oct 12, 2020 at 02:52:56PM +0200, David Hildenbrand wrote: >>>> We actually need one byte less (next_m

Re: [PATCH v1 15/29] virito-mem: document Sub Block Mode (SBM)

2020-10-15 Thread David Hildenbrand
On 12.10.20 14:53, David Hildenbrand wrote: > Let's add some documentation for the current mode - Sub Block Mode (SBM) - > to prepare for a new mode - Big Block Mode (BBM). > > Follow-up patches will properly factor out the existing Sub Block Mode > (SBM) and implement Device Block

Re: [PATCH v1 05/29] virtio-mem: generalize check for added memory

2020-10-15 Thread David Hildenbrand
On 15.10.20 10:28, Wei Yang wrote: > On Mon, Oct 12, 2020 at 02:52:59PM +0200, David Hildenbrand wrote: >> Let's check by traversing busy system RAM resources instead, to avoid >> relying on memory block states. >> >> Don't use walk_system_ram_range(), as that works on

Re: [PATCH v1 06/29] virtio-mem: generalize virtio_mem_owned_mb()

2020-10-15 Thread David Hildenbrand
On 15.10.20 10:32, Wei Yang wrote: > On Mon, Oct 12, 2020 at 02:53:00PM +0200, David Hildenbrand wrote: >> Avoid using memory block ids. Rename it to virtio_mem_contains_range(). >> >> Cc: "Michael S. Tsirkin" >> Cc: Jason Wang >> Cc: Pankaj

Re: [PATCH v1 02/29] virtio-mem: simplify calculation in virtio_mem_mb_state_prepare_next_mb()

2020-10-15 Thread David Hildenbrand
On 15.10.20 06:02, Wei Yang wrote: > On Mon, Oct 12, 2020 at 02:52:56PM +0200, David Hildenbrand wrote: >> We actually need one byte less (next_mb_id is exclusive, first_mb_id is >> inclusive). Simplify. >> >> Cc: "Michael S. Tsirkin" >> Cc: Jas

[PATCH v1 29/29] virtio-mem: Big Block Mode (BBM) - safe memory hotunplug

2020-10-12 Thread David Hildenbrand
w Morton Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 97 - 1 file changed, 95 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 6bcd0acbff32..09f11489be6f 100644 --- a/drivers/virti

[PATCH v1 26/29] virtio-mem: allow to force Big Block Mode (BBM) and set the big block size

2020-10-12 Thread David Hildenbrand
Jason Wang Cc: Pankaj Gupta Cc: Michal Hocko Cc: Oscar Salvador Cc: Wei Yang Cc: Andrew Morton Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 31 --- 1 file changed, 28 insertions(+), 3 deletions(-) diff --git a/drivers/virtio/virtio_mem.c

[PATCH v1 27/29] mm/memory_hotplug: extend offline_and_remove_memory() to handle more than one memory block

2020-10-12 Thread David Hildenbrand
: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Cc: Michal Hocko Cc: Oscar Salvador Cc: Wei Yang Cc: Andrew Morton Signed-off-by: David Hildenbrand --- mm/memory_hotplug.c | 105 +--- 1 file changed, 89 insertions(+), 16 deletions(-) diff

[PATCH v1 24/29] virtio-mem: print debug messages from virtio_mem_send_*_request()

2020-10-12 Thread David Hildenbrand
Let's move the existing dev_dbg() into the functions, print if something went wrong, and also print for virtio_mem_send_unplug_all_request(). Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_

[PATCH v1 25/29] virtio-mem: Big Block Mode (BBM) memory hotplug

2020-10-12 Thread David Hildenbrand
that we might see in the future. So we won't care how many Linux memory blocks a big block actually spans, and how the memory notifier is called. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Cc: Michal Hocko Cc: Oscar Salvador Cc: Wei Yang Cc: Andrew Morton Signed-off

[PATCH v1 28/29] virtio-mem: Big Block Mode (BBM) - basic memory hotunplug

2020-10-12 Thread David Hildenbrand
block sizes. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Cc: Michal Hocko Cc: Oscar Salvador Cc: Wei Yang Cc: Andrew Morton Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 156 +++- 1 file changed, 155 insert

[PATCH v1 23/29] virtio-mem: factor out adding/removing memory from Linux

2020-10-12 Thread David Hildenbrand
that in racy context soon (and we do have proper BUG_ON() statements in the current cases where it must never happen). Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 107 --

[PATCH v1 21/29] virtio-mem: memory notifier callbacks are specific to Sub Block Mode (SBM)

2020-10-12 Thread David Hildenbrand
Let's rename accordingly. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 29 +++-- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/driv

[PATCH v1 22/29] virtio-mem: memory block ids are specific to Sub Block Mode (SBM)

2020-10-12 Thread David Hildenbrand
Let's move first_mb_id/next_mb_id/last_usable_mb_id accordingly. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 44 ++--- 1 file changed, 22 insertions(+), 22 deletions(-)

[PATCH v1 20/29] virtio-mem: nb_sb_per_mb and subblock_size are specific to Sub Block Mode (SBM)

2020-10-12 Thread David Hildenbrand
Let's rename to "sbs_per_mb" and "sb_size" and move accordingly. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 96 ++--- 1 file changed, 48 insertions(+

[PATCH v1 18/29] virtio-mem: factor out calculation of the bit number within the sb_states bitmap

2020-10-12 Thread David Hildenbrand
The calculation is already complicated enough, let's limit it to one location. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 20 +++- 1 file changed, 15 insertions(+), 5 deletions(-)

[PATCH v1 17/29] virito-mem: subblock states are specific to Sub Block Mode (SBM)

2020-10-12 Thread David Hildenbrand
Let's rename and move accordingly. While at it, rename sb_bitmap to "sb_states". Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 118 +++- 1 file changed,

[PATCH v1 19/29] virito-mem: existing (un)plug functions are specific to Sub Block Mode (SBM)

2020-10-12 Thread David Hildenbrand
Let's rename them accordingly. virtio_mem_plug_request() and virtio_mem_unplug_request() will be handled separately. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 90 ++-

[PATCH v1 16/29] virtio-mem: memory block states are specific to Sub Block Mode (SBM)

2020-10-12 Thread David Hildenbrand
t" - "set_mb_state" / "get_mb_state" vs. "mb_set_state" / "mb_get_state" - Don't use lengthy "enum virtio_mem_smb_mb_state", simply use "uint8_t" Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David

[PATCH v1 11/29] virtio-mem: use "unsigned long" for nr_pages when fake onlining/offlining

2020-10-12 Thread David Hildenbrand
No harm done, but let's be consistent. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/driv

[PATCH v1 15/29] virito-mem: document Sub Block Mode (SBM)

2020-10-12 Thread David Hildenbrand
Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 15 +++ 1 file changed, 15 insertions(+) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index faeb759687fe..fd8685673fe4 100644 --- a/drivers/virtio/virtio_mem.c +++ b/driv

[PATCH v1 14/29] virtio-mem: retry fake-offlining via alloc_contig_range() on ZONE_MOVABLE

2020-10-12 Thread David Hildenbrand
ael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 37 ++--- 1 file changed, 26 insertions(+), 11 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index a2

[PATCH v1 13/29] virtio-mem: factor out handling of fake-offline pages in memory notifier

2020-10-12 Thread David Hildenbrand
Let's factor out the core pieces and place the implementation next to virtio_mem_fake_offline(). We'll reuse this functionality soon. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_

[PATCH v1 10/29] virtio-mem: generalize handling when memory is getting onlined deferred

2020-10-12 Thread David Hildenbrand
" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 95 - 1 file changed, 63 insertions(+), 32 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 8ea00f0b2ecd..cb2e8f254

[PATCH v1 12/29] virtio-mem: factor out fake-offlining into virtio_mem_fake_offline()

2020-10-12 Thread David Hildenbrand
... which now matches virtio_mem_fake_online(). We'll reuse this functionality soon. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 34 -- 1 file changed, 24 inserti

[PATCH v1 09/29] virtio-mem: don't always trigger the workqueue when offlining memory

2020-10-12 Thread David Hildenbrand
handling when offlining out of virtio_mem_notify_offline(), to share it with Device Block Mode (DBM) soon. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 40 ++--- 1 file c

[PATCH v1 08/29] virtio-mem: drop last_mb_id

2020-10-12 Thread David Hildenbrand
No longer used, let's drop it. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 4 1 file changed, 4 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 37

[PATCH v1 03/29] virtio-mem: simplify MAX_ORDER - 1 / pageblock_order handling

2020-10-12 Thread David Hildenbrand
Let's use pageblock_nr_pages and MAX_ORDER_NR_PAGES instead where possible, so we don't have do deal with allocation orders. Add a comment why we have that restriction for now. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- driv

[PATCH v1 05/29] virtio-mem: generalize check for added memory

2020-10-12 Thread David Hildenbrand
Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 19 +++ 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index b3eebac7191f..6bbd1cfd10d3 100644 --- a/drivers/virtio/virtio_mem.c +++ b/driv

[PATCH v1 04/29] virtio-mem: drop rc2 in virtio_mem_mb_plug_and_add()

2020-10-12 Thread David Hildenbrand
We can drop rc2, we don't actually need the value. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/drivers/virtio/virtio_mem.c b/driv

[PATCH v1 06/29] virtio-mem: generalize virtio_mem_owned_mb()

2020-10-12 Thread David Hildenbrand
Avoid using memory block ids. Rename it to virtio_mem_contains_range(). Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 9 + 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/driv

[PATCH v1 02/29] virtio-mem: simplify calculation in virtio_mem_mb_state_prepare_next_mb()

2020-10-12 Thread David Hildenbrand
We actually need one byte less (next_mb_id is exclusive, first_mb_id is inclusive). Simplify. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)

[PATCH v1 01/29] virtio-mem: determine nid only once using memory_add_physaddr_to_nid()

2020-10-12 Thread David Hildenbrand
Let's determine the target nid only once in case we have none specified - usually, we'll end up with node 0 either way. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 28 +++- 1 fi

[PATCH v1 00/29] virtio-mem: Big Block Mode (BBM)

2020-10-12 Thread David Hildenbrand
cks/review please) via the vhost tree once time has come. In the meantime, I'll do more testing. David Hildenbrand (29): virtio-mem: determine nid only once using memory_add_physaddr_to_nid() virtio-mem: simplify calculation in virtio_mem_mb_state_prepare_next_mb() virtio-mem: simplify MAX

[PATCH v1 07/29] virtio-mem: generalize virtio_mem_overlaps_range()

2020-10-12 Thread David Hildenbrand
Avoid using memory block ids. While at it, use uint64_t for address/size. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Pankaj Gupta Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 10 +++--- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/driv

Re: [PATCH] kernel/resource: Fix use of ternary condition in release_mem_region_adjustable

2020-09-22 Thread David Hildenbrand
ced and fixed > already, I did not find anything on LKML. Hasn't been noticed before (I guess most people build with GCC, which does not warn in this instance, at least for me) thanks! Commit ids are not stable yet, so Andrew will most probably squash it. Reviewed-by: David Hildenbra

Re: [PATCH] kernel/resource: make iomem_resource implicit in release_mem_region_adjustable()

2020-09-16 Thread David Hildenbrand
On 16.09.20 14:10, Wei Yang wrote: > On Wed, Sep 16, 2020 at 12:03:20PM +0200, David Hildenbrand wrote: >> On 16.09.20 12:02, Wei Yang wrote: >>> On Wed, Sep 16, 2020 at 09:30:41AM +0200, David Hildenbrand wrote: >>>> "mem" in the name already indicates the

Re: [PATCH] kernel/resource: make iomem_resource implicit in release_mem_region_adjustable()

2020-09-16 Thread David Hildenbrand
On 16.09.20 12:02, Wei Yang wrote: > On Wed, Sep 16, 2020 at 09:30:41AM +0200, David Hildenbrand wrote: >> "mem" in the name already indicates the root, similar to >> release_mem_region() and devm_request_mem_region(). Make it implicit. >> The only single caller alwa

[PATCH] kernel/resource: make iomem_resource implicit in release_mem_region_adjustable()

2020-09-16 Thread David Hildenbrand
lliams Cc: Jason Gunthorpe Cc: Kees Cook Cc: Ard Biesheuvel Cc: Pankaj Gupta Cc: Baoquan He Cc: Wei Yang Signed-off-by: David Hildenbrand --- Based on next-20200915. Follow up on "[PATCH v4 0/8] selective merging of system ram resources" [1] That's in next-20200915. As noted

Re: [PATCH v2 1/7] kernel/resource: make release_mem_region_adjustable() never fail

2020-09-15 Thread David Hildenbrand
On 15.09.20 11:06, Wei Yang wrote: > On Tue, Sep 15, 2020 at 09:35:30AM +0200, David Hildenbrand wrote: >> >>>> static int __ref try_remove_memory(int nid, u64 start, u64 size) >>>> { >>>>int rc = 0; >>>> @@ -1777,7 +1757,7 @@ static

Re: [PATCH v2 2/7] kernel/resource: move and rename IORESOURCE_MEM_DRIVER_MANAGED

2020-09-15 Thread David Hildenbrand
On 15.09.20 04:20, Wei Yang wrote: > On Tue, Sep 08, 2020 at 10:10:07PM +0200, David Hildenbrand wrote: >> IORESOURCE_MEM_DRIVER_MANAGED currently uses an unused PnP bit, which is >> always set to 0 by hardware. This is far from beautiful (and confusing), >> and the bit only

Re: [PATCH v2 1/7] kernel/resource: make release_mem_region_adjustable() never fail

2020-09-15 Thread David Hildenbrand
>> static int __ref try_remove_memory(int nid, u64 start, u64 size) >> { >> int rc = 0; >> @@ -1777,7 +1757,7 @@ static int __ref try_remove_memory(int nid, u64 start, >> u64 size) >> memblock_remove(start, size); >> } >> >> -__release_memory_resource(start, size); >>

Re: [PATCH v4 5/8] mm/memory_hotplug: MEMHP_MERGE_RESOURCE to specify merging of System RAM resources

2020-09-15 Thread David Hildenbrand
On 15.09.20 04:43, Wei Yang wrote: > On Fri, Sep 11, 2020 at 12:34:56PM +0200, David Hildenbrand wrote: >> Some add_memory*() users add memory in small, contiguous memory blocks. >> Examples include virtio-mem, hyper-v balloon, and the XEN balloon. >> >> This can quickl

Re: [PATCH 1/3] virtio-balloon: Constify id_table

2020-09-14 Thread David Hildenbrand
struct page_reporting_dev_info pr_dev_info; > }; > > -static struct virtio_device_id id_table[] = { > +static const struct virtio_device_id id_table[] = { > { VIRTIO_ID_BALLOON, VIRTIO_DEV_ANY_ID }, > { 0 }, > }; > Acked-by: David Hildenbrand -- Thanks, David / dhilde

Re: [PATCH 2/3] virtio_input: Constify id_table

2020-09-14 Thread David Hildenbrand
*vdev) > static unsigned int features[] = { > /* none */ > }; > -static struct virtio_device_id id_table[] = { > +static const struct virtio_device_id id_table[] = { > { VIRTIO_ID_INPUT, VIRTIO_DEV_ANY_ID }, > { 0 }, > }; >

Re: [PATCH 3/3] virtio-mem: Constify mem_id_table

2020-09-14 Thread David Hildenbrand
gt; }; > > -static struct virtio_device_id virtio_mem_id_table[] = { > +static const struct virtio_device_id virtio_mem_id_table[] = { > { VIRTIO_ID_MEM, VIRTIO_DEV_ANY_ID }, > { 0 }, > }; > Acked-by: David Hildenbrand -- Thanks, David / dhildenb _

[PATCH v4 8/8] hv_balloon: try to merge system ram resources

2020-09-11 Thread David Hildenbrand
minger Cc: Wei Liu Cc: Pankaj Gupta Cc: Baoquan He Cc: Wei Yang Signed-off-by: David Hildenbrand --- drivers/hv/hv_balloon.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/hv/hv_balloon.c b/drivers/hv/hv_balloon.c index 3c0d52e244520..b64d2efbefe71 100644 --- a/

[PATCH v4 7/8] xen/balloon: try to merge system ram resources

2020-09-11 Thread David Hildenbrand
: Roger Pau Monné Cc: Julien Grall Cc: Pankaj Gupta Cc: Baoquan He Cc: Wei Yang Signed-off-by: David Hildenbrand --- drivers/xen/balloon.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/xen/balloon.c b/drivers/xen/balloon.c index 9f40a294d398d..b57b2067ecbfb

[PATCH v4 6/8] virtio-mem: try to merge system ram resources

2020-09-11 Thread David Hildenbrand
the resource up again. Reviewed-by: Pankaj Gupta Cc: Andrew Morton Cc: Michal Hocko Cc: Dan Williams Cc: Michael S. Tsirkin Cc: Jason Wang Cc: Pankaj Gupta Cc: Baoquan He Cc: Wei Yang Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 3 ++- 1 file changed, 2 insertions(+), 1

[PATCH v4 3/8] mm/memory_hotplug: guard more declarations by CONFIG_MEMORY_HOTPLUG

2020-09-11 Thread David Hildenbrand
Hocko Cc: Dan Williams Cc: Pankaj Gupta Cc: Baoquan He Cc: Wei Yang Signed-off-by: David Hildenbrand --- include/linux/memory_hotplug.h | 12 +++- mm/sparse.c| 2 ++ 2 files changed, 5 insertions(+), 9 deletions(-) diff --git a/include/linux/memory_hotplug.h b

[PATCH v4 5/8] mm/memory_hotplug: MEMHP_MERGE_RESOURCE to specify merging of System RAM resources

2020-09-11 Thread David Hildenbrand
;K. Y. Srinivasan" Cc: Haiyang Zhang Cc: Stephen Hemminger Cc: Wei Liu Cc: Boris Ostrovsky Cc: Juergen Gross Cc: Stefano Stabellini Cc: Roger Pau Monné Cc: Julien Grall Cc: Pankaj Gupta Cc: Baoquan He Cc: Wei Yang Signed-off-by: David Hildenbrand --- include/linux/ioport.h | 4 ++

[PATCH v4 1/8] kernel/resource: make release_mem_region_adjustable() never fail

2020-09-11 Thread David Hildenbrand
- let's add WARN_ON_ONCE() in these cases so we can catch them. Cc: Andrew Morton Cc: Michal Hocko Cc: Dan Williams Cc: Jason Gunthorpe Cc: Kees Cook Cc: Ard Biesheuvel Cc: Pankaj Gupta Cc: Baoquan He Cc: Wei Yang Signed-off-by: David Hildenbrand --- include/linux/ioport.h | 4

  1   2   3   4   5   6   >