Re: [Qemu-devel] Why is vhost-user limited to 8 slots?

2019-03-26 Thread Raphael Norwitz
ping From: Raphael Norwitz Date: Monday, March 18, 2019 at 7:10 PM To: "m...@redhat.com" Cc: "qemu-devel@nongnu.org" , Malcolm Crossley , Mike Cui , Felipe Franciosi Subject: [Qemu-devel] Why is vhost-user limited to 8 slots? Hi MST, Looking at the kernel commit lo

[Qemu-devel] Why is vhost-user limited to 8 slots?

2019-03-18 Thread Raphael Norwitz
Hi MST, Looking at the kernel commit log, I see that commit c9ce42f72fd0ba180fd35539829e4139dca31494 introduced support for a configurable number of regions in vhost, but vhost-user is still limited to 8 slots. Is there a reason for that? Would you consider a patch to either statically

[Qemu-devel] [PATCH] vhost-user-scsi: prevent using uninitialized vqs

2019-06-11 Thread Raphael Norwitz
0 The qemu commit e6cc11d64fc998c11a4dfcde8fda3fc33a74d844 has already resolved the issue for vhost scsi devices but the fix was never applied to vhost-user scsi devices. Signed-off-by: Raphael Norwitz --- hw/scsi/vhost-user-scsi.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff

[Qemu-devel] Should memory hotplug work with vhost-user backends?

2019-07-02 Thread Raphael Norwitz
For background I am trying to work around a ram slot limit imposed by the vhost-user protocol. We are having trouble reconciling the comment here: https://github.com/qemu/qemu/blob/master/hw/virtio/vhost-user.c#L333 that “For non-vring specific requests, like VHOST_USER_SET_MEM_TABLE., we just

Re: [Qemu-devel] [PATCH] vhost-user-scsi: prevent using uninitialized vqs

2019-08-21 Thread Raphael Norwitz
On Fri, Jun 14, 2019 at 10:18:41AM +0100, Stefan Hajnoczi wrote: > On Tue, Jun 11, 2019 at 05:35:17PM -0700, Raphael Norwitz wrote: > > Of the 3 virtqueues, seabios only sets cmd, leaving ctrl > > and event without a physical address. This can cause > > vhost_verify_ring_p

Re: [Qemu-devel] [PATCH] vhost-user-scsi: prevent using uninitialized vqs

2019-08-22 Thread Raphael Norwitz
On Thu, Aug 22, 2019 at 10:58:02AM +0200, Philippe Mathieu-Daudé wrote: > Hi Raphael, Stefan, > > On 6/14/19 11:18 AM, Stefan Hajnoczi wrote: > > On Tue, Jun 11, 2019 at 05:35:17PM -0700, Raphael Norwitz wrote: > >> Of the 3 virtqueues, seabios only sets cmd, leaving ctr

[Qemu-devel] [PATCH 1/2] vhost-user-blk: prevent using uninitialized vqs

2019-08-22 Thread Raphael Norwitz
for ring 0 qemu-system-x86_64: Verify ring failure on region 0 This has already been fixed for vhost scsi devices and was recently vhost-user scsi devices. This commit fixes it for vhost-user-blk devices. Suggested-by: Phillippe Mathieu-Daude Signed-off-by: Raphael Norwitz --- hw/block/vhost

[Qemu-devel] [PATCH 2/2] backends/vhost-user.c: prevent using uninitialized vqs

2019-08-22 Thread Raphael Norwitz
in their realize() to initialize their vqs, was not being properly zeroing out the queues. This commit ensures hardware modules using the vhost_user_backend_dev_init() API properly zero out their vqs on initialization. Suggested-by: Philippe Mathieu-Daude Signed-off-by: Raphael Norwitz --- backends

Re: [Qemu-devel] Should memory hotplug work with vhost-user backends?

2019-07-09 Thread Raphael Norwitz
On Jul 3, 2019, at 11:57 AM, Michael S. Tsirkin mailto:m...@redhat.com>> wrote: On Tue, Jul 02, 2019 at 10:08:54PM +, Raphael Norwitz wrote: For background I am trying to work around a ram slot limit imposed by the vhost-user protocol. We are having trouble reconciling the commen

[PATCH] vhost-user: Refractor vhost_user_set_mem_table Functions

2019-11-05 Thread Raphael Norwitz
, vhost_user_fill_set_mem_table_msg(). No functionality is impacted. Signed-off-by: Raphael Norwitz --- hw/virtio/vhost-user.c | 140 +++-- 1 file changed, 65 insertions(+), 75 deletions(-) diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index

[PATCH] [RFC] vhost-user: clean up set_mem_table functions

2019-11-05 Thread Raphael Norwitz
The functions sending vhost-user set memory table messages are getting convoluted. The amount of nested logic is getting in the way of my development and it looks like some identical logic should be refractored out anyways. Here???s an RFC which cleans these functions up a bit. Raphael Raphael

[PATCH 1/2] vhost-user: add VHOST_USER_RESET_DEVICE to reset devices

2019-10-31 Thread Raphael Norwitz
. If it does so, the new message is used instead of sending a RESET_OWNER which has had inconsistent implementations. Signed-off-by: David Vrabel Signed-off-by: Raphael Norwitz --- docs/interop/vhost-user.rst | 15 +++ hw/virtio/vhost-user.c | 8 +++- 2 files changed, 22

[PATCH 0/2] vhost-user: add message for device reset

2019-10-31 Thread Raphael Norwitz
supported message. Raphael Raphael Norwitz (2): vhost-user: add VHOST_USER_RESET_DEVICE to reset devices vhost-user-scsi: reset the device if supported docs/interop/vhost-user.rst | 15 +++ hw/scsi/vhost-user-scsi.c | 24 hw/virtio/vhost-user.c

[PATCH 2/2] vhost-user-scsi: reset the device if supported

2019-10-31 Thread Raphael Norwitz
into an inoperable state. Signed-off-by: David Vrabel Signed-off-by: Raphael Norwitz --- hw/scsi/vhost-user-scsi.c | 24 1 file changed, 24 insertions(+) diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c index 6a6c15d..23f972d 100644 --- a/hw/scsi/vhost-user-scsi.c

Re: [PATCH 1/2] vhost-user: add VHOST_USER_RESET_DEVICE to reset devices

2019-12-12 Thread Raphael Norwitz
On Wed, Nov 06, 2019 at 06:36:01AM -0500, Michael S. Tsirkin wrote: > > On Tue, Oct 29, 2019 at 05:38:02PM -0400, Raphael Norwitz wrote: > > Add a VHOST_USER_RESET_DEVICE message which will reset the vhost user > > backend. Disabling all rings, and resetting all int

[RFC PATCH 3/3] Introduce Configurable Number of Memory Slots Exposed by vhost-user:

2019-12-16 Thread Raphael Norwitz
if the VHOST_USER_PROTOCOL_F_REPLY_ACK feature has also been negotiated. Signed-off-by: Raphael Norwitz --- docs/interop/vhost-user.rst | 43 hw/virtio/vhost-user.c | 251 2 files changed, 273 insertions(+), 21 deletions(-) diff --git a/docs/interop

[RFC PATCH 1/3] Fixed Error Handling in vhost_user_set_mem_table_postcopy

2019-12-16 Thread Raphael Norwitz
(), gracefully propogating an error up instead of corrupting memory and crashing qemu. Signed-off-by: Raphael Norwitz --- hw/virtio/vhost-user.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 02a9b25..f74ff3b 100644 --- a/hw

[RFC PATCH 0/3] vhost-user: Lift Max Ram Slots Limitation

2019-12-16 Thread Raphael Norwitz
? - Is there a cleaner way to ensure to ensure a postcopy migration cannot be started with this protocol feature enabled? Best, Raphael Raphael Norwitz (3): Fixed Error Handling in vhost_user_set_mem_table_postcopy vhost-user: Refactor vhost_user_set_mem_table Functions Introduce Configurable

[RFC PATCH 2/3] vhost-user: Refactor vhost_user_set_mem_table Functions

2019-12-16 Thread Raphael Norwitz
, vhost_user_fill_set_mem_table_msg(). No functionality is impacted. Signed-off-by: Raphael Norwitz --- hw/virtio/vhost-user.c | 144 +++-- 1 file changed, 66 insertions(+), 78 deletions(-) diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index

Long term approaches to mitigate device reset issue in vhost-user-scsi

2019-10-25 Thread Raphael Norwitz
Hi MST, We are trying to develop a long term fix to the following issue with vhost-user-scsi: When a live migration starts, Qemu sends a SET_VRING_ADDR message to update the VQ's flags (turning log on). We can't distinguish that message from the first SET_VRING_ADDR message sent after a device

Re: [PATCH v2 3/3] Lift max memory slots limit imposed by vhost-user

2020-02-24 Thread Raphael Norwitz
Ping On Sun, Feb 09, 2020 at 12:43:35PM -0500, Raphael Norwitz wrote: > > On Thu, Feb 06, 2020 at 03:32:38AM -0500, Michael S. Tsirkin wrote: > > > > On Wed, Jan 15, 2020 at 09:57:06PM -0500, Raphael Norwitz wrote: > > > The current vhost-user implementat

[PATCH v2 0/3] vhost-user: Lift Max Ram Slots Limitation

2020-01-27 Thread Raphael Norwitz
to query the backend every time vhost_backend_memslots_limit is called. Best, Raphael Raphael Norwitz (3): Fixed assert in vhost_user_set_mem_table_postcopy Refactor vhost_user_set_mem_table functions Lift max memory slots limit imposed by vhost-user docs/interop/vhost-user.rst | 43 + hw

[PATCH v2 3/3] Lift max memory slots limit imposed by vhost-user

2020-01-27 Thread Raphael Norwitz
with postcopy migration and cannot be enabled if the VHOST_USER_PROTOCOL_F_REPLY_ACK feature has also been negotiated. Signed-off-by: Raphael Norwitz Signed-off-by: Peter Turschmid Suggested-by: Mike Cui --- docs/interop/vhost-user.rst | 43 hw/virtio/vhost-user.c | 254

[PATCH v2 1/3] Fixed assert in vhost_user_set_mem_table_postcopy

2020-01-27 Thread Raphael Norwitz
during the postcopy step. This change moves an existing assert up such that attempting to construct a VHOST_USER_SET_MEM_TABLE message with too many memory regions will gracefully bring down qemu instead of corrupting memory. Signed-off-by: Raphael Norwitz Signed-off-by: Peter Turschmid --- hw

[PATCH v2 2/3] Refactor vhost_user_set_mem_table functions

2020-01-27 Thread Raphael Norwitz
, vhost_user_fill_set_mem_table_msg(). No functionality is impacted. Signed-off-by: Raphael Norwitz Signed-off-by: Peter Turschmid --- hw/virtio/vhost-user.c | 143 +++-- 1 file changed, 67 insertions(+), 76 deletions(-) diff --git a/hw/virtio/vhost-user.c

Re: [PATCH v2 0/3] vhost-user: Lift Max Ram Slots Limitation

2020-02-05 Thread Raphael Norwitz
Ping On Wed, Jan 15, 2020 at 09:57:03PM -0500, Raphael Norwitz wrote: > > In QEMU today, a VM with a vhost-user device can hot add memory a > maximum of 8 times. See these threads, among others: > > [1] https://lists.gnu.org/archive/html/qemu-devel/2019-07/msg01046.htm

Re: [PATCH v2 1/3] Fixed assert in vhost_user_set_mem_table_postcopy

2020-02-10 Thread Raphael Norwitz
Yes - it's just a cleanup. On Thu, Feb 06, 2020 at 03:20:01AM -0500, Michael S. Tsirkin wrote: > > On Thu, Feb 06, 2020 at 03:17:04AM -0500, Michael S. Tsirkin wrote: > > On Wed, Jan 15, 2020 at 09:57:04PM -0500, Raphael Norwitz wrote: > > > The current vhost_user_s

Re: [PATCH v2 2/3] Refactor vhost_user_set_mem_table functions

2020-02-10 Thread Raphael Norwitz
Sounds good On Thu, Feb 06, 2020 at 03:21:42AM -0500, Michael S. Tsirkin wrote: > > On Wed, Jan 15, 2020 at 09:57:05PM -0500, Raphael Norwitz wrote: > > vhost_user_set_mem_table() and vhost_user_set_mem_table_postcopy() have > > gotten convoluted, and have s

Re: [PATCH v2 3/3] Lift max memory slots limit imposed by vhost-user

2020-02-10 Thread Raphael Norwitz
On Thu, Feb 06, 2020 at 03:32:38AM -0500, Michael S. Tsirkin wrote: > > On Wed, Jan 15, 2020 at 09:57:06PM -0500, Raphael Norwitz wrote: > > The current vhost-user implementation in Qemu imposes a limit on the > > maximum number of memory slots exposed to a VM using a vho

Re: [PATCH v2 0/3] vhost-user: Lift Max Ram Slots Limitation

2020-02-10 Thread Raphael Norwitz
On Thu, Feb 06, 2020 at 03:33:13AM -0500, Michael S. Tsirkin wrote: > > On Wed, Jan 15, 2020 at 09:57:03PM -0500, Raphael Norwitz wrote: > > > > Changes since V1: > > * Kept the assert in vhost_user_set_mem_table_postcopy, but moved it > > to prevent corru

Re: [PATCH v2 0/3] vhost-user: Lift Max Ram Slots Limitation

2020-02-19 Thread Raphael Norwitz
On Mon, Feb 10, 2020 at 11:04:28AM -0500, Michael S. Tsirkin wrote: > > On Sun, Feb 09, 2020 at 12:14:42PM -0500, Raphael Norwitz wrote: > > On Thu, Feb 06, 2020 at 03:33:13AM -0500, Michael S. Tsirkin wrote: > > > > > > On Wed, Jan 15, 2020 at 09:57:03P

Re: [RFC PATCH 1/3] Fixed Error Handling in vhost_user_set_mem_table_postcopy

2020-01-15 Thread Raphael Norwitz
Makes sense - will fix On Tue, Jan 14, 2020 at 02:07:03AM -0500, Michael S. Tsirkin wrote: > > On Mon, Dec 09, 2019 at 02:00:45AM -0500, Raphael Norwitz wrote: > > The current vhost_user_set_mem_table_postcopy() implementation > > populates each region of the VHOST_USER_SET_MEM

Re: [RFC PATCH 3/3] Introduce Configurable Number of Memory Slots Exposed by vhost-user:

2020-01-15 Thread Raphael Norwitz
On Tue, Jan 14, 2020 at 02:12:46AM -0500, Michael S. Tsirkin wrote: > > On Mon, Dec 09, 2019 at 02:00:47AM -0500, Raphael Norwitz wrote: > > The current vhost-user implementation in Qemu imposes a limit on the > > maximum number of memory slots exposed to a VM using a vho

Re: [RFC PATCH 0/3] vhost-user: Lift Max Ram Slots Limitation

2020-01-13 Thread Raphael Norwitz
Ping On Mon, Dec 09, 2019 at 02:00:44AM -0500, Raphael Norwitz wrote: > > In QEMU today, a VM with a vhost-user device can hot add memory a > maximum of 8 times. See these threads, among others: > > [1] https://lists.gnu.org/archive/html/qemu-devel/2019-07/msg01046.htm

Qemu master crashing on boot when using file backend for memory

2020-03-11 Thread Raphael Norwitz
When I try run master qemu I am hitting a divide by zero error. It seems to be coming from util/oslib-posix.c in touch_all_pages(). see line 477: numpages_per_thread = numpages / memset_num_threads; Poking around the crash dumps, I can see that the smp_cpus parameter passed in to

Re: [Qemu-devel] Should memory hotplug work with vhost-user backends?

2020-04-10 Thread Raphael Norwitz
On Wed, Jul 03, 2019 at 11:04:31AM +0100, Stefan Hajnoczi wrote: > On Tue, Jul 02, 2019 at 10:08:54PM +0000, Raphael Norwitz wrote: > > For background I am trying to work around a ram slot limit imposed by the > > vhost-user protocol. We are having trouble reconciling the comment h

Re: [PATCH] MAINTAINERS: Add myself as vhost-user-blk maintainer

2020-04-10 Thread Raphael Norwitz
Ping On Thu, Mar 26, 2020 at 04:47:35AM -0400, Raphael Norwitz wrote: > As suggested by Michael, let's add me as a maintainer of > vhost-user-blk and vhost-user-scsi. > > CC: Michael S. Tsirkin > CC Peter Maydell > Signed-off-by: Raphael Norwitz > --- > MAINTAINER

Feature and message numbers missing in vhost-user.c

2020-04-10 Thread Raphael Norwitz
Hi MST, I see that a new feature, VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS (https://git.qemu.org/?p=qemu.git;a=blob;f=docs/interop/vhost-user.rst;h=3b1b6602c77932101e1e3b098e35c2497f5b0260;hb=HEAD#l817), and corresponding VHOST_USER_VRING_KICK

Re: [PATCH] resend slave fd to vhost when reconnect to vhost

2020-04-17 Thread Raphael Norwitz
On Fri, Apr 17, 2020 at 01:14:00PM +0800, ni.xun.in...@gmail.com wrote: > > From: Ni Xun > > when reconnecting to vhost server, it doesn't send slave fd to vhost > as the slave fd is only sent in vhost_user_init. also resend the slave fd > in vhost reconnect. > I don’t think that’s correct.

Re: [PATCH 2/4] vhost-user-blk: fix invalid memory access

2020-04-16 Thread Raphael Norwitz
> Stack after return: f5 > Stack use after scope: f8 > Global redzone: f9 > Global init order: f6 > Poisoned by user:f7 > Container overflow: fc > Array cookie:ac > Intra object redzone:bb > ASan interna

Re: [PATCH 4/4] vhost-user-blk: fix crash in realize process

2020-04-16 Thread Raphael Norwitz
Mostly looks good - just a few superficial notes. On Wed, Apr 15, 2020 at 11:28:26AM +0800, Li Feng wrote: > 1. set s->connected to true after vhost_dev_init; > 2. call vhost_dev_get_config when s->connected is true, otherwise the > hdev->host_ops will be nullptr. You mean hdev->vhost_ops,

Re: [PATCH 1/4] vhost-user-blk: delay vhost_user_blk_disconnect

2020-04-16 Thread Raphael Norwitz
On Wed, Apr 15, 2020 at 11:28:23AM +0800, Li Feng wrote: > > switch (event) { > case CHR_EVENT_OPENED: > @@ -363,7 +376,16 @@ static void vhost_user_blk_event(void *opaque, > QEMUChrEvent event) > } > break; > case CHR_EVENT_CLOSED: > -

[PATCH] MAINTAINERS: Add myself as vhost-user-blk maintainer

2020-03-31 Thread Raphael Norwitz
As suggested by Michael, let's add me as a maintainer of vhost-user-blk and vhost-user-scsi. CC: Michael S. Tsirkin CC Peter Maydell Signed-off-by: Raphael Norwitz --- MAINTAINERS | 12 1 file changed, 12 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index e580276

[PATCH] Refactor vhost_user_set_mem_table functions

2020-03-25 Thread Raphael Norwitz
, vhost_user_fill_set_mem_table_msg(). No functionality is impacted. Signed-off-by: Raphael Norwitz Signed-off-by: Peter Turschmid --- hw/virtio/vhost-user.c | 143 +++-- 1 file changed, 67 insertions(+), 76 deletions(-) diff --git a/hw/virtio/vhost-user.c

Re: [PATCH] fix vhost_user_blk_watch crash

2020-03-29 Thread Raphael Norwitz
On Mon, Mar 23, 2020 at 01:29:24PM +0800, Li Feng wrote: > > the G_IO_HUP is watched in tcp_chr_connect, and the callback > vhost_user_blk_watch is not needed, because tcp_chr_hup is registered as > callback. And it will close the tcp link. > > Signed-off-by: Li Feng Re

Re: [PATCH] fix vhost_user_blk_watch crash

2020-03-29 Thread Raphael Norwitz
On Sun, Mar 29, 2020 at 09:30:24AM -0400, Michael S. Tsirkin wrote: > > On Mon, Mar 23, 2020 at 01:29:24PM +0800, Li Feng wrote: > > the G_IO_HUP is watched in tcp_chr_connect, and the callback > > vhost_user_blk_watch is not needed, because tcp_chr_hup is registered as > > callback. And it will

Re: Qemu master crashing on boot when using file backend for memory

2020-03-24 Thread Raphael Norwitz
On Thu, Mar 12, 2020 at 11:54:50AM +0100, Igor Mammedov wrote: > > On Thu, 12 Mar 2020 01:36:48 -0400 > Raphael Norwitz wrote: > > > When I try run master qemu I am hitting a divide by zero error. It seems > > to be coming from util/oslib-posix.c in touch_a

Re: [Qemu-devel] Should memory hotplug work with vhost-user backends?

2020-04-28 Thread Raphael Norwitz
> > I've briefly looked through the libvhost-user code and the hot-add path > > looks safe enough to me (or at least no more broken than the regular > > vhost-user memory hot-add path). > > > > Can you elaborate a little more about why memory hot-add is unsafe with > > vhost-user device backends

Re: [RFC PATCH v1 1/7] contrib/vhost-user-blk: add option to simulate disconnect on init

2020-04-23 Thread Raphael Norwitz
I’m not opposed to adding this kind of debugging functionality to the vhost-user-blk sample. It could be helpful to easily test these cases in the future. That said, I'm not sure how others will feel about adding these kind of debugging capabilities to libvhost-user. Marc-Andre, thoughts? If we

Re: [RFC PATCH v1 5/7] vhost-user-blk: add mechanism to track the guest notifiers init state

2020-04-23 Thread Raphael Norwitz
There are some problems with this patch. It doesn't apply cleanly. Are you sure you’re developing on an up to date master branch? On Thu, Apr 23, 2020 at 09:39:36PM +0300, Dima Stepanov wrote: > > In case of the vhost-user devices the daemon can be killed at any > moment. Since QEMU supports the

Re: [PATCH v2 2/5] vhost: introduce wrappers to set guest notifiers for virtio device

2020-05-03 Thread Raphael Norwitz
I’m happy from the vhost, vhost-user-blk and vhost-user-scsi side. For other device types it looks pretty straightforward, but their maintainers should probably confirm. Since you plan to change the behavior of these helpers in subsequent patches, maybe consider sending the other device types

Re: [PATCH v2 4/5] vhost: check vring address before calling unmap

2020-05-03 Thread Raphael Norwitz
to SIGSEGV. Add checks for the vring address value before calling unmap. > Also add assert() in the vhost_memory_unmap() routine. > > Signed-off-by: Dima Stepanov Reviewed-by: Raphael Norwitz > --- > hw/virtio/vhost.c | 27 +-- > 1 file changed, 21 inser

Re: [PATCH v2 3/5] vhost-user-blk: add mechanism to track the guest notifiers init state

2020-05-03 Thread Raphael Norwitz
Apologies for mixing up patches last time. This looks good from a vhost-user-blk perspective, but I worry that some of these changes could impact other vhost device types. I agree with adding notifiers_set to struct vhost_dev, and setting it in vhost_dev_enable/disable notifiers, but is there any

checkpatch error checking target arch in libvhost-user

2020-05-17 Thread Raphael Norwitz
Hey Marc-Andre, I'm working on a patchset with changes to libvhost-user. I'm hitting the following checkpatch error: Checking 0011-Lift-max-ram-slots-limit-in-libvhost-user.patch... WARNING: architecture specific defines should be avoided #117: FILE: contrib/libvhost-user/libvhost-user.h:38:

Re: [PATCH v2 5/5] vhost: add device started check in migration set log

2020-05-06 Thread Raphael Norwitz
As you correctly point out, this code needs to be looked at more carefully so that if the device does disconnect in the background we can handle the migration path gracefully. In particular, we need to decide whether a migration should be allowed to continue if a device disconnects durning the

Re: [PATCH v2 5/5] vhost: add device started check in migration set log

2020-05-10 Thread Raphael Norwitz
On Thu, May 7, 2020 at 11:35 AM Dima Stepanov wrote: > > What do you think? > Apologies - I tripped over the if (dev->started && r < 0) check. Never-mind my point with race conditions and failing migrations. Rather than modifying vhost_dev_set_log(), it may be clearer to put a check after

Re: checkpatch error checking target arch in libvhost-user

2020-05-18 Thread Raphael Norwitz
> Can we make it a run-time value instead? It's definitely possible, but will require a bit of refactoring. How about for now I set a reasonable maximum which is supported by all platforms (32)? That still increases it a lot from 8, and we can figure out how best increase it even more for other

[PATCH v3 03/10] Add VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS

2020-05-19 Thread Raphael Norwitz
is currently set to 8 because of underlying protocol limitations. The returned value will be stored inside the VhostUserState struct so that on device reconnect we can verify that the ram slot limitation has not decreased since the last time the device connected. Signed-off-by: Raphael Norwitz Signed-off

[PATCH v3 04/10] Transmit vhost-user memory regions individually

2020-05-19 Thread Raphael Norwitz
virtual device state. Existing backends which do not support VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS are unaffected. Signed-off-by: Raphael Norwitz Signed-off-by: Swapnil Ingle Signed-off-by: Peter Turschmid Suggested-by: Mike Cui --- docs/interop/vhost-user.rst | 33 ++- hw/virtio/vhost

[PATCH v3 02/10] Add vhost-user helper to get MemoryRegion data

2020-05-19 Thread Raphael Norwitz
. With VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS, this logic will be needed in multiple places, so before feature support is added it should be moved to a helper function. This helper is also used to simplify the vhost_user_can_merge() function. Signed-off-by: Raphael Norwitz --- hw/virtio/vhost-user.c | 25

[PATCH v3 06/10] Refactor out libvhost-user fault generation logic

2020-05-19 Thread Raphael Norwitz
. This logic will also be needed by the postcopy path once the VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS feature is supported. Signed-off-by: Raphael Norwitz --- contrib/libvhost-user/libvhost-user.c | 147 ++ 1 file changed, 79 insertions(+), 68 deletions(-) diff

[PATCH v3 08/10] Support adding individual regions in libvhost-user

2020-05-19 Thread Raphael Norwitz
messages are received. Qemu only sends VHOST_USER_ADD_MEM_REG messages when the VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS feature is negotiated, and since it is not yet supported in libvhost-user, this new functionality is not yet used. Signed-off-by: Raphael Norwitz --- contrib/libvhost-user

[PATCH v3 01/10] Add helper to populate vhost-user message regions

2020-05-19 Thread Raphael Norwitz
When setting vhost-user memory tables, memory region descriptors must be copied from the vhost_dev struct to the vhost-user message. To avoid duplicating code in setting the memory tables, we should use a helper to populate this field. This change adds this helper. Signed-off-by: Raphael Norwitz

[PATCH v3 00/10] vhost-user: Lift Max Ram Slots Limitation

2020-05-19 Thread Raphael Norwitz
a single VHOST_USER_GET_MAX_MEMSLOTS message at startup and cache the returned value so that QEMU does not need to query the backend every time vhost_backend_memslots_limit is called. Best, Raphael Raphael Norwitz (10): Add helper to populate vhost-user message regions Add vhost

[PATCH v3 05/10] Lift max memory slots limit imposed by vhost-user

2020-05-19 Thread Raphael Norwitz
backends which do not support VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS are unaffected. Signed-off-by: Raphael Norwitz Signed-off-by: Peter Turschmid Suggested-by: Mike Cui --- docs/interop/vhost-user.rst | 7 +++--- hw/virtio/vhost-user.c | 56

[PATCH v3 07/10] Support ram slot configuration in libvhost-user

2020-05-19 Thread Raphael Norwitz
-user does not yet support the vhost-user protocol feature VHOST_USER_PROTOCOL_F_CONFIGIRE_MEM_SLOTS, so qemu should never send the VHOST_USER_GET_MAX_MEM_SLOTS message. Therefore this new functionality is not currently used. Signed-off-by: Raphael Norwitz --- contrib/libvhost-user/libvhost-user.c

[PATCH v3 10/10] Lift max ram slots limit in libvhost-user

2020-05-19 Thread Raphael Norwitz
. Signed-off-by: Raphael Norwitz --- contrib/libvhost-user/libvhost-user.c | 17 + contrib/libvhost-user/libvhost-user.h | 15 +++ 2 files changed, 20 insertions(+), 12 deletions(-) diff --git a/contrib/libvhost-user/libvhost-user.c b/contrib/libvhost-user/libvhost

[PATCH v3 09/10] Support individual region unmap in libvhost-user

2020-05-19 Thread Raphael Norwitz
. Signed-off-by: Raphael Norwitz --- contrib/libvhost-user/libvhost-user.c | 63 +++ contrib/libvhost-user/libvhost-user.h | 1 + 2 files changed, 64 insertions(+) diff --git a/contrib/libvhost-user/libvhost-user.c b/contrib/libvhost-user/libvhost-user.c index

[PATCH] Fix erroneous double negation in conditional

2020-05-07 Thread Raphael Norwitz
In vhost_migration_log() there is the following check: if(!!enable == dev->log_enabled) { return 0; } The double negative “!!” is unnecessary and bad coding style. This change removes it. Signed-off-by: Raphael Norwitz --- hw/virtio/vhost.c | 2 +- 1 file changed, 1 insert

[PATCH v2] Fix parameter type in vhost migration log path

2020-05-07 Thread Raphael Norwitz
to bool. Signed-off-by: Raphael Norwitz --- hw/virtio/vhost.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index aff98a0..aa06a36 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -809,12 +809,12 @@ err_features: retur

[PATCH v4 01/10] Add helper to populate vhost-user message regions

2020-05-20 Thread Raphael Norwitz
When setting vhost-user memory tables, memory region descriptors must be copied from the vhost_dev struct to the vhost-user message. To avoid duplicating code in setting the memory tables, we should use a helper to populate this field. This change adds this helper. Signed-off-by: Raphael Norwitz

[PATCH v4 03/10] Add VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS

2020-05-20 Thread Raphael Norwitz
is currently set to 8 because of underlying protocol limitations. The returned value will be stored inside the VhostUserState struct so that on device reconnect we can verify that the ram slot limitation has not decreased since the last time the device connected. Signed-off-by: Raphael Norwitz Signed-off

[PATCH v4 04/10] Transmit vhost-user memory regions individually

2020-05-20 Thread Raphael Norwitz
virtual device state. Existing backends which do not support VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS are unaffected. Signed-off-by: Raphael Norwitz Signed-off-by: Swapnil Ingle Signed-off-by: Peter Turschmid Suggested-by: Mike Cui --- docs/interop/vhost-user.rst | 33 ++- hw/virtio/vhost

[PATCH v4 02/10] Add vhost-user helper to get MemoryRegion data

2020-05-20 Thread Raphael Norwitz
. With VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS, this logic will be needed in multiple places, so before feature support is added it should be moved to a helper function. This helper is also used to simplify the vhost_user_can_merge() function. Signed-off-by: Raphael Norwitz --- hw/virtio/vhost-user.c | 25

[PATCH v4 00/10] vhost-user: Lift Max Ram Slots Limitation

2020-05-20 Thread Raphael Norwitz
the backend every time vhost_backend_memslots_limit is called. Best, Raphael Raphael Norwitz (10): Add helper to populate vhost-user message regions Add vhost-user helper to get MemoryRegion data Add VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS Transmit vhost-user memory regions individually

[PATCH v4 06/10] Refactor out libvhost-user fault generation logic

2020-05-20 Thread Raphael Norwitz
. This logic will also be needed by the postcopy path once the VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS feature is supported. Signed-off-by: Raphael Norwitz --- contrib/libvhost-user/libvhost-user.c | 147 ++ 1 file changed, 79 insertions(+), 68 deletions(-) diff

[PATCH v4 08/10] Support adding individual regions in libvhost-user

2020-05-20 Thread Raphael Norwitz
messages are received. Qemu only sends VHOST_USER_ADD_MEM_REG messages when the VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS feature is negotiated, and since it is not yet supported in libvhost-user, this new functionality is not yet used. Signed-off-by: Raphael Norwitz --- contrib/libvhost-user

[PATCH v4 10/10] Lift max ram slots limit in libvhost-user

2020-05-20 Thread Raphael Norwitz
. Signed-off-by: Raphael Norwitz --- contrib/libvhost-user/libvhost-user.c | 17 + contrib/libvhost-user/libvhost-user.h | 15 +++ 2 files changed, 20 insertions(+), 12 deletions(-) diff --git a/contrib/libvhost-user/libvhost-user.c b/contrib/libvhost-user/libvhost

[PATCH v4 09/10] Support individual region unmap in libvhost-user

2020-05-20 Thread Raphael Norwitz
. Signed-off-by: Raphael Norwitz --- contrib/libvhost-user/libvhost-user.c | 63 +++ contrib/libvhost-user/libvhost-user.h | 1 + 2 files changed, 64 insertions(+) diff --git a/contrib/libvhost-user/libvhost-user.c b/contrib/libvhost-user/libvhost-user.c index

[PATCH v4 05/10] Lift max memory slots limit imposed by vhost-user

2020-05-20 Thread Raphael Norwitz
backends which do not support VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS are unaffected. Signed-off-by: Raphael Norwitz Signed-off-by: Peter Turschmid Suggested-by: Mike Cui --- docs/interop/vhost-user.rst | 7 +++--- hw/virtio/vhost-user.c | 56

[PATCH v4 07/10] Support ram slot configuration in libvhost-user

2020-05-20 Thread Raphael Norwitz
-user does not yet support the vhost-user protocol feature VHOST_USER_PROTOCOL_F_CONFIGIRE_MEM_SLOTS, so qemu should never send the VHOST_USER_GET_MAX_MEM_SLOTS message. Therefore this new functionality is not currently used. Signed-off-by: Raphael Norwitz --- contrib/libvhost-user/libvhost-user.c

Re: [PATCH 3/5] vhost-user-blk: add VIRTIO_F_RING_PACKED feature bit

2020-05-24 Thread Raphael Norwitz
device backend can declare whether or not it supports the packed > ring layout. > > Signed-off-by: Stefan Hajnoczi Reviewed-by: Raphael Norwitz > --- > hw/block/vhost-user-blk.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/hw/block/vhost-user-blk.c b/hw/bl

Re: [PATCH 4/5] vhost-scsi: add VIRTIO_F_VERSION_1 and VIRTIO_F_RING_PACKED

2020-05-24 Thread Raphael Norwitz
port them if we want to use them. > > Signed-off-by: Stefan Hajnoczi Reviewed-by: Raphael Norwitz > --- > hw/scsi/vhost-scsi.c | 2 ++ > hw/scsi/vhost-user-scsi.c | 2 ++ > 2 files changed, 4 insertions(+) > > diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.

Re: [PATCH v3 2/2] vhost-user-blk: delay vhost_user_blk_disconnect

2020-05-24 Thread Raphael Norwitz
I'm mostly happy with this. A couple comments. On Wed, May 20, 2020 at 11:54 AM Dima Stepanov wrote: > > A socket write during vhost-user communication may trigger a disconnect > event, calling vhost_user_blk_disconnect() and clearing all the > vhost_dev structures holding data that vhost-user

Re: [PATCH v3 2/2] vhost-user-blk: delay vhost_user_blk_disconnect

2020-05-25 Thread Raphael Norwitz
On Mon, May 25, 2020 at 4:58 AM Dima Stepanov wrote: > > On Wed, May 20, 2020 at 06:53:13PM +0300, Dima Stepanov wrote: > > A socket write during vhost-user communication may trigger a disconnect > > event, calling vhost_user_blk_disconnect() and clearing all the > > vhost_dev structures holding

Re: [PATCH v4 2/2] vhost-user-blk: delay vhost_user_blk_disconnect

2020-05-30 Thread Raphael Norwitz
BH changes are based on the similar changes for the vhost-user-net > device: > commit e7c83a885f865128ae3cf1946f8cb538b63cbfba > "vhost-user: delay vhost_user_stop" > > Signed-off-by: Dima Stepanov Reviewed-by: Raphael Norwitz Li Feng - would you also li

Re: [PATCH v4 5/5] vhost-user-blk: default num_queues to -smp N

2020-05-30 Thread Raphael Norwitz
I'm happy with the code but as David pointed out with virtio-scsi, we should probably add a comment about virtio_pci_optimal_num_queues() capping the number of VQs here too. On Wed, May 27, 2020 at 6:34 AM Stefan Hajnoczi wrote: > > Automatically size the number of request virtqueues to match

Re: [PATCH v4 2/5] virtio-scsi: introduce a constant for fixed virtqueues

2020-05-30 Thread Raphael Norwitz
S(_dev->bus)); > diff --git a/hw/virtio/virtio-scsi-pci.c b/hw/virtio/virtio-scsi-pci.c > index e82e7e5680..c52d68053a 100644 > --- a/hw/virtio/virtio-scsi-pci.c > +++ b/hw/virtio/virtio-scsi-pci.c > @@ -51,7 +51,8 @@ static void virtio_scsi_pci_realize(VirtIOPCIProxy > *vpci_dev, Error **errp) > char *bus_name; > > if (vpci_dev->nvectors == DEV_NVECTORS_UNSPECIFIED) { > -vpci_dev->nvectors = vs->conf.num_queues + 3; > +vpci_dev->nvectors = vs->conf.num_queues + > + VIRTIO_SCSI_VQ_NUM_FIXED + 1; > } > > /* > -- > 2.25.4 > Reviewed-by: Raphael Norwitz

[PATCH] Fix vhost-user buffer over-read on ram hot-unplug

2020-07-16 Thread Raphael Norwitz
region, Qemu would read an out of bounds value and potentially crash. This change fixes the memmove() bounds such that this erroneous read can never happen. Signed-off-by: Peter Turschmid Signed-off-by: Raphael Norwitz --- hw/virtio/vhost-user.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion

Re: Adding VHOST_USER_PROTOCOL_F_CONFIG_MEM_SLOTS to 5.1 release notes

2020-07-29 Thread Raphael Norwitz
> On Tue, Jul 28, 2020 at 09:16:10PM -0600, Raphael Norwitz wrote: > > Hi mst, > > > > Looking at the current changelog > > https://wiki.qemu.org/ChangeLog/5.1#virtio, I don't see any mention of > > the VHOST_USER_PROTOCOL_F_CONFIG_MEM_SLOTS protocol feature. It is

Re: Adding VHOST_USER_PROTOCOL_F_CONFIG_MEM_SLOTS to 5.1 release notes

2020-08-05 Thread Raphael Norwitz
ping On Wed, Jul 29, 2020 at 9:17 PM Raphael Norwitz wrote: > > How about something like: > "A new feature, VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS, has been > added to the vhost-user protocol which, when negotiated, changes the > way QEMU transmit memory regions to back

Re: [PATCH 05/11] hw/virtio/vhost-user:Remove dead assignment in scrub_shadow_regions()

2020-08-13 Thread Raphael Norwitz
;userspace_addr, , ); > ^~ > > Reported-by: Euler Robot > Signed-off-by: Chen Qun > --- > Cc: "Michael S. Tsirkin" > Cc: Raphael Norwitz > --- > hw/virtio/vhost-user.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/hw/vi

Re: [PATCH v5 5/5] vhost-user-blk: default num_queues to -smp N

2020-07-09 Thread Raphael Norwitz
On Mon, Jul 6, 2020 at 7:00 AM Stefan Hajnoczi wrote: > > Automatically size the number of request virtqueues to match the number > of vCPUs. This ensures that completion interrupts are handled on the > same vCPU that submitted the request. No IPI is necessary to complete > an I/O request and

Re: [PATCH v5 5/5] vhost-user-blk: default num_queues to -smp N

2020-07-10 Thread Raphael Norwitz
On Fri, Jul 10, 2020 at 5:53 AM Stefan Hajnoczi wrote: > > On Thu, Jul 09, 2020 at 11:02:24AM -0700, Raphael Norwitz wrote: > > On Mon, Jul 6, 2020 at 7:00 AM Stefan Hajnoczi wrote: > > > diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c > > > index

[PATCH] Stop vhost-user sending uninitialized mmap_offsets

2020-06-22 Thread Raphael Norwitz
mmap_offset. Fixes: CID-1429802, CID-1429803, CID-1429804 Suggested-by: Peter Maydell Signed-off-by: Raphael Norwitz --- hw/virtio/vhost-user.c | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 4d6cd4e

[PATCH v2] Stop vhost-user sending uninitialized mmap_offsets

2020-06-22 Thread Raphael Norwitz
mmap_offset. Fixes: ece99091c2d0aeb23734289a50ef2ff4e0a08929 Fixes: f1aeb14b0809e313c74244d838645ed25e85ea63 Reported-by: Coverity (CIDs 1429802, 1429803 and 1429804) Suggested-by: Peter Maydell Signed-off-by: Raphael Norwitz --- hw/virtio/vhost-user.c | 12 ++-- 1 file changed, 6

Re: [PULL v2 39/58] Transmit vhost-user memory regions individually

2020-06-22 Thread Raphael Norwitz
On Fri, Jun 19, 2020 at 6:03 AM Peter Maydell wrote: > > > As noted in my other email, I think the best fix for this is to > have vhost_user_fill_msg_region() take an extra mmap_offset > argument to fill in the mmap_offset itself. In this callsite in > send_add_regions() we would pass in 'offset'

Re: [PATCH v4 04/10] Transmit vhost-user memory regions individually

2020-06-09 Thread Raphael Norwitz
On Thu, Jun 4, 2020 at 10:45 AM Marc-André Lureau wrote: > > Hi > > On Thu, May 21, 2020 at 7:00 AM Raphael Norwitz > wrote: >> >> With this change, when the VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS >> protocol feature has been negotiated, Qemu no longer se

Re: [PATCH v4 00/10] vhost-user: Lift Max Ram Slots Limitation

2020-06-03 Thread Raphael Norwitz
ping On Thu, May 21, 2020 at 1:00 AM Raphael Norwitz wrote: > > In QEMU today, a VM with a vhost-user device can hot add memory a > maximum of 8 times. See these threads, among others: > > [1] https://lists.gnu.org/archive/html/qemu-devel/2019-07/msg01046.html > https://lis

Re: [PATCH] Fix vhost-user buffer over-read on ram hot-unplug

2020-07-22 Thread Raphael Norwitz
ping On Thu, Jul 16, 2020 at 10:21 PM Raphael Norwitz wrote: > > The VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS vhost-user protocol > feature introduced a shadow-table, used by the backend to dynamically > determine how a vdev's memory regions have changed si

  1   2   >