Re: vdpa legacy guest support (was Re: [PATCH] vdpa/mlx5: set_features should allow reset to zero)
On Fri, Dec 17, 2021 at 10:01 AM Michael S. Tsirkin wrote: > > On Fri, Dec 17, 2021 at 09:57:38AM +0800, Jason Wang wrote: > > On Fri, Dec 17, 2021 at 6:32 AM Si-Wei Liu wrote: > > > > > > > > > > > > On 12/15/2021 6:53 PM, Jason Wang wrote: > > > > On Thu, Dec 16, 2021 at 10:02 AM Si-Wei Liu > > > > wrote: > > > >> > > > >> > > > >> On 12/15/2021 1:33 PM, Michael S. Tsirkin wrote: > > > >>> On Wed, Dec 15, 2021 at 12:52:20PM -0800, Si-Wei Liu wrote: > > > On 12/14/2021 6:06 PM, Jason Wang wrote: > > > > On Wed, Dec 15, 2021 at 9:05 AM Si-Wei Liu > > > > wrote: > > > >> On 12/13/2021 9:06 PM, Michael S. Tsirkin wrote: > > > >>> On Mon, Dec 13, 2021 at 05:59:45PM -0800, Si-Wei Liu wrote: > > > On 12/12/2021 1:26 AM, Michael S. Tsirkin wrote: > > > > On Fri, Dec 10, 2021 at 05:44:15PM -0800, Si-Wei Liu wrote: > > > >> Sorry for reviving this ancient thread. I was kinda lost for > > > >> the conclusion > > > >> it ended up with. I have the following questions, > > > >> > > > >> 1. legacy guest support: from the past conversations it > > > >> doesn't seem the > > > >> support will be completely dropped from the table, is my > > > >> understanding > > > >> correct? Actually we're interested in supporting virtio v0.95 > > > >> guest for x86, > > > >> which is backed by the spec at > > > >> https://urldefense.com/v3/__https://ozlabs.org/*rusty/virtio-spec/virtio-0.9.5.pdf__;fg!!ACWV5N9M2RV99hQ!dTKmzJwwRsFM7BtSuTDu1cNly5n4XCotH0WYmidzGqHSXt40i7ZU43UcNg7GYxZg$ > > > >> . Though I'm not sure > > > >> if there's request/need to support wilder legacy virtio > > > >> versions earlier > > > >> beyond. > > > > I personally feel it's less work to add in kernel than try to > > > > work around it in userspace. Jason feels differently. > > > > Maybe post the patches and this will prove to Jason it's not > > > > too terrible? > > > I suppose if the vdpa vendor does support 0.95 in the datapath > > > and ring > > > layout level and is limited to x86 only, there should be easy > > > way out. > > > >>> Note a subtle difference: what matters is that guest, not host is > > > >>> x86. > > > >>> Matters for emulators which might reorder memory accesses. > > > >>> I guess this enforcement belongs in QEMU then? > > > >> Right, I mean to get started, the initial guest driver support and > > > >> the > > > >> corresponding QEMU support for transitional vdpa backend can be > > > >> limited > > > >> to x86 guest/host only. Since the config space is emulated in > > > >> QEMU, I > > > >> suppose it's not hard to enforce in QEMU. > > > > It's more than just config space, most devices have headers before > > > > the buffer. > > > The ordering in datapath (data VQs) would have to rely on vendor's > > > support. > > > Since ORDER_PLATFORM is pretty new (v1.1), I guess vdpa h/w vendor > > > nowadays > > > can/should well support the case when ORDER_PLATFORM is not acked by > > > the > > > driver (actually this feature is filtered out by the QEMU vhost-vdpa > > > driver > > > today), even with v1.0 spec conforming and modern only vDPA device. > > > The > > > control VQ is implemented in software in the kernel, which can be > > > easily > > > accommodated/fixed when needed. > > > > > > >> QEMU can drive GET_LEGACY, > > > >> GET_ENDIAN et al ioctls in advance to get the capability from the > > > >> individual vendor driver. For that, we need another negotiation > > > >> protocol > > > >> similar to vhost_user's protocol_features between the vdpa kernel > > > >> and > > > >> QEMU, way before the guest driver is ever probed and its feature > > > >> negotiation kicks in. Not sure we need a GET_MEMORY_ORDER ioctl > > > >> call > > > >> from the device, but we can assume weak ordering for legacy at this > > > >> point (x86 only)? > > > > I'm lost here, we have get_features() so: > > > I assume here you refer to get_device_features() that Eli just > > > changed the > > > name. > > > > 1) VERSION_1 means the device uses LE if provided, otherwise natvie > > > > 2) ORDER_PLATFORM means device requires platform ordering > > > > > > > > Any reason for having a new API for this? > > > Are you going to enforce all vDPA hardware vendors to support the > > > transitional model for legacy guest? > > > > Do we really have other choices? > > > > > > > > I suspect the legacy device is never implemented by any vendor: > > > > > > > > 1) no virtio way to detect host endian > > > This is even true for transitional device that is conforming to the > > > spec, right? > > > > For hardware, yes. > > > > >
Re: vdpa legacy guest support (was Re: [PATCH] vdpa/mlx5: set_features should allow reset to zero)
On Fri, Dec 17, 2021 at 9:08 AM Si-Wei Liu wrote: > > > > On 12/15/2021 7:43 PM, Jason Wang wrote: > > On Thu, Dec 16, 2021 at 4:52 AM Si-Wei Liu wrote: > >> > >> > >> On 12/14/2021 6:06 PM, Jason Wang wrote: > >>> On Wed, Dec 15, 2021 at 9:05 AM Si-Wei Liu wrote: > > On 12/13/2021 9:06 PM, Michael S. Tsirkin wrote: > > On Mon, Dec 13, 2021 at 05:59:45PM -0800, Si-Wei Liu wrote: > >> On 12/12/2021 1:26 AM, Michael S. Tsirkin wrote: > >>> On Fri, Dec 10, 2021 at 05:44:15PM -0800, Si-Wei Liu wrote: > Sorry for reviving this ancient thread. I was kinda lost for the > conclusion > it ended up with. I have the following questions, > > 1. legacy guest support: from the past conversations it doesn't seem > the > support will be completely dropped from the table, is my > understanding > correct? Actually we're interested in supporting virtio v0.95 guest > for x86, > which is backed by the spec at > https://urldefense.com/v3/__https://ozlabs.org/*rusty/virtio-spec/virtio-0.9.5.pdf__;fg!!ACWV5N9M2RV99hQ!dTKmzJwwRsFM7BtSuTDu1cNly5n4XCotH0WYmidzGqHSXt40i7ZU43UcNg7GYxZg$ > . Though I'm not sure > if there's request/need to support wilder legacy virtio versions > earlier > beyond. > >>> I personally feel it's less work to add in kernel than try to > >>> work around it in userspace. Jason feels differently. > >>> Maybe post the patches and this will prove to Jason it's not > >>> too terrible? > >> I suppose if the vdpa vendor does support 0.95 in the datapath and ring > >> layout level and is limited to x86 only, there should be easy way out. > > Note a subtle difference: what matters is that guest, not host is x86. > > Matters for emulators which might reorder memory accesses. > > I guess this enforcement belongs in QEMU then? > Right, I mean to get started, the initial guest driver support and the > corresponding QEMU support for transitional vdpa backend can be limited > to x86 guest/host only. Since the config space is emulated in QEMU, I > suppose it's not hard to enforce in QEMU. > >>> It's more than just config space, most devices have headers before the > >>> buffer. > >> The ordering in datapath (data VQs) would have to rely on vendor's > >> support. Since ORDER_PLATFORM is pretty new (v1.1), I guess vdpa h/w > >> vendor nowadays can/should well support the case when ORDER_PLATFORM is > >> not acked by the driver (actually this feature is filtered out by the > >> QEMU vhost-vdpa driver today), even with v1.0 spec conforming and modern > >> only vDPA device. > > That's a bug that needs to be fixed. > > > >> The control VQ is implemented in software in the > >> kernel, which can be easily accommodated/fixed when needed. > >> > QEMU can drive GET_LEGACY, > GET_ENDIAN et al ioctls in advance to get the capability from the > individual vendor driver. For that, we need another negotiation protocol > similar to vhost_user's protocol_features between the vdpa kernel and > QEMU, way before the guest driver is ever probed and its feature > negotiation kicks in. Not sure we need a GET_MEMORY_ORDER ioctl call > from the device, but we can assume weak ordering for legacy at this > point (x86 only)? > >>> I'm lost here, we have get_features() so: > >> I assume here you refer to get_device_features() that Eli just changed > >> the name. > >>> 1) VERSION_1 means the device uses LE if provided, otherwise natvie > >>> 2) ORDER_PLATFORM means device requires platform ordering > >>> > >>> Any reason for having a new API for this? > >> Are you going to enforce all vDPA hardware vendors to support the > >> transitional model for legacy guest? meaning guest not acknowledging > >> VERSION_1 would use the legacy interfaces captured in the spec section > >> 7.4 (regarding ring layout, native endianness, message framing, vq > >> alignment of 4096, 32bit feature, no features_ok bit in status, IO port > >> interface i.e. all the things) instead? Noted we don't yet have a > >> set_device_features() that allows the vdpa device to tell whether it is > >> operating in transitional or modern-only mode. For software virtio, all > >> support for the legacy part in a transitional model has been built up > >> there already, however, it's not easy for vDPA vendors to implement all > >> the requirements for an all-or-nothing legacy guest support (big endian > >> guest for example). To these vendors, the legacy support within a > >> transitional model is more of feature to them and it's best to leave > >> some flexibility for them to implement partial support for legacy. That > >> in turn calls out the need for a vhost-user protocol feature like > >> negotiation API that can prohibit those unsupported guest setups to as > >> early as backend_init before launching the VM. > >> > >> >
Re: vdpa legacy guest support (was Re: [PATCH] vdpa/mlx5: set_features should allow reset to zero)
On Fri, Dec 17, 2021 at 09:57:38AM +0800, Jason Wang wrote: > On Fri, Dec 17, 2021 at 6:32 AM Si-Wei Liu wrote: > > > > > > > > On 12/15/2021 6:53 PM, Jason Wang wrote: > > > On Thu, Dec 16, 2021 at 10:02 AM Si-Wei Liu wrote: > > >> > > >> > > >> On 12/15/2021 1:33 PM, Michael S. Tsirkin wrote: > > >>> On Wed, Dec 15, 2021 at 12:52:20PM -0800, Si-Wei Liu wrote: > > On 12/14/2021 6:06 PM, Jason Wang wrote: > > > On Wed, Dec 15, 2021 at 9:05 AM Si-Wei Liu > > > wrote: > > >> On 12/13/2021 9:06 PM, Michael S. Tsirkin wrote: > > >>> On Mon, Dec 13, 2021 at 05:59:45PM -0800, Si-Wei Liu wrote: > > On 12/12/2021 1:26 AM, Michael S. Tsirkin wrote: > > > On Fri, Dec 10, 2021 at 05:44:15PM -0800, Si-Wei Liu wrote: > > >> Sorry for reviving this ancient thread. I was kinda lost for the > > >> conclusion > > >> it ended up with. I have the following questions, > > >> > > >> 1. legacy guest support: from the past conversations it doesn't > > >> seem the > > >> support will be completely dropped from the table, is my > > >> understanding > > >> correct? Actually we're interested in supporting virtio v0.95 > > >> guest for x86, > > >> which is backed by the spec at > > >> https://urldefense.com/v3/__https://ozlabs.org/*rusty/virtio-spec/virtio-0.9.5.pdf__;fg!!ACWV5N9M2RV99hQ!dTKmzJwwRsFM7BtSuTDu1cNly5n4XCotH0WYmidzGqHSXt40i7ZU43UcNg7GYxZg$ > > >> . Though I'm not sure > > >> if there's request/need to support wilder legacy virtio versions > > >> earlier > > >> beyond. > > > I personally feel it's less work to add in kernel than try to > > > work around it in userspace. Jason feels differently. > > > Maybe post the patches and this will prove to Jason it's not > > > too terrible? > > I suppose if the vdpa vendor does support 0.95 in the datapath and > > ring > > layout level and is limited to x86 only, there should be easy way > > out. > > >>> Note a subtle difference: what matters is that guest, not host is > > >>> x86. > > >>> Matters for emulators which might reorder memory accesses. > > >>> I guess this enforcement belongs in QEMU then? > > >> Right, I mean to get started, the initial guest driver support and > > >> the > > >> corresponding QEMU support for transitional vdpa backend can be > > >> limited > > >> to x86 guest/host only. Since the config space is emulated in QEMU, I > > >> suppose it's not hard to enforce in QEMU. > > > It's more than just config space, most devices have headers before > > > the buffer. > > The ordering in datapath (data VQs) would have to rely on vendor's > > support. > > Since ORDER_PLATFORM is pretty new (v1.1), I guess vdpa h/w vendor > > nowadays > > can/should well support the case when ORDER_PLATFORM is not acked by > > the > > driver (actually this feature is filtered out by the QEMU vhost-vdpa > > driver > > today), even with v1.0 spec conforming and modern only vDPA device. The > > control VQ is implemented in software in the kernel, which can be > > easily > > accommodated/fixed when needed. > > > > >> QEMU can drive GET_LEGACY, > > >> GET_ENDIAN et al ioctls in advance to get the capability from the > > >> individual vendor driver. For that, we need another negotiation > > >> protocol > > >> similar to vhost_user's protocol_features between the vdpa kernel and > > >> QEMU, way before the guest driver is ever probed and its feature > > >> negotiation kicks in. Not sure we need a GET_MEMORY_ORDER ioctl call > > >> from the device, but we can assume weak ordering for legacy at this > > >> point (x86 only)? > > > I'm lost here, we have get_features() so: > > I assume here you refer to get_device_features() that Eli just changed > > the > > name. > > > 1) VERSION_1 means the device uses LE if provided, otherwise natvie > > > 2) ORDER_PLATFORM means device requires platform ordering > > > > > > Any reason for having a new API for this? > > Are you going to enforce all vDPA hardware vendors to support the > > transitional model for legacy guest? > > > Do we really have other choices? > > > > > > I suspect the legacy device is never implemented by any vendor: > > > > > > 1) no virtio way to detect host endian > > This is even true for transitional device that is conforming to the > > spec, right? > > For hardware, yes. > > > The transport specific way to detect host endian is still > > being discussed and the spec revision is not finalized yet so far as I > > see. Why this suddenly becomes a requirement/blocker for h/w vendors to > > implement the transitional model? > > It's not a sudden blocker, the problem has existed since day 0 if I
Re: vdpa legacy guest support (was Re: [PATCH] vdpa/mlx5: set_features should allow reset to zero)
On Fri, Dec 17, 2021 at 6:32 AM Si-Wei Liu wrote: > > > > On 12/15/2021 6:53 PM, Jason Wang wrote: > > On Thu, Dec 16, 2021 at 10:02 AM Si-Wei Liu wrote: > >> > >> > >> On 12/15/2021 1:33 PM, Michael S. Tsirkin wrote: > >>> On Wed, Dec 15, 2021 at 12:52:20PM -0800, Si-Wei Liu wrote: > On 12/14/2021 6:06 PM, Jason Wang wrote: > > On Wed, Dec 15, 2021 at 9:05 AM Si-Wei Liu > > wrote: > >> On 12/13/2021 9:06 PM, Michael S. Tsirkin wrote: > >>> On Mon, Dec 13, 2021 at 05:59:45PM -0800, Si-Wei Liu wrote: > On 12/12/2021 1:26 AM, Michael S. Tsirkin wrote: > > On Fri, Dec 10, 2021 at 05:44:15PM -0800, Si-Wei Liu wrote: > >> Sorry for reviving this ancient thread. I was kinda lost for the > >> conclusion > >> it ended up with. I have the following questions, > >> > >> 1. legacy guest support: from the past conversations it doesn't > >> seem the > >> support will be completely dropped from the table, is my > >> understanding > >> correct? Actually we're interested in supporting virtio v0.95 > >> guest for x86, > >> which is backed by the spec at > >> https://urldefense.com/v3/__https://ozlabs.org/*rusty/virtio-spec/virtio-0.9.5.pdf__;fg!!ACWV5N9M2RV99hQ!dTKmzJwwRsFM7BtSuTDu1cNly5n4XCotH0WYmidzGqHSXt40i7ZU43UcNg7GYxZg$ > >> . Though I'm not sure > >> if there's request/need to support wilder legacy virtio versions > >> earlier > >> beyond. > > I personally feel it's less work to add in kernel than try to > > work around it in userspace. Jason feels differently. > > Maybe post the patches and this will prove to Jason it's not > > too terrible? > I suppose if the vdpa vendor does support 0.95 in the datapath and > ring > layout level and is limited to x86 only, there should be easy way > out. > >>> Note a subtle difference: what matters is that guest, not host is x86. > >>> Matters for emulators which might reorder memory accesses. > >>> I guess this enforcement belongs in QEMU then? > >> Right, I mean to get started, the initial guest driver support and the > >> corresponding QEMU support for transitional vdpa backend can be limited > >> to x86 guest/host only. Since the config space is emulated in QEMU, I > >> suppose it's not hard to enforce in QEMU. > > It's more than just config space, most devices have headers before the > > buffer. > The ordering in datapath (data VQs) would have to rely on vendor's > support. > Since ORDER_PLATFORM is pretty new (v1.1), I guess vdpa h/w vendor > nowadays > can/should well support the case when ORDER_PLATFORM is not acked by the > driver (actually this feature is filtered out by the QEMU vhost-vdpa > driver > today), even with v1.0 spec conforming and modern only vDPA device. The > control VQ is implemented in software in the kernel, which can be easily > accommodated/fixed when needed. > > >> QEMU can drive GET_LEGACY, > >> GET_ENDIAN et al ioctls in advance to get the capability from the > >> individual vendor driver. For that, we need another negotiation > >> protocol > >> similar to vhost_user's protocol_features between the vdpa kernel and > >> QEMU, way before the guest driver is ever probed and its feature > >> negotiation kicks in. Not sure we need a GET_MEMORY_ORDER ioctl call > >> from the device, but we can assume weak ordering for legacy at this > >> point (x86 only)? > > I'm lost here, we have get_features() so: > I assume here you refer to get_device_features() that Eli just changed > the > name. > > 1) VERSION_1 means the device uses LE if provided, otherwise natvie > > 2) ORDER_PLATFORM means device requires platform ordering > > > > Any reason for having a new API for this? > Are you going to enforce all vDPA hardware vendors to support the > transitional model for legacy guest? > > Do we really have other choices? > > > > I suspect the legacy device is never implemented by any vendor: > > > > 1) no virtio way to detect host endian > This is even true for transitional device that is conforming to the > spec, right? For hardware, yes. > The transport specific way to detect host endian is still > being discussed and the spec revision is not finalized yet so far as I > see. Why this suddenly becomes a requirement/blocker for h/w vendors to > implement the transitional model? It's not a sudden blocker, the problem has existed since day 0 if I was not wrong. That's why the problem looks a little bit complicated and why it would be much simpler if we stick to modern devices. > Even if the spec is out, this is > pretty new and I suspect not all vendor would follow right away. I hope > the software framework can be tolerant with h/w vendors not
Re: vdpa legacy guest support (was Re: [PATCH] vdpa/mlx5: set_features should allow reset to zero)
On 12/15/2021 7:43 PM, Jason Wang wrote: On Thu, Dec 16, 2021 at 4:52 AM Si-Wei Liu wrote: On 12/14/2021 6:06 PM, Jason Wang wrote: On Wed, Dec 15, 2021 at 9:05 AM Si-Wei Liu wrote: On 12/13/2021 9:06 PM, Michael S. Tsirkin wrote: On Mon, Dec 13, 2021 at 05:59:45PM -0800, Si-Wei Liu wrote: On 12/12/2021 1:26 AM, Michael S. Tsirkin wrote: On Fri, Dec 10, 2021 at 05:44:15PM -0800, Si-Wei Liu wrote: Sorry for reviving this ancient thread. I was kinda lost for the conclusion it ended up with. I have the following questions, 1. legacy guest support: from the past conversations it doesn't seem the support will be completely dropped from the table, is my understanding correct? Actually we're interested in supporting virtio v0.95 guest for x86, which is backed by the spec at https://urldefense.com/v3/__https://ozlabs.org/*rusty/virtio-spec/virtio-0.9.5.pdf__;fg!!ACWV5N9M2RV99hQ!dTKmzJwwRsFM7BtSuTDu1cNly5n4XCotH0WYmidzGqHSXt40i7ZU43UcNg7GYxZg$ . Though I'm not sure if there's request/need to support wilder legacy virtio versions earlier beyond. I personally feel it's less work to add in kernel than try to work around it in userspace. Jason feels differently. Maybe post the patches and this will prove to Jason it's not too terrible? I suppose if the vdpa vendor does support 0.95 in the datapath and ring layout level and is limited to x86 only, there should be easy way out. Note a subtle difference: what matters is that guest, not host is x86. Matters for emulators which might reorder memory accesses. I guess this enforcement belongs in QEMU then? Right, I mean to get started, the initial guest driver support and the corresponding QEMU support for transitional vdpa backend can be limited to x86 guest/host only. Since the config space is emulated in QEMU, I suppose it's not hard to enforce in QEMU. It's more than just config space, most devices have headers before the buffer. The ordering in datapath (data VQs) would have to rely on vendor's support. Since ORDER_PLATFORM is pretty new (v1.1), I guess vdpa h/w vendor nowadays can/should well support the case when ORDER_PLATFORM is not acked by the driver (actually this feature is filtered out by the QEMU vhost-vdpa driver today), even with v1.0 spec conforming and modern only vDPA device. That's a bug that needs to be fixed. The control VQ is implemented in software in the kernel, which can be easily accommodated/fixed when needed. QEMU can drive GET_LEGACY, GET_ENDIAN et al ioctls in advance to get the capability from the individual vendor driver. For that, we need another negotiation protocol similar to vhost_user's protocol_features between the vdpa kernel and QEMU, way before the guest driver is ever probed and its feature negotiation kicks in. Not sure we need a GET_MEMORY_ORDER ioctl call from the device, but we can assume weak ordering for legacy at this point (x86 only)? I'm lost here, we have get_features() so: I assume here you refer to get_device_features() that Eli just changed the name. 1) VERSION_1 means the device uses LE if provided, otherwise natvie 2) ORDER_PLATFORM means device requires platform ordering Any reason for having a new API for this? Are you going to enforce all vDPA hardware vendors to support the transitional model for legacy guest? meaning guest not acknowledging VERSION_1 would use the legacy interfaces captured in the spec section 7.4 (regarding ring layout, native endianness, message framing, vq alignment of 4096, 32bit feature, no features_ok bit in status, IO port interface i.e. all the things) instead? Noted we don't yet have a set_device_features() that allows the vdpa device to tell whether it is operating in transitional or modern-only mode. For software virtio, all support for the legacy part in a transitional model has been built up there already, however, it's not easy for vDPA vendors to implement all the requirements for an all-or-nothing legacy guest support (big endian guest for example). To these vendors, the legacy support within a transitional model is more of feature to them and it's best to leave some flexibility for them to implement partial support for legacy. That in turn calls out the need for a vhost-user protocol feature like negotiation API that can prohibit those unsupported guest setups to as early as backend_init before launching the VM. I checked with Eli and other Mellanox/NVDIA folks for hardware/firmware level 0.95 support, it seems all the ingredient had been there already dated back to the DPDK days. The only major thing limiting is in the vDPA software that the current vdpa core has the assumption around VIRTIO_F_ACCESS_PLATFORM for a few DMA setup ops, which is virtio 1.0 only. 2. suppose some form of legacy guest support needs to be there, how do we deal with the bogus assumption below in vdpa_get_config() in the short term? It looks one of the intuitive fix is to move the vdpa_set_features call out of vdpa_get_config() to vdpa_set_config(). /*
Re: vdpa legacy guest support (was Re: [PATCH] vdpa/mlx5: set_features should allow reset to zero)
On 12/15/2021 6:53 PM, Jason Wang wrote: On Thu, Dec 16, 2021 at 10:02 AM Si-Wei Liu wrote: On 12/15/2021 1:33 PM, Michael S. Tsirkin wrote: On Wed, Dec 15, 2021 at 12:52:20PM -0800, Si-Wei Liu wrote: On 12/14/2021 6:06 PM, Jason Wang wrote: On Wed, Dec 15, 2021 at 9:05 AM Si-Wei Liu wrote: On 12/13/2021 9:06 PM, Michael S. Tsirkin wrote: On Mon, Dec 13, 2021 at 05:59:45PM -0800, Si-Wei Liu wrote: On 12/12/2021 1:26 AM, Michael S. Tsirkin wrote: On Fri, Dec 10, 2021 at 05:44:15PM -0800, Si-Wei Liu wrote: Sorry for reviving this ancient thread. I was kinda lost for the conclusion it ended up with. I have the following questions, 1. legacy guest support: from the past conversations it doesn't seem the support will be completely dropped from the table, is my understanding correct? Actually we're interested in supporting virtio v0.95 guest for x86, which is backed by the spec at https://urldefense.com/v3/__https://ozlabs.org/*rusty/virtio-spec/virtio-0.9.5.pdf__;fg!!ACWV5N9M2RV99hQ!dTKmzJwwRsFM7BtSuTDu1cNly5n4XCotH0WYmidzGqHSXt40i7ZU43UcNg7GYxZg$ . Though I'm not sure if there's request/need to support wilder legacy virtio versions earlier beyond. I personally feel it's less work to add in kernel than try to work around it in userspace. Jason feels differently. Maybe post the patches and this will prove to Jason it's not too terrible? I suppose if the vdpa vendor does support 0.95 in the datapath and ring layout level and is limited to x86 only, there should be easy way out. Note a subtle difference: what matters is that guest, not host is x86. Matters for emulators which might reorder memory accesses. I guess this enforcement belongs in QEMU then? Right, I mean to get started, the initial guest driver support and the corresponding QEMU support for transitional vdpa backend can be limited to x86 guest/host only. Since the config space is emulated in QEMU, I suppose it's not hard to enforce in QEMU. It's more than just config space, most devices have headers before the buffer. The ordering in datapath (data VQs) would have to rely on vendor's support. Since ORDER_PLATFORM is pretty new (v1.1), I guess vdpa h/w vendor nowadays can/should well support the case when ORDER_PLATFORM is not acked by the driver (actually this feature is filtered out by the QEMU vhost-vdpa driver today), even with v1.0 spec conforming and modern only vDPA device. The control VQ is implemented in software in the kernel, which can be easily accommodated/fixed when needed. QEMU can drive GET_LEGACY, GET_ENDIAN et al ioctls in advance to get the capability from the individual vendor driver. For that, we need another negotiation protocol similar to vhost_user's protocol_features between the vdpa kernel and QEMU, way before the guest driver is ever probed and its feature negotiation kicks in. Not sure we need a GET_MEMORY_ORDER ioctl call from the device, but we can assume weak ordering for legacy at this point (x86 only)? I'm lost here, we have get_features() so: I assume here you refer to get_device_features() that Eli just changed the name. 1) VERSION_1 means the device uses LE if provided, otherwise natvie 2) ORDER_PLATFORM means device requires platform ordering Any reason for having a new API for this? Are you going to enforce all vDPA hardware vendors to support the transitional model for legacy guest? Do we really have other choices? I suspect the legacy device is never implemented by any vendor: 1) no virtio way to detect host endian This is even true for transitional device that is conforming to the spec, right? The transport specific way to detect host endian is still being discussed and the spec revision is not finalized yet so far as I see. Why this suddenly becomes a requirement/blocker for h/w vendors to implement the transitional model? Even if the spec is out, this is pretty new and I suspect not all vendor would follow right away. I hope the software framework can be tolerant with h/w vendors not supporting host endianess (BE specifically) or not detecting it if they would like to support a transitional device for legacy. 2) bypass IOMMU with translated requests 3) PIO port Yes we have enp_vdpa, but it's more like a "transitional device" for legacy only guests. meaning guest not acknowledging VERSION_1 would use the legacy interfaces captured in the spec section 7.4 (regarding ring layout, native endianness, message framing, vq alignment of 4096, 32bit feature, no features_ok bit in status, IO port interface i.e. all the things) instead? Note that we only care about the datapath, control path is mediated anyhow. So feature_ok and IO port isn't an issue. The rest looks like a must for the hardware. H/W vendors can opt out not implementing transitional interfaces at all which limits itself a modern only device. Set endianess detection (via transport specific means) aside, for vendors that wishes to support transitional device with legacy interface, is it a hard stop to
[bug report] new kvmalloc() WARN() triggered by DRM ioctls tracking
Hi DRM Devs,
In commit 7661809d493b ("mm: don't allow oversized kvmalloc() calls")
from July, Linus added a WARN_ONCE() for "crazy" allocations over 2GB.
I have a static checker warning for this and most of the warnings are
from DRM ioctls.
drivers/gpu/drm/lima/lima_drv.c:124 lima_ioctl_gem_submit() warn: uncapped user
size for kvmalloc() will WARN
drivers/gpu/drm/radeon/radeon_cs.c:291 radeon_cs_parser_init() warn: uncapped
user size for kvmalloc() will WARN
drivers/gpu/drm/v3d/v3d_gem.c:311 v3d_lookup_bos() warn: uncapped user size for
kvmalloc() will WARN
drivers/gpu/drm/v3d/v3d_gem.c:319 v3d_lookup_bos() warn: uncapped user size for
kvmalloc() will WARN
drivers/gpu/drm/v3d/v3d_gem.c:601 v3d_get_multisync_post_deps() warn: uncapped
user size for kvmalloc() will WARN
drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c:476 etnaviv_ioctl_gem_submit()
warn: uncapped user size for kvmalloc() will WARN
drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c:477 etnaviv_ioctl_gem_submit()
warn: uncapped user size for kvmalloc() will WARN
drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c:478 etnaviv_ioctl_gem_submit()
warn: uncapped user size for kvmalloc() will WARN
drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c:479 etnaviv_ioctl_gem_submit()
warn: uncapped user size for kvmalloc() will WARN
drivers/gpu/drm/virtio/virtgpu_ioctl.c:186 virtio_gpu_execbuffer_ioctl() warn:
uncapped user size for kvmalloc() will WARN
drivers/gpu/drm/panfrost/panfrost_drv.c:198 panfrost_copy_in_sync() warn:
uncapped user size for kvmalloc() will WARN
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c:120 amdgpu_cs_parser_init() warn:
uncapped user size for kvmalloc() will WARN
These ioctls can all trigger the stack dump. The line numbers are from
linux next (next-20211214).
I feel like ideally if this could be fixed in a central way, but if not
then hopefully I've added the relevant lists to the CC.
regards,
dan carpenter
___
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization
