Re: Question about virtio-net: implement RX RSS processing
Thank you for the answers and I will study the vhost-net patches set you
mentioned
On Fri, 3 Jul 2020, Jason Wang wrote:
>
> On 2020/7/3 上午1:07, Vincent Li wrote:
> > Hi,
> >
> > I noticed the [PULL V2 02/33] virtio-net: implement RX RSS processing
> > https://marc.info/?l=qemu-devel&m=159248675403246&w=2 and cloned
> > https://github.com/jasowang/qemu.git tags/net-pull-request for testing the
> > RX RSS feature, but I am not clear how to test this feature and see if it
> > meets my needs.
>
>
> Yuri may know more but I think the only driver that supports RSS is Windows
> driver currently.
>
>
> >
> > I am running F-Stack Nginx applications
> > https://github.com/F-Stack/f-stack/tree/dev/app/nginx-1.16.1 in KVM guest,
> > F-Stack is FreeBSD TCP/IP stack ported to support DPDK, and F-Stack set
> > RSS mode as code below
> > https://github.com/F-Stack/f-stack/blob/dev/lib/ff_dpdk_if.c#L605
> >
> > /* Set RSS mode */
> > uint64_t default_rss_hf = ETH_RSS_PROTO_MASK;
> > port_conf.rxmode.mq_mode = ETH_MQ_RX_RSS;
> > port_conf.rx_adv_conf.rss_conf.rss_hf = default_rss_hf;
> > if (dev_info.hash_key_size == 52) {
> > port_conf.rx_adv_conf.rss_conf.rss_key = default_rsskey_52bytes;
> > port_conf.rx_adv_conf.rss_conf.rss_key_len = 52;
> > use_rsskey_52bytes = 1;
> > } else {
> > port_conf.rx_adv_conf.rss_conf.rss_key = default_rsskey_40bytes;
> > port_conf.rx_adv_conf.rss_conf.rss_key_len = 40;
> > }
> > port_conf.rx_adv_conf.rss_conf.rss_hf &=
> > dev_info.flow_type_rss_offloads;
> > if (port_conf.rx_adv_conf.rss_conf.rss_hf != ETH_RSS_PROTO_MASK) {
> > printf("Port %u modified RSS hash function based on hardware
> > support,"
> > "requested:%#"PRIx64" configured:%#"PRIx64"\n",
> > port_id, default_rss_hf, port_conf.rx_adv_conf.rss_conf.rss_hf);
> > }
> >
> > But DPDK virtio PMD does not support RSS as below commit shows:
> >
> > commit 13b3137f3b7c8f866947a9b34e06a8aec0d084f7
> > Author: Dilshod Urazov
> > Date: Wed Oct 9 13:32:07 2019 +0100
> >
> > net/virtio: reject unsupported Rx multi-queue modes
> > This driver supports none of DCB, RSS or VMDQ modes, therefore
> > must
> > check and return error if configured incorrectly.
> > Virtio can distribute Rx packets across multi-queue, but there is
> > no controls (algorithm, redirection table, hash function) except
> > number of Rx queues and ETH_MQ_RX_NONE is the best fit meaning
> > no method is enforced on how to route packets to MQs.
> > Fixes: c1f86306a026 ("virtio: add new driver")
> > Cc: sta...@dpdk.org
> > Signed-off-by: Dilshod Urazov
> > Signed-off-by: Andrew Rybchenko
> > Reviewed-by: Maxime Coquelin
> >
> > diff --git a/drivers/net/virtio/virtio_ethdev.c
> > b/drivers/net/virtio/virtio_ethdev.c
> > index 0a2ed2e50..76bd40a3e 100644
> > --- a/drivers/net/virtio/virtio_ethdev.c
> > +++ b/drivers/net/virtio/virtio_ethdev.c
> > @@ -2066,6 +2066,13 @@ virtio_dev_configure(struct rte_eth_dev *dev)
> > PMD_INIT_LOG(DEBUG, "configure");
> > req_features = VIRTIO_PMD_DEFAULT_GUEST_FEATURES;
> > + if (rxmode->mq_mode != ETH_MQ_RX_NONE) {
> > + PMD_DRV_LOG(ERR,
> > + "Unsupported Rx multi queue mode %d",
> > + rxmode->mq_mode);
> > + return -EINVAL;
> > + }
> > +
> > if (dev->data->dev_conf.intr_conf.rxq) {
> > ret = virtio_init_device(dev, hw->req_guest_features);
> > if (ret < 0)
> >
> >
> >
> > So the problem is I can't run F-Stack Applications in KVM/Qemu guest with
> > multi queue/vCPU/RSS support, this problem seems apply to DPDK TCP
> > applications require multi queue/RSS support in KVM/Qemu guest, for
> > example mTCP https://github.com/mtcp-stack/mtcp I tested has similar
> > problem.
> >
> > I am not clear on the picture of how everything work together for this
> > virtio-net RSS feature.
> >
> > I have read following blogs
> >
> > https://www.redhat.com/en/blog/introduction-virtio-networking-and-vhost-net
> > https://www.redhat.com/en/blog/how-vhost-user-came-being-virtio-networking-and-dpdk
> >
> > Someone told me that in order for DPDK frond end virtio PMD in guest support
> > RSS, the backend
> > also needs to support RSS, including vhost-net and vhost-user, it should
> > have nothing to do with this Qemu virtio-net RSS, is that correct? if
> > correct, I have following questions:
> >
> > 1, What is the use case for this Qemu virtio-net RSS?
>
>
> It's just RSS as what other device can provide for steering or balancing.
>
>
> > 2, How to test the use case?
>
>
> Need use windows guest.
>
>
> > 3, Are there any plan to improve vhost-net/vhost-user, DPDK virtio PMD to
> > support RSS?
>
>
> For vhost-net, Sameeh posted a eBPF based soluti
Re: Question about virtio-net: implement RX RSS processing
On 2020/7/3 上午1:07, Vincent Li wrote:
Hi,
I noticed the [PULL V2 02/33] virtio-net: implement RX RSS processing
https://marc.info/?l=qemu-devel&m=159248675403246&w=2 and cloned
https://github.com/jasowang/qemu.git tags/net-pull-request for testing the
RX RSS feature, but I am not clear how to test this feature and see if it
meets my needs.
Yuri may know more but I think the only driver that supports RSS is
Windows driver currently.
I am running F-Stack Nginx applications
https://github.com/F-Stack/f-stack/tree/dev/app/nginx-1.16.1 in KVM guest,
F-Stack is FreeBSD TCP/IP stack ported to support DPDK, and F-Stack set
RSS mode as code below
https://github.com/F-Stack/f-stack/blob/dev/lib/ff_dpdk_if.c#L605
/* Set RSS mode */
uint64_t default_rss_hf = ETH_RSS_PROTO_MASK;
port_conf.rxmode.mq_mode = ETH_MQ_RX_RSS;
port_conf.rx_adv_conf.rss_conf.rss_hf = default_rss_hf;
if (dev_info.hash_key_size == 52) {
port_conf.rx_adv_conf.rss_conf.rss_key = default_rsskey_52bytes;
port_conf.rx_adv_conf.rss_conf.rss_key_len = 52;
use_rsskey_52bytes = 1;
} else {
port_conf.rx_adv_conf.rss_conf.rss_key = default_rsskey_40bytes;
port_conf.rx_adv_conf.rss_conf.rss_key_len = 40;
}
port_conf.rx_adv_conf.rss_conf.rss_hf &= dev_info.flow_type_rss_offloads;
if (port_conf.rx_adv_conf.rss_conf.rss_hf != ETH_RSS_PROTO_MASK) {
printf("Port %u modified RSS hash function based on hardware support,"
"requested:%#"PRIx64" configured:%#"PRIx64"\n",
port_id, default_rss_hf, port_conf.rx_adv_conf.rss_conf.rss_hf);
}
But DPDK virtio PMD does not support RSS as below commit shows:
commit 13b3137f3b7c8f866947a9b34e06a8aec0d084f7
Author: Dilshod Urazov
Date: Wed Oct 9 13:32:07 2019 +0100
net/virtio: reject unsupported Rx multi-queue modes
This driver supports none of DCB, RSS or VMDQ modes, therefore must
check and return error if configured incorrectly.
Virtio can distribute Rx packets across multi-queue, but there is
no controls (algorithm, redirection table, hash function) except
number of Rx queues and ETH_MQ_RX_NONE is the best fit meaning
no method is enforced on how to route packets to MQs.
Fixes: c1f86306a026 ("virtio: add new driver")
Cc: sta...@dpdk.org
Signed-off-by: Dilshod Urazov
Signed-off-by: Andrew Rybchenko
Reviewed-by: Maxime Coquelin
diff --git a/drivers/net/virtio/virtio_ethdev.c
b/drivers/net/virtio/virtio_ethdev.c
index 0a2ed2e50..76bd40a3e 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -2066,6 +2066,13 @@ virtio_dev_configure(struct rte_eth_dev *dev)
PMD_INIT_LOG(DEBUG, "configure");
req_features = VIRTIO_PMD_DEFAULT_GUEST_FEATURES;
+ if (rxmode->mq_mode != ETH_MQ_RX_NONE) {
+ PMD_DRV_LOG(ERR,
+ "Unsupported Rx multi queue mode %d",
+ rxmode->mq_mode);
+ return -EINVAL;
+ }
+
if (dev->data->dev_conf.intr_conf.rxq) {
ret = virtio_init_device(dev, hw->req_guest_features);
if (ret < 0)
So the problem is I can't run F-Stack Applications in KVM/Qemu guest with
multi queue/vCPU/RSS support, this problem seems apply to DPDK TCP
applications require multi queue/RSS support in KVM/Qemu guest, for
example mTCP https://github.com/mtcp-stack/mtcp I tested has similar
problem.
I am not clear on the picture of how everything work together for this
virtio-net RSS feature.
I have read following blogs
https://www.redhat.com/en/blog/introduction-virtio-networking-and-vhost-net
https://www.redhat.com/en/blog/how-vhost-user-came-being-virtio-networking-and-dpdk
Someone told me that in order for DPDK frond end virtio PMD in guest support
RSS, the backend
also needs to support RSS, including vhost-net and vhost-user, it should
have nothing to do with this Qemu virtio-net RSS, is that correct? if
correct, I have following questions:
1, What is the use case for this Qemu virtio-net RSS?
It's just RSS as what other device can provide for steering or balancing.
2, How to test the use case?
Need use windows guest.
3, Are there any plan to improve vhost-net/vhost-user, DPDK virtio PMD to
support RSS?
For vhost-net, Sameeh posted a eBPF based solution RFC[1], we need some
one to carry on the work. It doesn't request any extension to vhost-net
thanks to the steering eBPF support in tuntap.
For vhost-user, we need probably extend vhost-user protocols first.
You're welcome to contribute patches.
[1] https://patchwork.kernel.org/cover/10581921/
Thanks
For 3, I think this is important for KVM/Qemu/OVS-DPDK/Vhost-net environment
for DPDK TCP/UDP applications.
Note I have no problem running F-Stack or mTCP applications in VMware ESXi
guest environment with multi queue/vCPU/R
Question about virtio-net: implement RX RSS processing
Hi,
I noticed the [PULL V2 02/33] virtio-net: implement RX RSS processing
https://marc.info/?l=qemu-devel&m=159248675403246&w=2 and cloned
https://github.com/jasowang/qemu.git tags/net-pull-request for testing the
RX RSS feature, but I am not clear how to test this feature and see if it
meets my needs.
I am running F-Stack Nginx applications
https://github.com/F-Stack/f-stack/tree/dev/app/nginx-1.16.1 in KVM guest,
F-Stack is FreeBSD TCP/IP stack ported to support DPDK, and F-Stack set
RSS mode as code below
https://github.com/F-Stack/f-stack/blob/dev/lib/ff_dpdk_if.c#L605
/* Set RSS mode */
uint64_t default_rss_hf = ETH_RSS_PROTO_MASK;
port_conf.rxmode.mq_mode = ETH_MQ_RX_RSS;
port_conf.rx_adv_conf.rss_conf.rss_hf = default_rss_hf;
if (dev_info.hash_key_size == 52) {
port_conf.rx_adv_conf.rss_conf.rss_key = default_rsskey_52bytes;
port_conf.rx_adv_conf.rss_conf.rss_key_len = 52;
use_rsskey_52bytes = 1;
} else {
port_conf.rx_adv_conf.rss_conf.rss_key = default_rsskey_40bytes;
port_conf.rx_adv_conf.rss_conf.rss_key_len = 40;
}
port_conf.rx_adv_conf.rss_conf.rss_hf &= dev_info.flow_type_rss_offloads;
if (port_conf.rx_adv_conf.rss_conf.rss_hf != ETH_RSS_PROTO_MASK) {
printf("Port %u modified RSS hash function based on hardware support,"
"requested:%#"PRIx64" configured:%#"PRIx64"\n",
port_id, default_rss_hf, port_conf.rx_adv_conf.rss_conf.rss_hf);
}
But DPDK virtio PMD does not support RSS as below commit shows:
commit 13b3137f3b7c8f866947a9b34e06a8aec0d084f7
Author: Dilshod Urazov
Date: Wed Oct 9 13:32:07 2019 +0100
net/virtio: reject unsupported Rx multi-queue modes
This driver supports none of DCB, RSS or VMDQ modes, therefore must
check and return error if configured incorrectly.
Virtio can distribute Rx packets across multi-queue, but there is
no controls (algorithm, redirection table, hash function) except
number of Rx queues and ETH_MQ_RX_NONE is the best fit meaning
no method is enforced on how to route packets to MQs.
Fixes: c1f86306a026 ("virtio: add new driver")
Cc: sta...@dpdk.org
Signed-off-by: Dilshod Urazov
Signed-off-by: Andrew Rybchenko
Reviewed-by: Maxime Coquelin
diff --git a/drivers/net/virtio/virtio_ethdev.c
b/drivers/net/virtio/virtio_ethdev.c
index 0a2ed2e50..76bd40a3e 100644
--- a/drivers/net/virtio/virtio_ethdev.c
+++ b/drivers/net/virtio/virtio_ethdev.c
@@ -2066,6 +2066,13 @@ virtio_dev_configure(struct rte_eth_dev *dev)
PMD_INIT_LOG(DEBUG, "configure");
req_features = VIRTIO_PMD_DEFAULT_GUEST_FEATURES;
+ if (rxmode->mq_mode != ETH_MQ_RX_NONE) {
+ PMD_DRV_LOG(ERR,
+ "Unsupported Rx multi queue mode %d",
+ rxmode->mq_mode);
+ return -EINVAL;
+ }
+
if (dev->data->dev_conf.intr_conf.rxq) {
ret = virtio_init_device(dev, hw->req_guest_features);
if (ret < 0)
So the problem is I can't run F-Stack Applications in KVM/Qemu guest with
multi queue/vCPU/RSS support, this problem seems apply to DPDK TCP
applications require multi queue/RSS support in KVM/Qemu guest, for
example mTCP https://github.com/mtcp-stack/mtcp I tested has similar
problem.
I am not clear on the picture of how everything work together for this
virtio-net RSS feature.
I have read following blogs
https://www.redhat.com/en/blog/introduction-virtio-networking-and-vhost-net
https://www.redhat.com/en/blog/how-vhost-user-came-being-virtio-networking-and-dpdk
Someone told me that in order for DPDK frond end virtio PMD in guest support
RSS, the backend
also needs to support RSS, including vhost-net and vhost-user, it should
have nothing to do with this Qemu virtio-net RSS, is that correct? if
correct, I have following questions:
1, What is the use case for this Qemu virtio-net RSS?
2, How to test the use case?
3, Are there any plan to improve vhost-net/vhost-user, DPDK virtio PMD to
support RSS?
For 3, I think this is important for KVM/Qemu/OVS-DPDK/Vhost-net environment
for DPDK TCP/UDP applications.
Note I have no problem running F-Stack or mTCP applications in VMware ESXi
guest environment with multi queue/vCPU/RSS requirement since DPDK vmxnet3
PMD support RSS and I assume VMware ESXi backend support RSS, so it looks
VMware has
advantage over this.
Thank you for your patience to read this email
Regards,
Vincent
