Re: Packets dropped by virtual NICs
On 3/29/21 11:38 AM, Silvia Fichera wrote: Hi Michal, these are the steps: - start the vm with qemu sudo qemu-system-x86_64-spice -m 2048 -enable-kvm -smp 3 -cdrom /home/machine_A/ubuntu.iso -netdev tap,ifname=tap0,vhost=on,id=n1,vhostforce=on,queues=3,script=/etc/tap0_ifup.sh,downscript=/etc/tap0_ifdown.sh -device virtio-net-pci,netdev=n1,mac=9E:58:00:d2:53:03,mq=on,vectors=8 -netdev tap,ifname=tap1,vhost=on,id=n2,vhostforce=on,queues=3,script=/etc/tap1_ifup.sh,downscript=/etc/tap1_ifdown.sh -device virtio-net-pci,netdev=n2,mac=7A:53:00:d1:59:04,mq=on,vectors=8 -device virtio-net,netdev=network2 -netdev user,id=network2 -hda switch_A.img - attach the tap to the bridges in the host machine (to have the traffic coming from the outsider traffic generator injected in the vm) - in the VM: - enable ip_forward - enable promiscuous mode for the interface This sounds fishy. Why is this needed? - send the "tc qdisc" command to configure the output NIC with 2 traffic classes (TC) assigned to 2 different queues - add the iptables mangle rules to the POSTROUTING chain to assign TC1 to the traffic with dport and TC0 to the traffic with dport I'm sending 2 UDP flows, I have no loss under 3Mbps each. If I capture traffic with tcpdump on the ingress and on the egress nic (of the VM), I see a difference of 50% of packets When I did a test on the host machine, to check if it has the same problem, I've reached an aggregated traffic of 90Mbps with no loss. That's why I think that there is some misconfiguration on the virtual NIC. Well, that's fairly easy to test - create those two TAPs, do the setup you're doing and instad of spawning qemu, run iperf or spirent and tell it to use those TAPs. I think you'll find the problem elsewhere. Anyway, this problem is not libvirt related really and I guess you can find more elaborate replies on a list where TC developers hang out. Looks like they're using latrc: https://lartc.org/#mailinglist Michal
Re: Packets dropped by virtual NICs
Hi Michal, these are the steps: - start the vm with qemu sudo qemu-system-x86_64-spice -m 2048 -enable-kvm -smp 3 -cdrom /home/machine_A/ubuntu.iso -netdev tap,ifname=tap0,vhost=on,id=n1,vhostforce=on,queues=3,script=/etc/tap0_ifup.sh,downscript=/etc/tap0_ifdown.sh -device virtio-net-pci,netdev=n1,mac=9E:58:00:d2:53:03,mq=on,vectors=8 -netdev tap,ifname=tap1,vhost=on,id=n2,vhostforce=on,queues=3,script=/etc/tap1_ifup.sh,downscript=/etc/tap1_ifdown.sh -device virtio-net-pci,netdev=n2,mac=7A:53:00:d1:59:04,mq=on,vectors=8 -device virtio-net,netdev=network2 -netdev user,id=network2 -hda switch_A.img - attach the tap to the bridges in the host machine (to have the traffic coming from the outsider traffic generator injected in the vm) - in the VM: - enable ip_forward - enable promiscuous mode for the interface - send the "tc qdisc" command to configure the output NIC with 2 traffic classes (TC) assigned to 2 different queues - add the iptables mangle rules to the POSTROUTING chain to assign TC1 to the traffic with dport and TC0 to the traffic with dport I'm sending 2 UDP flows, I have no loss under 3Mbps each. If I capture traffic with tcpdump on the ingress and on the egress nic (of the VM), I see a difference of 50% of packets When I did a test on the host machine, to check if it has the same problem, I've reached an aggregated traffic of 90Mbps with no loss. That's why I think that there is some misconfiguration on the virtual NIC. Thanks Silvia On Mon, Mar 29, 2021 at 11:13 AM Michal Privoznik wrote: > On 3/27/21 2:53 PM, Silvia Fichera wrote: > > Hi all, > > I want to use tc qdisc settings in a network coposed of several qemu > > VMs, connected through bridges and tap interfaces. > > I generate traffic with a spirent. Everything is fine when the > > scheduling discipline is not installed but when I run the command to set > > taprio queues traffic on the VM's NIC the traffic is dropped, i can send > > max 1mbps. > > I think that there is something missing in the virtual NIC configuration > > or setup. With ethtool i can see that queues are configured. I've also > > noticed the BQL equals to 0, that is different than the physical machine > > (BQL=18600) where everything works correctly. > > I've read that it could be because NIC drivers do not support that > setting. > > > > Do you have any suggestions? > > Hey, > > I'm not familiar with taprio, but what's implemented in libvirt is htb > and sfq and that works well. Are you setting qdisc-s yourself or > modifying libvirt created structure? > > Are you setting these qdiscs from the host, right? > I know that when changing QoS settings (when libvirt changes qdics/class > layout) for a brief moment packets are not transmitted from/to guest. I > suspect that kernel is freeing up queues or something. But this does not > look like your case, does it? > > Michal > > -- Silvia Fichera
Re: Packets dropped by virtual NICs
On 3/27/21 2:53 PM, Silvia Fichera wrote: Hi all, I want to use tc qdisc settings in a network coposed of several qemu VMs, connected through bridges and tap interfaces. I generate traffic with a spirent. Everything is fine when the scheduling discipline is not installed but when I run the command to set taprio queues traffic on the VM's NIC the traffic is dropped, i can send max 1mbps. I think that there is something missing in the virtual NIC configuration or setup. With ethtool i can see that queues are configured. I've also noticed the BQL equals to 0, that is different than the physical machine (BQL=18600) where everything works correctly. I've read that it could be because NIC drivers do not support that setting. Do you have any suggestions? Hey, I'm not familiar with taprio, but what's implemented in libvirt is htb and sfq and that works well. Are you setting qdisc-s yourself or modifying libvirt created structure? Are you setting these qdiscs from the host, right? I know that when changing QoS settings (when libvirt changes qdics/class layout) for a brief moment packets are not transmitted from/to guest. I suspect that kernel is freeing up queues or something. But this does not look like your case, does it? Michal
Packets dropped by virtual NICs
Hi all, I want to use tc qdisc settings in a network coposed of several qemu VMs, connected through bridges and tap interfaces. I generate traffic with a spirent. Everything is fine when the scheduling discipline is not installed but when I run the command to set taprio queues traffic on the VM's NIC the traffic is dropped, i can send max 1mbps. I think that there is something missing in the virtual NIC configuration or setup. With ethtool i can see that queues are configured. I've also noticed the BQL equals to 0, that is different than the physical machine (BQL=18600) where everything works correctly. I've read that it could be because NIC drivers do not support that setting. Do you have any suggestions? Thanks you all Silvia
