Re: [ovs-dev] Flow Control and Port Mirroring
On Sat, 30 Oct 2010 01:29:33 pm Simon Horman wrote: [ CCed VHOST contacts ] On Thu, Oct 28, 2010 at 01:22:02PM -0700, Jesse Gross wrote: On Thu, Oct 28, 2010 at 4:54 AM, Simon Horman ho...@verge.net.au wrote: My reasoning is that in the non-mirroring case the guest is limited by the external interface through wich the packets eventually flow - that is 1Gbit/s. But in the mirrored either there is no flow control or the flow control is acting on the rate of dummy0, which is essentailly infinate. Before investigating this any further I wanted to ask if this behaviour is intentional. It's not intentional but I can take a guess at what is happening. When we send the packet to a mirror, the skb is cloned but only the original skb is charged to the sender. If the original packet is delivered to localhost then it will be freed quickly and no longer accounted for, despite the fact that the real packet is still sitting in the transmit queue on the NIC. The UDP stack will then send the next packet, limited only by the speed of the CPU. That would explain what I have observed. I can't find the thread (what is ovs-dev?), but I think the tap device has this fundamental feature: you can blast as many packets as you want through it. If that's a bad thing, we have to look harder... Cheers, Rusty. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ovs-dev] Flow Control and Port Mirroring
On Mon, Nov 08, 2010 at 01:41:23PM +1030, Rusty Russell wrote: On Sat, 30 Oct 2010 01:29:33 pm Simon Horman wrote: [ CCed VHOST contacts ] On Thu, Oct 28, 2010 at 01:22:02PM -0700, Jesse Gross wrote: On Thu, Oct 28, 2010 at 4:54 AM, Simon Horman ho...@verge.net.au wrote: My reasoning is that in the non-mirroring case the guest is limited by the external interface through wich the packets eventually flow - that is 1Gbit/s. But in the mirrored either there is no flow control or the flow control is acting on the rate of dummy0, which is essentailly infinate. Before investigating this any further I wanted to ask if this behaviour is intentional. It's not intentional but I can take a guess at what is happening. When we send the packet to a mirror, the skb is cloned but only the original skb is charged to the sender. If the original packet is delivered to localhost then it will be freed quickly and no longer accounted for, despite the fact that the real packet is still sitting in the transmit queue on the NIC. The UDP stack will then send the next packet, limited only by the speed of the CPU. That would explain what I have observed. I can't find the thread (what is ovs-dev?), Sorry, yes its on ovs-dev. http://openvswitch.org/pipermail/dev_openvswitch.org/2010-October/003806.html but I think the tap device has this fundamental feature: you can blast as many packets as you want through it. If that's a bad thing, we have to look harder... There does seem to be flow control in the non-mirrored case. So I suspect its occurring at the skb level but that breaks down when a clone occurs. It would seem that fragment level flow control would help this problem (which is basically what Xen's netback/netfront has), but by this point I am speculating wildly. I'll try and find out exactly where the problem is occurring in order for us to have a more informed discussion. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [ovs-dev] Flow Control and Port Mirroring
[ CCed VHOST contacts ] On Thu, Oct 28, 2010 at 01:22:02PM -0700, Jesse Gross wrote: On Thu, Oct 28, 2010 at 4:54 AM, Simon Horman ho...@verge.net.au wrote: My reasoning is that in the non-mirroring case the guest is limited by the external interface through wich the packets eventually flow - that is 1Gbit/s. But in the mirrored either there is no flow control or the flow control is acting on the rate of dummy0, which is essentailly infinate. Before investigating this any further I wanted to ask if this behaviour is intentional. It's not intentional but I can take a guess at what is happening. When we send the packet to a mirror, the skb is cloned but only the original skb is charged to the sender. If the original packet is delivered to localhost then it will be freed quickly and no longer accounted for, despite the fact that the real packet is still sitting in the transmit queue on the NIC. The UDP stack will then send the next packet, limited only by the speed of the CPU. That would explain what I have observed. Normally, this would be tracked by accounting for the memory charged to the socket. However, I know that Xen tracks whether the actual pages of memory have been freed, which should avoid this problem since the memory won't be released util the last packet has been sent. I don't know what KVM virtio does but I'm guessing that it similar to the former, since this problem is occurring. I am also familiar of how Xen tracks pages but less sure of the virtio side of things. While it would be easy to charge the socket for all clones, I also want to be careful about over accounting of the same data, leading to a very small effective socket buffer. Agreed, we don't want to see over-charging. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
