Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On Monday, December 03, 2012 08:01:11 AM Stephen Hemminger wrote: > On Mon, 03 Dec 2012 14:45:46 +0800 > > Jason Wang wrote: > > On Tuesday, November 27, 2012 08:49:19 AM Stephen Hemminger wrote: > > > On Tue, 27 Nov 2012 14:45:13 +0800 > > > > > > Jason Wang wrote: > > > > On 11/27/2012 01:37 AM, Stephen Hemminger wrote: > > > > > On Mon, 26 Nov 2012 15:56:52 +0800 > > > > > > > > > > Jason Wang wrote: > > > > >> Some deivces do not free the old tx skbs immediately after it has > > > > >> been > > > > >> sent > > > > >> (usually in tx interrupt). One such example is virtio-net which > > > > >> optimizes for virt and only free the possible old tx skbs during > > > > >> the > > > > >> next packet sending. This would lead the pktgen to wait forever in > > > > >> the > > > > >> refcount of the skb if no other pakcet will be sent afterwards. > > > > >> > > > > >> Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY > > > > >> which could notify the pktgen that the device does not free skb > > > > >> immediately after it has been sent and let it not to wait for the > > > > >> refcount to be one. > > > > >> > > > > >> Signed-off-by: Jason Wang > > > > > > > > > > Another alternative would be using skb_orphan() and skb->destructor. > > > > > There are other cases where skb's are not freed right away. > > > > > -- > > > > > To unsubscribe from this list: send the line "unsubscribe netdev" in > > > > > the body of a message to majord...@vger.kernel.org > > > > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > > > > > > Hi Stephen: > > > > > > > > Do you mean registering a skb->destructor for pktgen then set and > > > > check > > > > bits in skb->tx_flag? > > > > > > Yes. Register a destructor that does something like update a counter > > > (number of packets pending), then just spin while number of packets > > > pending is over threshold. > > > > Have some experiments on this, looks like it does not work weel when > > clone_skb is used. For driver that call skb_orphan() in ndo_start_xmit, > > the destructor is only called when the first packet were sent, but what > > we need to know is when the last were sent. Any thoughts on this or we > > can just introduce another flag (anyway we have something like > > IFF_TX_SKB_SHARING) ? > > The SKB_SHARING flag looks like the best solution then. > Surprisingly, transmit buffer completion is a major bottleneck for 10G > devices, and I suspect more changes will come. It works, but we may lose some chances to use clone_skb and stress the device and driver more. I'm thinking maybe we can turn back to my original RFC to introduce another flag. This flag maybe also useful for BQL and zerocopy in the future since both of them are sensitive to the transmit buffer completion. > -- > To unsubscribe from this list: send the line "unsubscribe netdev" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On Monday, December 03, 2012 08:01:11 AM Stephen Hemminger wrote: On Mon, 03 Dec 2012 14:45:46 +0800 Jason Wang jasow...@redhat.com wrote: On Tuesday, November 27, 2012 08:49:19 AM Stephen Hemminger wrote: On Tue, 27 Nov 2012 14:45:13 +0800 Jason Wang jasow...@redhat.com wrote: On 11/27/2012 01:37 AM, Stephen Hemminger wrote: On Mon, 26 Nov 2012 15:56:52 +0800 Jason Wang jasow...@redhat.com wrote: Some deivces do not free the old tx skbs immediately after it has been sent (usually in tx interrupt). One such example is virtio-net which optimizes for virt and only free the possible old tx skbs during the next packet sending. This would lead the pktgen to wait forever in the refcount of the skb if no other pakcet will be sent afterwards. Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY which could notify the pktgen that the device does not free skb immediately after it has been sent and let it not to wait for the refcount to be one. Signed-off-by: Jason Wang jasow...@redhat.com Another alternative would be using skb_orphan() and skb-destructor. There are other cases where skb's are not freed right away. -- To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Hi Stephen: Do you mean registering a skb-destructor for pktgen then set and check bits in skb-tx_flag? Yes. Register a destructor that does something like update a counter (number of packets pending), then just spin while number of packets pending is over threshold. Have some experiments on this, looks like it does not work weel when clone_skb is used. For driver that call skb_orphan() in ndo_start_xmit, the destructor is only called when the first packet were sent, but what we need to know is when the last were sent. Any thoughts on this or we can just introduce another flag (anyway we have something like IFF_TX_SKB_SHARING) ? The SKB_SHARING flag looks like the best solution then. Surprisingly, transmit buffer completion is a major bottleneck for 10G devices, and I suspect more changes will come. It works, but we may lose some chances to use clone_skb and stress the device and driver more. I'm thinking maybe we can turn back to my original RFC to introduce another flag. This flag maybe also useful for BQL and zerocopy in the future since both of them are sensitive to the transmit buffer completion. -- To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On Monday, December 03, 2012 08:01:11 AM Stephen Hemminger wrote: > On Mon, 03 Dec 2012 14:45:46 +0800 > > Jason Wang wrote: > > On Tuesday, November 27, 2012 08:49:19 AM Stephen Hemminger wrote: > > > On Tue, 27 Nov 2012 14:45:13 +0800 > > > > > > Jason Wang wrote: > > > > On 11/27/2012 01:37 AM, Stephen Hemminger wrote: > > > > > On Mon, 26 Nov 2012 15:56:52 +0800 > > > > > > > > > > Jason Wang wrote: > > > > >> Some deivces do not free the old tx skbs immediately after it has > > > > >> been > > > > >> sent > > > > >> (usually in tx interrupt). One such example is virtio-net which > > > > >> optimizes for virt and only free the possible old tx skbs during > > > > >> the > > > > >> next packet sending. This would lead the pktgen to wait forever in > > > > >> the > > > > >> refcount of the skb if no other pakcet will be sent afterwards. > > > > >> > > > > >> Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY > > > > >> which could notify the pktgen that the device does not free skb > > > > >> immediately after it has been sent and let it not to wait for the > > > > >> refcount to be one. > > > > >> > > > > >> Signed-off-by: Jason Wang > > > > > > > > > > Another alternative would be using skb_orphan() and skb->destructor. > > > > > There are other cases where skb's are not freed right away. > > > > > -- > > > > > To unsubscribe from this list: send the line "unsubscribe netdev" in > > > > > the body of a message to majord...@vger.kernel.org > > > > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > > > > > > Hi Stephen: > > > > > > > > Do you mean registering a skb->destructor for pktgen then set and > > > > check > > > > bits in skb->tx_flag? > > > > > > Yes. Register a destructor that does something like update a counter > > > (number of packets pending), then just spin while number of packets > > > pending is over threshold. > > > > Have some experiments on this, looks like it does not work weel when > > clone_skb is used. For driver that call skb_orphan() in ndo_start_xmit, > > the destructor is only called when the first packet were sent, but what > > we need to know is when the last were sent. Any thoughts on this or we > > can just introduce another flag (anyway we have something like > > IFF_TX_SKB_SHARING) ? > > The SKB_SHARING flag looks like the best solution then. > Surprisingly, transmit buffer completion is a major bottleneck for 10G > devices, and I suspect more changes will come. It works, but we may lose some chances to use clone_skb and stress the device and driver more. I'm thinking maybe we can turn back to my original RFC to introduce another flag. This flag maybe also useful for BQL and zerocopy in the future since both of them are sensitive to the transmit buffer completion. > -- > To unsubscribe from this list: send the line "unsubscribe netdev" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On Monday, December 03, 2012 08:01:11 AM Stephen Hemminger wrote: On Mon, 03 Dec 2012 14:45:46 +0800 Jason Wang jasow...@redhat.com wrote: On Tuesday, November 27, 2012 08:49:19 AM Stephen Hemminger wrote: On Tue, 27 Nov 2012 14:45:13 +0800 Jason Wang jasow...@redhat.com wrote: On 11/27/2012 01:37 AM, Stephen Hemminger wrote: On Mon, 26 Nov 2012 15:56:52 +0800 Jason Wang jasow...@redhat.com wrote: Some deivces do not free the old tx skbs immediately after it has been sent (usually in tx interrupt). One such example is virtio-net which optimizes for virt and only free the possible old tx skbs during the next packet sending. This would lead the pktgen to wait forever in the refcount of the skb if no other pakcet will be sent afterwards. Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY which could notify the pktgen that the device does not free skb immediately after it has been sent and let it not to wait for the refcount to be one. Signed-off-by: Jason Wang jasow...@redhat.com Another alternative would be using skb_orphan() and skb-destructor. There are other cases where skb's are not freed right away. -- To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Hi Stephen: Do you mean registering a skb-destructor for pktgen then set and check bits in skb-tx_flag? Yes. Register a destructor that does something like update a counter (number of packets pending), then just spin while number of packets pending is over threshold. Have some experiments on this, looks like it does not work weel when clone_skb is used. For driver that call skb_orphan() in ndo_start_xmit, the destructor is only called when the first packet were sent, but what we need to know is when the last were sent. Any thoughts on this or we can just introduce another flag (anyway we have something like IFF_TX_SKB_SHARING) ? The SKB_SHARING flag looks like the best solution then. Surprisingly, transmit buffer completion is a major bottleneck for 10G devices, and I suspect more changes will come. It works, but we may lose some chances to use clone_skb and stress the device and driver more. I'm thinking maybe we can turn back to my original RFC to introduce another flag. This flag maybe also useful for BQL and zerocopy in the future since both of them are sensitive to the transmit buffer completion. -- To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On Mon, 03 Dec 2012 14:45:46 +0800 Jason Wang wrote: > On Tuesday, November 27, 2012 08:49:19 AM Stephen Hemminger wrote: > > On Tue, 27 Nov 2012 14:45:13 +0800 > > > > Jason Wang wrote: > > > On 11/27/2012 01:37 AM, Stephen Hemminger wrote: > > > > On Mon, 26 Nov 2012 15:56:52 +0800 > > > > > > > > Jason Wang wrote: > > > >> Some deivces do not free the old tx skbs immediately after it has been > > > >> sent > > > >> (usually in tx interrupt). One such example is virtio-net which > > > >> optimizes for virt and only free the possible old tx skbs during the > > > >> next packet sending. This would lead the pktgen to wait forever in the > > > >> refcount of the skb if no other pakcet will be sent afterwards. > > > >> > > > >> Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY > > > >> which could notify the pktgen that the device does not free skb > > > >> immediately after it has been sent and let it not to wait for the > > > >> refcount to be one. > > > >> > > > >> Signed-off-by: Jason Wang > > > > > > > > Another alternative would be using skb_orphan() and skb->destructor. > > > > There are other cases where skb's are not freed right away. > > > > -- > > > > To unsubscribe from this list: send the line "unsubscribe netdev" in > > > > the body of a message to majord...@vger.kernel.org > > > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > > > > Hi Stephen: > > > > > > Do you mean registering a skb->destructor for pktgen then set and check > > > bits in skb->tx_flag? > > > > Yes. Register a destructor that does something like update a counter (number > > of packets pending), then just spin while number of packets pending is over > > threshold. > > Have some experiments on this, looks like it does not work weel when > clone_skb > is used. For driver that call skb_orphan() in ndo_start_xmit, the destructor > is only called when the first packet were sent, but what we need to know is > when the last were sent. Any thoughts on this or we can just introduce > another > flag (anyway we have something like IFF_TX_SKB_SHARING) ? > The SKB_SHARING flag looks like the best solution then. Surprisingly, transmit buffer completion is a major bottleneck for 10G devices, and I suspect more changes will come. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On Mon, 03 Dec 2012 14:45:46 +0800 Jason Wang jasow...@redhat.com wrote: On Tuesday, November 27, 2012 08:49:19 AM Stephen Hemminger wrote: On Tue, 27 Nov 2012 14:45:13 +0800 Jason Wang jasow...@redhat.com wrote: On 11/27/2012 01:37 AM, Stephen Hemminger wrote: On Mon, 26 Nov 2012 15:56:52 +0800 Jason Wang jasow...@redhat.com wrote: Some deivces do not free the old tx skbs immediately after it has been sent (usually in tx interrupt). One such example is virtio-net which optimizes for virt and only free the possible old tx skbs during the next packet sending. This would lead the pktgen to wait forever in the refcount of the skb if no other pakcet will be sent afterwards. Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY which could notify the pktgen that the device does not free skb immediately after it has been sent and let it not to wait for the refcount to be one. Signed-off-by: Jason Wang jasow...@redhat.com Another alternative would be using skb_orphan() and skb-destructor. There are other cases where skb's are not freed right away. -- To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Hi Stephen: Do you mean registering a skb-destructor for pktgen then set and check bits in skb-tx_flag? Yes. Register a destructor that does something like update a counter (number of packets pending), then just spin while number of packets pending is over threshold. Have some experiments on this, looks like it does not work weel when clone_skb is used. For driver that call skb_orphan() in ndo_start_xmit, the destructor is only called when the first packet were sent, but what we need to know is when the last were sent. Any thoughts on this or we can just introduce another flag (anyway we have something like IFF_TX_SKB_SHARING) ? The SKB_SHARING flag looks like the best solution then. Surprisingly, transmit buffer completion is a major bottleneck for 10G devices, and I suspect more changes will come. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On Tuesday, November 27, 2012 08:49:19 AM Stephen Hemminger wrote: > On Tue, 27 Nov 2012 14:45:13 +0800 > > Jason Wang wrote: > > On 11/27/2012 01:37 AM, Stephen Hemminger wrote: > > > On Mon, 26 Nov 2012 15:56:52 +0800 > > > > > > Jason Wang wrote: > > >> Some deivces do not free the old tx skbs immediately after it has been > > >> sent > > >> (usually in tx interrupt). One such example is virtio-net which > > >> optimizes for virt and only free the possible old tx skbs during the > > >> next packet sending. This would lead the pktgen to wait forever in the > > >> refcount of the skb if no other pakcet will be sent afterwards. > > >> > > >> Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY > > >> which could notify the pktgen that the device does not free skb > > >> immediately after it has been sent and let it not to wait for the > > >> refcount to be one. > > >> > > >> Signed-off-by: Jason Wang > > > > > > Another alternative would be using skb_orphan() and skb->destructor. > > > There are other cases where skb's are not freed right away. > > > -- > > > To unsubscribe from this list: send the line "unsubscribe netdev" in > > > the body of a message to majord...@vger.kernel.org > > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > > Hi Stephen: > > > > Do you mean registering a skb->destructor for pktgen then set and check > > bits in skb->tx_flag? > > Yes. Register a destructor that does something like update a counter (number > of packets pending), then just spin while number of packets pending is over > threshold. Have some experiments on this, looks like it does not work weel when clone_skb is used. For driver that call skb_orphan() in ndo_start_xmit, the destructor is only called when the first packet were sent, but what we need to know is when the last were sent. Any thoughts on this or we can just introduce another flag (anyway we have something like IFF_TX_SKB_SHARING) ? Thanks > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On Tuesday, November 27, 2012 08:49:19 AM Stephen Hemminger wrote: On Tue, 27 Nov 2012 14:45:13 +0800 Jason Wang jasow...@redhat.com wrote: On 11/27/2012 01:37 AM, Stephen Hemminger wrote: On Mon, 26 Nov 2012 15:56:52 +0800 Jason Wang jasow...@redhat.com wrote: Some deivces do not free the old tx skbs immediately after it has been sent (usually in tx interrupt). One such example is virtio-net which optimizes for virt and only free the possible old tx skbs during the next packet sending. This would lead the pktgen to wait forever in the refcount of the skb if no other pakcet will be sent afterwards. Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY which could notify the pktgen that the device does not free skb immediately after it has been sent and let it not to wait for the refcount to be one. Signed-off-by: Jason Wang jasow...@redhat.com Another alternative would be using skb_orphan() and skb-destructor. There are other cases where skb's are not freed right away. -- To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Hi Stephen: Do you mean registering a skb-destructor for pktgen then set and check bits in skb-tx_flag? Yes. Register a destructor that does something like update a counter (number of packets pending), then just spin while number of packets pending is over threshold. Have some experiments on this, looks like it does not work weel when clone_skb is used. For driver that call skb_orphan() in ndo_start_xmit, the destructor is only called when the first packet were sent, but what we need to know is when the last were sent. Any thoughts on this or we can just introduce another flag (anyway we have something like IFF_TX_SKB_SHARING) ? Thanks -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On Thu, 29 Nov 2012 13:30:13 +0200 "Michael S. Tsirkin" wrote: > On Thu, Nov 29, 2012 at 06:13:13PM +0800, Jason Wang wrote: > > On Wednesday, November 28, 2012 08:53:05 AM Stephen Hemminger wrote: > > > On Wed, 28 Nov 2012 14:48:52 +0800 > > > > > > Jason Wang wrote: > > > > On 11/28/2012 12:49 AM, Stephen Hemminger wrote: > > > > > On Tue, 27 Nov 2012 14:45:13 +0800 > > > > > > > > > > Jason Wang wrote: > > > > >> On 11/27/2012 01:37 AM, Stephen Hemminger wrote: > > > > >>> On Mon, 26 Nov 2012 15:56:52 +0800 > > > > >>> > > > > >>> Jason Wang wrote: > > > > Some deivces do not free the old tx skbs immediately after it has > > > > been sent > > > > (usually in tx interrupt). One such example is virtio-net which > > > > optimizes for virt and only free the possible old tx skbs during > > > > the > > > > next packet sending. This would lead the pktgen to wait forever in > > > > the refcount of the skb if no other pakcet will be sent afterwards. > > > > > > > > Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY > > > > which could notify the pktgen that the device does not free skb > > > > immediately after it has been sent and let it not to wait for the > > > > refcount to be one. > > > > > > > > Signed-off-by: Jason Wang > > > > >>> > > > > >>> Another alternative would be using skb_orphan() and skb->destructor. > > > > >>> There are other cases where skb's are not freed right away. > > > > >>> -- > > > > >>> To unsubscribe from this list: send the line "unsubscribe netdev" in > > > > >>> the body of a message to majord...@vger.kernel.org > > > > >>> More majordomo info at http://vger.kernel.org/majordomo-info.html > > > > >> > > > > >> Hi Stephen: > > > > >> > > > > >> Do you mean registering a skb->destructor for pktgen then set and > > > > >> check > > > > >> bits in skb->tx_flag? > > > > > > > > > > Yes. Register a destructor that does something like update a counter > > > > > (number of packets pending), then just spin while number of packets > > > > > pending is over threshold. > > > > > -- > > > > > > > > Not sure this is the best method, since pktgen was used to test the tx > > > > process of the device driver and NIC. If we use skb_orhpan(), we would > > > > miss the test of tx completion part. > > > > > > There are other places that delay freeing and your solution would mean > > > finding and fixing all those. Code that does that already has to use > > > skb_orphan() to work, and I was looking for a way that could use that. > > > Introducing another flag value seems like a long term burden. > > > > > > > Get the point, will draft another version. > > > > > Alternatively, virtio could do cleanup more aggressively. Maybe in > > > response > > > to ring getting half full, or add a cleanup timer or something to avoid > > > the > > > problem. > > Timer would prevent complete deadlock but it is very expensive > in the virt scenario. > pulling at ring half full would only help if ring gets half full :) > which it does not have to. A timer that fires once (when idle) to mop up the last packet in a stream would be not very expensive. Most of the time, it would just be continually rescheduled. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On Thu, Nov 29, 2012 at 06:13:13PM +0800, Jason Wang wrote: > On Wednesday, November 28, 2012 08:53:05 AM Stephen Hemminger wrote: > > On Wed, 28 Nov 2012 14:48:52 +0800 > > > > Jason Wang wrote: > > > On 11/28/2012 12:49 AM, Stephen Hemminger wrote: > > > > On Tue, 27 Nov 2012 14:45:13 +0800 > > > > > > > > Jason Wang wrote: > > > >> On 11/27/2012 01:37 AM, Stephen Hemminger wrote: > > > >>> On Mon, 26 Nov 2012 15:56:52 +0800 > > > >>> > > > >>> Jason Wang wrote: > > > Some deivces do not free the old tx skbs immediately after it has > > > been sent > > > (usually in tx interrupt). One such example is virtio-net which > > > optimizes for virt and only free the possible old tx skbs during the > > > next packet sending. This would lead the pktgen to wait forever in > > > the refcount of the skb if no other pakcet will be sent afterwards. > > > > > > Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY > > > which could notify the pktgen that the device does not free skb > > > immediately after it has been sent and let it not to wait for the > > > refcount to be one. > > > > > > Signed-off-by: Jason Wang > > > >>> > > > >>> Another alternative would be using skb_orphan() and skb->destructor. > > > >>> There are other cases where skb's are not freed right away. > > > >>> -- > > > >>> To unsubscribe from this list: send the line "unsubscribe netdev" in > > > >>> the body of a message to majord...@vger.kernel.org > > > >>> More majordomo info at http://vger.kernel.org/majordomo-info.html > > > >> > > > >> Hi Stephen: > > > >> > > > >> Do you mean registering a skb->destructor for pktgen then set and check > > > >> bits in skb->tx_flag? > > > > > > > > Yes. Register a destructor that does something like update a counter > > > > (number of packets pending), then just spin while number of packets > > > > pending is over threshold. > > > > -- > > > > > > Not sure this is the best method, since pktgen was used to test the tx > > > process of the device driver and NIC. If we use skb_orhpan(), we would > > > miss the test of tx completion part. > > > > There are other places that delay freeing and your solution would mean > > finding and fixing all those. Code that does that already has to use > > skb_orphan() to work, and I was looking for a way that could use that. > > Introducing another flag value seems like a long term burden. > > > > Get the point, will draft another version. > > > Alternatively, virtio could do cleanup more aggressively. Maybe in response > > to ring getting half full, or add a cleanup timer or something to avoid the > > problem. Timer would prevent complete deadlock but it is very expensive in the virt scenario. pulling at ring half full would only help if ring gets half full :) which it does not have to. > > May worth to try. Another method is that virtio has a feature to notify guest > when tx ring is empty, we could free the old tx skbs there. > But it may brings > extra overhead. If we could let virtio_net free the old tx skb timely, it > would be easier to bring BQL support to virtio_net also. > > Thanks We used to use notify on empty - it's still very slow. > > > > > > > > -- > > To unsubscribe from this list: send the line "unsubscribe netdev" in > > the body of a message to majord...@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On Wednesday, November 28, 2012 08:53:05 AM Stephen Hemminger wrote: > On Wed, 28 Nov 2012 14:48:52 +0800 > > Jason Wang wrote: > > On 11/28/2012 12:49 AM, Stephen Hemminger wrote: > > > On Tue, 27 Nov 2012 14:45:13 +0800 > > > > > > Jason Wang wrote: > > >> On 11/27/2012 01:37 AM, Stephen Hemminger wrote: > > >>> On Mon, 26 Nov 2012 15:56:52 +0800 > > >>> > > >>> Jason Wang wrote: > > Some deivces do not free the old tx skbs immediately after it has > > been sent > > (usually in tx interrupt). One such example is virtio-net which > > optimizes for virt and only free the possible old tx skbs during the > > next packet sending. This would lead the pktgen to wait forever in > > the refcount of the skb if no other pakcet will be sent afterwards. > > > > Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY > > which could notify the pktgen that the device does not free skb > > immediately after it has been sent and let it not to wait for the > > refcount to be one. > > > > Signed-off-by: Jason Wang > > >>> > > >>> Another alternative would be using skb_orphan() and skb->destructor. > > >>> There are other cases where skb's are not freed right away. > > >>> -- > > >>> To unsubscribe from this list: send the line "unsubscribe netdev" in > > >>> the body of a message to majord...@vger.kernel.org > > >>> More majordomo info at http://vger.kernel.org/majordomo-info.html > > >> > > >> Hi Stephen: > > >> > > >> Do you mean registering a skb->destructor for pktgen then set and check > > >> bits in skb->tx_flag? > > > > > > Yes. Register a destructor that does something like update a counter > > > (number of packets pending), then just spin while number of packets > > > pending is over threshold. > > > -- > > > > Not sure this is the best method, since pktgen was used to test the tx > > process of the device driver and NIC. If we use skb_orhpan(), we would > > miss the test of tx completion part. > > There are other places that delay freeing and your solution would mean > finding and fixing all those. Code that does that already has to use > skb_orphan() to work, and I was looking for a way that could use that. > Introducing another flag value seems like a long term burden. > Get the point, will draft another version. > Alternatively, virtio could do cleanup more aggressively. Maybe in response > to ring getting half full, or add a cleanup timer or something to avoid the > problem. May worth to try. Another method is that virtio has a feature to notify guest when tx ring is empty, we could free the old tx skbs there. But it may brings extra overhead. If we could let virtio_net free the old tx skb timely, it would be easier to bring BQL support to virtio_net also. Thanks > > > > -- > To unsubscribe from this list: send the line "unsubscribe netdev" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On Wednesday, November 28, 2012 08:53:05 AM Stephen Hemminger wrote: On Wed, 28 Nov 2012 14:48:52 +0800 Jason Wang jasow...@redhat.com wrote: On 11/28/2012 12:49 AM, Stephen Hemminger wrote: On Tue, 27 Nov 2012 14:45:13 +0800 Jason Wang jasow...@redhat.com wrote: On 11/27/2012 01:37 AM, Stephen Hemminger wrote: On Mon, 26 Nov 2012 15:56:52 +0800 Jason Wang jasow...@redhat.com wrote: Some deivces do not free the old tx skbs immediately after it has been sent (usually in tx interrupt). One such example is virtio-net which optimizes for virt and only free the possible old tx skbs during the next packet sending. This would lead the pktgen to wait forever in the refcount of the skb if no other pakcet will be sent afterwards. Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY which could notify the pktgen that the device does not free skb immediately after it has been sent and let it not to wait for the refcount to be one. Signed-off-by: Jason Wang jasow...@redhat.com Another alternative would be using skb_orphan() and skb-destructor. There are other cases where skb's are not freed right away. -- To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Hi Stephen: Do you mean registering a skb-destructor for pktgen then set and check bits in skb-tx_flag? Yes. Register a destructor that does something like update a counter (number of packets pending), then just spin while number of packets pending is over threshold. -- Not sure this is the best method, since pktgen was used to test the tx process of the device driver and NIC. If we use skb_orhpan(), we would miss the test of tx completion part. There are other places that delay freeing and your solution would mean finding and fixing all those. Code that does that already has to use skb_orphan() to work, and I was looking for a way that could use that. Introducing another flag value seems like a long term burden. Get the point, will draft another version. Alternatively, virtio could do cleanup more aggressively. Maybe in response to ring getting half full, or add a cleanup timer or something to avoid the problem. May worth to try. Another method is that virtio has a feature to notify guest when tx ring is empty, we could free the old tx skbs there. But it may brings extra overhead. If we could let virtio_net free the old tx skb timely, it would be easier to bring BQL support to virtio_net also. Thanks -- To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On Thu, Nov 29, 2012 at 06:13:13PM +0800, Jason Wang wrote: On Wednesday, November 28, 2012 08:53:05 AM Stephen Hemminger wrote: On Wed, 28 Nov 2012 14:48:52 +0800 Jason Wang jasow...@redhat.com wrote: On 11/28/2012 12:49 AM, Stephen Hemminger wrote: On Tue, 27 Nov 2012 14:45:13 +0800 Jason Wang jasow...@redhat.com wrote: On 11/27/2012 01:37 AM, Stephen Hemminger wrote: On Mon, 26 Nov 2012 15:56:52 +0800 Jason Wang jasow...@redhat.com wrote: Some deivces do not free the old tx skbs immediately after it has been sent (usually in tx interrupt). One such example is virtio-net which optimizes for virt and only free the possible old tx skbs during the next packet sending. This would lead the pktgen to wait forever in the refcount of the skb if no other pakcet will be sent afterwards. Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY which could notify the pktgen that the device does not free skb immediately after it has been sent and let it not to wait for the refcount to be one. Signed-off-by: Jason Wang jasow...@redhat.com Another alternative would be using skb_orphan() and skb-destructor. There are other cases where skb's are not freed right away. -- To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Hi Stephen: Do you mean registering a skb-destructor for pktgen then set and check bits in skb-tx_flag? Yes. Register a destructor that does something like update a counter (number of packets pending), then just spin while number of packets pending is over threshold. -- Not sure this is the best method, since pktgen was used to test the tx process of the device driver and NIC. If we use skb_orhpan(), we would miss the test of tx completion part. There are other places that delay freeing and your solution would mean finding and fixing all those. Code that does that already has to use skb_orphan() to work, and I was looking for a way that could use that. Introducing another flag value seems like a long term burden. Get the point, will draft another version. Alternatively, virtio could do cleanup more aggressively. Maybe in response to ring getting half full, or add a cleanup timer or something to avoid the problem. Timer would prevent complete deadlock but it is very expensive in the virt scenario. pulling at ring half full would only help if ring gets half full :) which it does not have to. May worth to try. Another method is that virtio has a feature to notify guest when tx ring is empty, we could free the old tx skbs there. But it may brings extra overhead. If we could let virtio_net free the old tx skb timely, it would be easier to bring BQL support to virtio_net also. Thanks We used to use notify on empty - it's still very slow. -- To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On Thu, 29 Nov 2012 13:30:13 +0200 Michael S. Tsirkin m...@redhat.com wrote: On Thu, Nov 29, 2012 at 06:13:13PM +0800, Jason Wang wrote: On Wednesday, November 28, 2012 08:53:05 AM Stephen Hemminger wrote: On Wed, 28 Nov 2012 14:48:52 +0800 Jason Wang jasow...@redhat.com wrote: On 11/28/2012 12:49 AM, Stephen Hemminger wrote: On Tue, 27 Nov 2012 14:45:13 +0800 Jason Wang jasow...@redhat.com wrote: On 11/27/2012 01:37 AM, Stephen Hemminger wrote: On Mon, 26 Nov 2012 15:56:52 +0800 Jason Wang jasow...@redhat.com wrote: Some deivces do not free the old tx skbs immediately after it has been sent (usually in tx interrupt). One such example is virtio-net which optimizes for virt and only free the possible old tx skbs during the next packet sending. This would lead the pktgen to wait forever in the refcount of the skb if no other pakcet will be sent afterwards. Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY which could notify the pktgen that the device does not free skb immediately after it has been sent and let it not to wait for the refcount to be one. Signed-off-by: Jason Wang jasow...@redhat.com Another alternative would be using skb_orphan() and skb-destructor. There are other cases where skb's are not freed right away. -- To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Hi Stephen: Do you mean registering a skb-destructor for pktgen then set and check bits in skb-tx_flag? Yes. Register a destructor that does something like update a counter (number of packets pending), then just spin while number of packets pending is over threshold. -- Not sure this is the best method, since pktgen was used to test the tx process of the device driver and NIC. If we use skb_orhpan(), we would miss the test of tx completion part. There are other places that delay freeing and your solution would mean finding and fixing all those. Code that does that already has to use skb_orphan() to work, and I was looking for a way that could use that. Introducing another flag value seems like a long term burden. Get the point, will draft another version. Alternatively, virtio could do cleanup more aggressively. Maybe in response to ring getting half full, or add a cleanup timer or something to avoid the problem. Timer would prevent complete deadlock but it is very expensive in the virt scenario. pulling at ring half full would only help if ring gets half full :) which it does not have to. A timer that fires once (when idle) to mop up the last packet in a stream would be not very expensive. Most of the time, it would just be continually rescheduled. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On Wed, 28 Nov 2012 14:48:52 +0800 Jason Wang wrote: > On 11/28/2012 12:49 AM, Stephen Hemminger wrote: > > On Tue, 27 Nov 2012 14:45:13 +0800 > > Jason Wang wrote: > > > >> On 11/27/2012 01:37 AM, Stephen Hemminger wrote: > >>> On Mon, 26 Nov 2012 15:56:52 +0800 > >>> Jason Wang wrote: > >>> > Some deivces do not free the old tx skbs immediately after it has been > sent > (usually in tx interrupt). One such example is virtio-net which > optimizes for > virt and only free the possible old tx skbs during the next packet > sending. This > would lead the pktgen to wait forever in the refcount of the skb if no > other > pakcet will be sent afterwards. > > Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY which > could > notify the pktgen that the device does not free skb immediately after it > has > been sent and let it not to wait for the refcount to be one. > > Signed-off-by: Jason Wang > >>> Another alternative would be using skb_orphan() and skb->destructor. > >>> There are other cases where skb's are not freed right away. > >>> -- > >>> To unsubscribe from this list: send the line "unsubscribe netdev" in > >>> the body of a message to majord...@vger.kernel.org > >>> More majordomo info at http://vger.kernel.org/majordomo-info.html > >> Hi Stephen: > >> > >> Do you mean registering a skb->destructor for pktgen then set and check > >> bits in skb->tx_flag? > > Yes. Register a destructor that does something like update a counter > > (number of packets pending), > > then just spin while number of packets pending is over threshold. > > -- > > Not sure this is the best method, since pktgen was used to test the tx > process of the device driver and NIC. If we use skb_orhpan(), we would > miss the test of tx completion part. There are other places that delay freeing and your solution would mean finding and fixing all those. Code that does that already has to use skb_orphan() to work, and I was looking for a way that could use that. Introducing another flag value seems like a long term burden. Alternatively, virtio could do cleanup more aggressively. Maybe in response to ring getting half full, or add a cleanup timer or something to avoid the problem. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On Wed, 28 Nov 2012 14:48:52 +0800 Jason Wang jasow...@redhat.com wrote: On 11/28/2012 12:49 AM, Stephen Hemminger wrote: On Tue, 27 Nov 2012 14:45:13 +0800 Jason Wang jasow...@redhat.com wrote: On 11/27/2012 01:37 AM, Stephen Hemminger wrote: On Mon, 26 Nov 2012 15:56:52 +0800 Jason Wang jasow...@redhat.com wrote: Some deivces do not free the old tx skbs immediately after it has been sent (usually in tx interrupt). One such example is virtio-net which optimizes for virt and only free the possible old tx skbs during the next packet sending. This would lead the pktgen to wait forever in the refcount of the skb if no other pakcet will be sent afterwards. Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY which could notify the pktgen that the device does not free skb immediately after it has been sent and let it not to wait for the refcount to be one. Signed-off-by: Jason Wang jasow...@redhat.com Another alternative would be using skb_orphan() and skb-destructor. There are other cases where skb's are not freed right away. -- To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Hi Stephen: Do you mean registering a skb-destructor for pktgen then set and check bits in skb-tx_flag? Yes. Register a destructor that does something like update a counter (number of packets pending), then just spin while number of packets pending is over threshold. -- Not sure this is the best method, since pktgen was used to test the tx process of the device driver and NIC. If we use skb_orhpan(), we would miss the test of tx completion part. There are other places that delay freeing and your solution would mean finding and fixing all those. Code that does that already has to use skb_orphan() to work, and I was looking for a way that could use that. Introducing another flag value seems like a long term burden. Alternatively, virtio could do cleanup more aggressively. Maybe in response to ring getting half full, or add a cleanup timer or something to avoid the problem. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On 11/28/2012 12:49 AM, Stephen Hemminger wrote: On Tue, 27 Nov 2012 14:45:13 +0800 Jason Wang wrote: On 11/27/2012 01:37 AM, Stephen Hemminger wrote: On Mon, 26 Nov 2012 15:56:52 +0800 Jason Wang wrote: Some deivces do not free the old tx skbs immediately after it has been sent (usually in tx interrupt). One such example is virtio-net which optimizes for virt and only free the possible old tx skbs during the next packet sending. This would lead the pktgen to wait forever in the refcount of the skb if no other pakcet will be sent afterwards. Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY which could notify the pktgen that the device does not free skb immediately after it has been sent and let it not to wait for the refcount to be one. Signed-off-by: Jason Wang Another alternative would be using skb_orphan() and skb->destructor. There are other cases where skb's are not freed right away. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Hi Stephen: Do you mean registering a skb->destructor for pktgen then set and check bits in skb->tx_flag? Yes. Register a destructor that does something like update a counter (number of packets pending), then just spin while number of packets pending is over threshold. -- Not sure this is the best method, since pktgen was used to test the tx process of the device driver and NIC. If we use skb_orhpan(), we would miss the test of tx completion part. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On Tue, 27 Nov 2012 14:45:13 +0800 Jason Wang wrote: > On 11/27/2012 01:37 AM, Stephen Hemminger wrote: > > On Mon, 26 Nov 2012 15:56:52 +0800 > > Jason Wang wrote: > > > >> Some deivces do not free the old tx skbs immediately after it has been sent > >> (usually in tx interrupt). One such example is virtio-net which optimizes > >> for > >> virt and only free the possible old tx skbs during the next packet > >> sending. This > >> would lead the pktgen to wait forever in the refcount of the skb if no > >> other > >> pakcet will be sent afterwards. > >> > >> Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY which > >> could > >> notify the pktgen that the device does not free skb immediately after it > >> has > >> been sent and let it not to wait for the refcount to be one. > >> > >> Signed-off-by: Jason Wang > > Another alternative would be using skb_orphan() and skb->destructor. > > There are other cases where skb's are not freed right away. > > -- > > To unsubscribe from this list: send the line "unsubscribe netdev" in > > the body of a message to majord...@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html > Hi Stephen: > > Do you mean registering a skb->destructor for pktgen then set and check > bits in skb->tx_flag? Yes. Register a destructor that does something like update a counter (number of packets pending), then just spin while number of packets pending is over threshold. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On Tue, 27 Nov 2012 14:45:13 +0800 Jason Wang jasow...@redhat.com wrote: On 11/27/2012 01:37 AM, Stephen Hemminger wrote: On Mon, 26 Nov 2012 15:56:52 +0800 Jason Wang jasow...@redhat.com wrote: Some deivces do not free the old tx skbs immediately after it has been sent (usually in tx interrupt). One such example is virtio-net which optimizes for virt and only free the possible old tx skbs during the next packet sending. This would lead the pktgen to wait forever in the refcount of the skb if no other pakcet will be sent afterwards. Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY which could notify the pktgen that the device does not free skb immediately after it has been sent and let it not to wait for the refcount to be one. Signed-off-by: Jason Wang jasow...@redhat.com Another alternative would be using skb_orphan() and skb-destructor. There are other cases where skb's are not freed right away. -- To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Hi Stephen: Do you mean registering a skb-destructor for pktgen then set and check bits in skb-tx_flag? Yes. Register a destructor that does something like update a counter (number of packets pending), then just spin while number of packets pending is over threshold. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On 11/28/2012 12:49 AM, Stephen Hemminger wrote: On Tue, 27 Nov 2012 14:45:13 +0800 Jason Wang jasow...@redhat.com wrote: On 11/27/2012 01:37 AM, Stephen Hemminger wrote: On Mon, 26 Nov 2012 15:56:52 +0800 Jason Wang jasow...@redhat.com wrote: Some deivces do not free the old tx skbs immediately after it has been sent (usually in tx interrupt). One such example is virtio-net which optimizes for virt and only free the possible old tx skbs during the next packet sending. This would lead the pktgen to wait forever in the refcount of the skb if no other pakcet will be sent afterwards. Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY which could notify the pktgen that the device does not free skb immediately after it has been sent and let it not to wait for the refcount to be one. Signed-off-by: Jason Wang jasow...@redhat.com Another alternative would be using skb_orphan() and skb-destructor. There are other cases where skb's are not freed right away. -- To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Hi Stephen: Do you mean registering a skb-destructor for pktgen then set and check bits in skb-tx_flag? Yes. Register a destructor that does something like update a counter (number of packets pending), then just spin while number of packets pending is over threshold. -- Not sure this is the best method, since pktgen was used to test the tx process of the device driver and NIC. If we use skb_orhpan(), we would miss the test of tx completion part. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On 11/27/2012 01:37 AM, Stephen Hemminger wrote: On Mon, 26 Nov 2012 15:56:52 +0800 Jason Wang wrote: Some deivces do not free the old tx skbs immediately after it has been sent (usually in tx interrupt). One such example is virtio-net which optimizes for virt and only free the possible old tx skbs during the next packet sending. This would lead the pktgen to wait forever in the refcount of the skb if no other pakcet will be sent afterwards. Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY which could notify the pktgen that the device does not free skb immediately after it has been sent and let it not to wait for the refcount to be one. Signed-off-by: Jason Wang Another alternative would be using skb_orphan() and skb->destructor. There are other cases where skb's are not freed right away. -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Hi Stephen: Do you mean registering a skb->destructor for pktgen then set and check bits in skb->tx_flag? -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On Mon, 26 Nov 2012 15:56:52 +0800 Jason Wang wrote: > Some deivces do not free the old tx skbs immediately after it has been sent > (usually in tx interrupt). One such example is virtio-net which optimizes for > virt and only free the possible old tx skbs during the next packet sending. > This > would lead the pktgen to wait forever in the refcount of the skb if no other > pakcet will be sent afterwards. > > Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY which could > notify the pktgen that the device does not free skb immediately after it has > been sent and let it not to wait for the refcount to be one. > > Signed-off-by: Jason Wang Another alternative would be using skb_orphan() and skb->destructor. There are other cases where skb's are not freed right away. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On Mon, 26 Nov 2012 15:56:52 +0800 Jason Wang jasow...@redhat.com wrote: Some deivces do not free the old tx skbs immediately after it has been sent (usually in tx interrupt). One such example is virtio-net which optimizes for virt and only free the possible old tx skbs during the next packet sending. This would lead the pktgen to wait forever in the refcount of the skb if no other pakcet will be sent afterwards. Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY which could notify the pktgen that the device does not free skb immediately after it has been sent and let it not to wait for the refcount to be one. Signed-off-by: Jason Wang jasow...@redhat.com Another alternative would be using skb_orphan() and skb-destructor. There are other cases where skb's are not freed right away. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [net-next RFC] pktgen: don't wait for the device who doesn't free skb immediately after sent
On 11/27/2012 01:37 AM, Stephen Hemminger wrote: On Mon, 26 Nov 2012 15:56:52 +0800 Jason Wang jasow...@redhat.com wrote: Some deivces do not free the old tx skbs immediately after it has been sent (usually in tx interrupt). One such example is virtio-net which optimizes for virt and only free the possible old tx skbs during the next packet sending. This would lead the pktgen to wait forever in the refcount of the skb if no other pakcet will be sent afterwards. Solving this issue by introducing a new flag IFF_TX_SKB_FREE_DELAY which could notify the pktgen that the device does not free skb immediately after it has been sent and let it not to wait for the refcount to be one. Signed-off-by: Jason Wang jasow...@redhat.com Another alternative would be using skb_orphan() and skb-destructor. There are other cases where skb's are not freed right away. -- To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Hi Stephen: Do you mean registering a skb-destructor for pktgen then set and check bits in skb-tx_flag? -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
