On Wed, Jan 16, 2013 at 01:38:01PM +1100, Darren Tucker wrote:
> On Mon, Jan 14, 2013 at 10:10:55PM +1100, Darren Tucker wrote:
> > On my ALIX, it increase the IP routing throughput from 80Mbit/s to
> > 85Mbit/s while reducing the interrupt CPU usage from 99% to 80%.
>
> It turns out that due to an error on my part, most of this improvment
> was due to one of the test kernels being complied without POOL_DEBUG
> while the "baseline" one was. Sigh. I think there's still some gains
> to be had, though.
OK, here's another diff which does seem to help. It seems that there's
two different bits in the TX descriptor that control interrupts. Quoting
from the VT6105M spec:
TDES1 bit 23: "Interrupt Control. 0: No interrupt when Transmit OK, 1:
Interrupt when Transmit OK"
TDES3 bit 0: "Interrupt Control. 0: issue interrupt for this packet. 1:
no interrupt generated"
Why two bits with apparently similar functionality? Beats me, but only
the second actually seems to do anything on my VT6105M here. CPU usage
for routing 85 mbit/s drops CPU usage from about 75% to about 50%
(famous last words). Your milage may vary and so on.
Index: if_vr.c
===================================================================
RCS file: /cvs/src/sys/dev/pci/if_vr.c,v
retrieving revision 1.124
diff -u -p -r1.124 if_vr.c
--- if_vr.c 16 Jan 2013 06:15:50 -0000 1.124
+++ if_vr.c 17 Jan 2013 04:15:05 -0000
@@ -724,7 +724,7 @@ vr_list_tx_init(struct vr_softc *sc)
cd = &sc->vr_cdata;
ld = sc->vr_ldata;
- cd->vr_tx_cnt = 0;
+ cd->vr_tx_cnt = cd->vr_tx_pkts = 0;
for (i = 0; i < VR_TX_LIST_CNT; i++) {
cd->vr_tx_chain[i].vr_ptr = &ld->vr_tx_list[i];
@@ -1292,7 +1292,15 @@ vr_encap(struct vr_softc *sc, struct vr_
}
/* Set EOP on the last descriptor */
- f->vr_ctl |= htole32(VR_TXCTL_LASTFRAG | VR_TXCTL_FINT);
+ f->vr_ctl |= htole32(VR_TXCTL_LASTFRAG);
+
+ /* set the disable-interrupt bit except on every Nth packet */
+ if (sc->vr_cdata.vr_tx_pkts++ % 16)
+ f->vr_next |= htole32(VR_TXNEXT_INTDISABLE);
+ else {
+ sc->vr_cdata.vr_tx_pkts = 0;
+ f->vr_ctl |= htole32(VR_TXCTL_FINT);
+ }
return (0);
}
@@ -1581,6 +1589,11 @@ vr_watchdog(struct ifnet *ifp)
struct vr_softc *sc;
sc = ifp->if_softc;
+
+ /* Reclaim first as we don't request an interrupt for every packet. */
+ vr_txeof(sc);
+ if (sc->vr_cdata.vr_tx_cnt == 0);
+ return;
ifp->if_oerrors++;
printf("%s: watchdog timeout\n", sc->sc_dev.dv_xname);
Index: if_vrreg.h
===================================================================
RCS file: /cvs/src/sys/dev/pci/if_vrreg.h,v
retrieving revision 1.33
diff -u -p -r1.33 if_vrreg.h
--- if_vrreg.h 16 Jan 2013 05:25:57 -0000 1.33
+++ if_vrreg.h 17 Jan 2013 04:15:05 -0000
@@ -429,6 +429,9 @@ struct vr_desc {
#define VR_TXCTL_LASTFRAG 0x00400000
#define VR_TXCTL_FINT 0x00800000
+/* TDES3 aka vr_next */
+#define VR_TXNEXT_INTDISABLE 0x00000001
+
#define VR_MAXFRAGS 8
#define VR_RX_LIST_CNT 128
#define VR_TX_LIST_CNT 128
@@ -467,6 +470,7 @@ struct vr_chain_data {
struct vr_chain *vr_tx_cons;
struct vr_chain *vr_tx_prod;
int vr_tx_cnt;
+ int vr_tx_pkts;
};
struct vr_mii_frame {
--
Darren Tucker (dtucker at zip.com.au)
GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4 37C9 C982 80C7 8FF4 FA69
Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.
