try interface rx backpressure again

[David Gwynne]

i backed this out before the 6.5 release because of bad interactions
with virtual interfaces like vlan and trunk. those should now be fixed,
so we can try the other backpressure mechanism again.

the summary is that we count the number of attempts to queue packets for
the system to process rather than the number of packets. this should
result in a more responsive system by better limiting the work a network
taskq does per interface.

this interacted badly with things like vlan because they work on a
packet at a time, however, they directly dispatch processing now so
there's no operations that they count and get backpressure for.

ok? even if it's to throw it in the tree to be backed out when we hit a
problem again.

Index: ifq.c
===================================================================
RCS file: /cvs/src/sys/net/ifq.c,v
retrieving revision 1.31
diff -u -p -r1.31 ifq.c
--- ifq.c       16 Apr 2019 04:04:19 -0000      1.31
+++ ifq.c       12 Jun 2019 01:19:51 -0000
@@ -498,8 +498,8 @@ ifiq_destroy(struct ifiqueue *ifiq)
        ml_purge(&ifiq->ifiq_ml);
 }
 
-unsigned int ifiq_maxlen_drop = 2048 * 5;
-unsigned int ifiq_maxlen_return = 2048 * 3;
+unsigned int ifiq_pressure_drop = 16;
+unsigned int ifiq_pressure_return = 2;
 
 int
 ifiq_input(struct ifiqueue *ifiq, struct mbuf_list *ml)
@@ -508,7 +508,7 @@ ifiq_input(struct ifiqueue *ifiq, struct
        struct mbuf *m;
        uint64_t packets;
        uint64_t bytes = 0;
-       unsigned int len;
+       unsigned int pressure;
 #if NBPFILTER > 0
        caddr_t if_bpf;
 #endif
@@ -552,8 +552,8 @@ ifiq_input(struct ifiqueue *ifiq, struct
        ifiq->ifiq_packets += packets;
        ifiq->ifiq_bytes += bytes;
 
-       len = ml_len(&ifiq->ifiq_ml);
-       if (len > ifiq_maxlen_drop)
+       pressure = ++ifiq->ifiq_pressure;
+       if (pressure > ifiq_pressure_drop)
                ifiq->ifiq_qdrops += ml_len(ml);
        else
                ml_enlist(&ifiq->ifiq_ml, ml);
@@ -564,7 +564,7 @@ ifiq_input(struct ifiqueue *ifiq, struct
        else
                ml_purge(ml);
 
-       return (len > ifiq_maxlen_return);
+       return (pressure > ifiq_pressure_return);
 }
 
 void
@@ -599,6 +599,7 @@ ifiq_process(void *arg)
                return;
 
        mtx_enter(&ifiq->ifiq_mtx);
+       ifiq->ifiq_pressure = 0;
        ml = ifiq->ifiq_ml;
        ml_init(&ifiq->ifiq_ml);
        mtx_leave(&ifiq->ifiq_mtx);