Andrei-Marius Radu <[email protected]> writes:
> Hello,
>
> This behavior can be seen fairly easy by running netstat multiple times
> in quick succession. The values reported seem to depend on the CPU on
> which netstat is scheduled to run.
>
> I have seems this in multiple machines, both physical (my laptop) and
> VMs.
>
> On an idle VM right after it booted (the output is taken on the serial
> console):
>
> # sysctl kern.version
> kern.version=OpenBSD 6.1 (GENERIC.MP) #19: Fri Mar 31 13:19:19 MDT 2017
> [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
>
> # sysctl hw.ncpu
> hw.ncpu=4
>
> # while true; do netstat -s -p tcp | egrep '[0-9]+ packets received$'; sleep
> 0.1; done
> 2 packets received
> 8 packets received
> 8 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 8 packets received
> 8 packets received
> 8 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 8 packets received
> 8 packets received
> 8 packets received
> 8 packets received
> 8 packets received
> 8 packets received
> ^C
>
> After I SSH into the VM to create some TCP traffic:
>
> # while true; do netstat -s -p tcp | egrep '[0-9]+ packets received$'; sleep
> 0.1; done
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 108 packets received
> 108 packets received
> 108 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> 108 packets received
> 108 packets received
> 0 packets received
> 0 packets received
> < ... many repetitons of the '0 packets' output ... >
> 0 packets received
> 0 packets received
> 108 packets received
> 0 packets received
> 0 packets received
> < ... many repetitons of the '0 packets' output ... >
> 0 packets received
> 0 packets received
> 11 packets received
> 0 packets received
> 0 packets received
> 0 packets received
> ^C
>
> Replacing counters_enter() with counters_read() in
> sys/netinet/tcp_usrreq.c tcp_sysctl_tcpstat() fixes this issue for me
> (as explained in COUNTERS_ALLOC(9) ). Please see the small diff below.
> However I'm not sure how this falls into the current state of the SMP
> network stack work, maybe it's a known limitation but I couldn't find
> any reference to it.
>
> I've done a manual 'regression' test with the fix by capturing all the
> VM traffic from the VM host and then comparing the actual packets in the
> capture with the netstat -s -p tcp output and the counters are correct.
Thanks for the detailed report, I have committed your fix.
> Cheers,
> Andrei.
>
> diff --git a/sys/netinet/tcp_usrreq.c b/sys/netinet/tcp_usrreq.c
> index cf1e088ac72..13be425f9dd 100644
> --- a/sys/netinet/tcp_usrreq.c
> +++ b/sys/netinet/tcp_usrreq.c
> @@ -826,15 +826,14 @@ tcp_ident(void *oldp, size_t *oldlenp, void *newp,
> size_t newlen, int dodrop)
> int
> tcp_sysctl_tcpstat(void *oldp, size_t *oldlenp, void *newp)
> {
> + uint64_t counters[tcps_ncounters];
> struct tcpstat tcpstat;
> - struct counters_ref cr;
> - uint64_t *counters;
> struct syn_cache_set *set;
> int i = 0;
>
> #define ASSIGN(field) do { tcpstat.field = counters[i++]; } while (0)
>
> - counters = counters_enter(&cr, tcpcounters);
> + counters_read(tcpcounters, counters, nitems(counters));
> ASSIGN(tcps_connattempt);
> ASSIGN(tcps_accepts);
> ASSIGN(tcps_connects);
> @@ -932,7 +931,6 @@ tcp_sysctl_tcpstat(void *oldp, size_t *oldlenp, void
> *newp)
> ASSIGN(tcps_sack_rexmit_bytes);
> ASSIGN(tcps_sack_rcv_opts);
> ASSIGN(tcps_sack_snd_opts);
> - counters_leave(&cr, tcpcounters);
>
> #undef ASSIGN
>
>
--
jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF DDCC 0DFA 74AE 1524 E7EE
