Hello,
This behavior can be seen fairly easy by running netstat multiple times
in quick succession. The values reported seem to depend on the CPU on
which netstat is scheduled to run.
I have seems this in multiple machines, both physical (my laptop) and
VMs.
On an idle VM right after it booted (the output is taken on the serial
console):
# sysctl kern.version
kern.version=OpenBSD 6.1 (GENERIC.MP) #19: Fri Mar 31 13:19:19 MDT 2017
[email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP
# sysctl hw.ncpu
hw.ncpu=4
# while true; do netstat -s -p tcp | egrep '[0-9]+ packets received$'; sleep
0.1; done
2 packets received
8 packets received
8 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
8 packets received
8 packets received
8 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
8 packets received
8 packets received
8 packets received
8 packets received
8 packets received
8 packets received
^C
After I SSH into the VM to create some TCP traffic:
# while true; do netstat -s -p tcp | egrep '[0-9]+ packets received$'; sleep
0.1; done
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
108 packets received
108 packets received
108 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
0 packets received
108 packets received
108 packets received
0 packets received
0 packets received
< ... many repetitons of the '0 packets' output ... >
0 packets received
0 packets received
108 packets received
0 packets received
0 packets received
< ... many repetitons of the '0 packets' output ... >
0 packets received
0 packets received
11 packets received
0 packets received
0 packets received
0 packets received
^C
Replacing counters_enter() with counters_read() in
sys/netinet/tcp_usrreq.c tcp_sysctl_tcpstat() fixes this issue for me
(as explained in COUNTERS_ALLOC(9) ). Please see the small diff below.
However I'm not sure how this falls into the current state of the SMP
network stack work, maybe it's a known limitation but I couldn't find
any reference to it.
I've done a manual 'regression' test with the fix by capturing all the
VM traffic from the VM host and then comparing the actual packets in the
capture with the netstat -s -p tcp output and the counters are correct.
Cheers,
Andrei.
diff --git a/sys/netinet/tcp_usrreq.c b/sys/netinet/tcp_usrreq.c
index cf1e088ac72..13be425f9dd 100644
--- a/sys/netinet/tcp_usrreq.c
+++ b/sys/netinet/tcp_usrreq.c
@@ -826,15 +826,14 @@ tcp_ident(void *oldp, size_t *oldlenp, void *newp, size_t
newlen, int dodrop)
int
tcp_sysctl_tcpstat(void *oldp, size_t *oldlenp, void *newp)
{
+ uint64_t counters[tcps_ncounters];
struct tcpstat tcpstat;
- struct counters_ref cr;
- uint64_t *counters;
struct syn_cache_set *set;
int i = 0;
#define ASSIGN(field) do { tcpstat.field = counters[i++]; } while (0)
- counters = counters_enter(&cr, tcpcounters);
+ counters_read(tcpcounters, counters, nitems(counters));
ASSIGN(tcps_connattempt);
ASSIGN(tcps_accepts);
ASSIGN(tcps_connects);
@@ -932,7 +931,6 @@ tcp_sysctl_tcpstat(void *oldp, size_t *oldlenp, void *newp)
ASSIGN(tcps_sack_rexmit_bytes);
ASSIGN(tcps_sack_rcv_opts);
ASSIGN(tcps_sack_snd_opts);
- counters_leave(&cr, tcpcounters);
#undef ASSIGN
diff --git a/sys/netinet/tcp_usrreq.c b/sys/netinet/tcp_usrreq.c
index cf1e088ac72..13be425f9dd 100644
--- a/sys/netinet/tcp_usrreq.c
+++ b/sys/netinet/tcp_usrreq.c
@@ -826,15 +826,14 @@ tcp_ident(void *oldp, size_t *oldlenp, void *newp, size_t
newlen, int dodrop)
int
tcp_sysctl_tcpstat(void *oldp, size_t *oldlenp, void *newp)
{
+ uint64_t counters[tcps_ncounters];
struct tcpstat tcpstat;
- struct counters_ref cr;
- uint64_t *counters;
struct syn_cache_set *set;
int i = 0;
#define ASSIGN(field) do { tcpstat.field = counters[i++]; } while (0)
- counters = counters_enter(&cr, tcpcounters);
+ counters_read(tcpcounters, counters, nitems(counters));
ASSIGN(tcps_connattempt);
ASSIGN(tcps_accepts);
ASSIGN(tcps_connects);
@@ -932,7 +931,6 @@ tcp_sysctl_tcpstat(void *oldp, size_t *oldlenp, void *newp)
ASSIGN(tcps_sack_rexmit_bytes);
ASSIGN(tcps_sack_rcv_opts);
ASSIGN(tcps_sack_snd_opts);
- counters_leave(&cr, tcpcounters);
#undef ASSIGN
