[Bug 206932] Realtek 8111 card stops responding under high load in netmap mode

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206932

Mark Linimon  changed:

   What|Removed |Added

 Resolution|--- |Overcome By Events
 Status|In Progress |Closed

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Bug 206932] Realtek 8111 card stops responding under high load in netmap mode

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206932

--- Comment #10 from hippi-vik...@protonmail.com ---
Hi Vincenzo,

Sorry for the late reply! I was stress-testing the firewall and so far it seems
tha above was just a one-off issue unrelated to load or Suricata. I continued
to see the same errors with very little to no load at all even with Suricata
stopped. After a reboot the card seems stable (no errors) even when loading it
as much as I can (with the slow upstream cable connection I have).
I think it probably might be safe to close off this issue.

Thank you for your help!
Cheers,
Istvan

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Bug 206932] Realtek 8111 card stops responding under high load in netmap mode

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206932

--- Comment #9 from Vincenzo Maffione  ---
(In reply to hippi-viking from comment #8)

Sounds good. If you provide suricata configuration files I can try to reproduce
your issue...
Also please check the netmap version coming with your HardenedBSD, as I don't
know how much that is aligned with FreeBSD.

The fact that you are using the official Realtek drivers explain why it looks
like your re0 has 1024 descriptors (RX and TX), while the driver in FreeBSD
only allows for 256 (non configurable).

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Bug 206932] Realtek 8111 card stops responding under high load in netmap mode

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206932

--- Comment #8 from hippi-vik...@protonmail.com ---
(In reply to Vincenzo Maffione from comment #7)

Hello :)

I suspect this issue is caused by suricata for me running in IPS mode. I don't
know the exact command line used but will look into it - and run your code as
well - when I will have some more time.

The background is that this is a Fujitsu Futro S900 thin client running
Opnsense - which is based on HardenedBSD, being close to vanilla FreeBSD in
this issue, but(?) supposedly using the latest official Realtek driver - I
don't know exactly which re0 driver FreeBSD is using at the moment. The
integrated re0 nic (RTL8111E according to dmidecode) is connected to the ISP's
cable modem at a 100Mbps full-duplex connection. The upline connection is
10/2Mbps, I get the kring errors when downloading at full-speed.

Thanks for looking into this!
Cheers,
Istvan

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Bug 206932] Realtek 8111 card stops responding under high load in netmap mode

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206932

--- Comment #7 from Vincenzo Maffione  ---
(In reply to hippi-viking from comment #6)

Hi,
 Can you please specify the exact command line (e.g. pkt-gen if you are using
pkt-gen) that causes the issue?

Also, can you build and run this program and reports its output? It will just
print the number of rings and slots.


#include 
#include 
#include 
#define NETMAP_WITH_LIBS
#include 
#include 

int main(int argc, char **argv)
{
const char *ifname = argv[1];
struct nm_desc *nmd;
char name[100];

assert(argc >= 2);
snprintf(name, sizeof(name), "netmap:%s", ifname);
nmd = nm_open(name, NULL, 0, NULL);
assert(nmd);
printf("txr %u rxr %u txd %u rxd %u\n",
nmd->nifp->ni_tx_rings,
nmd->nifp->ni_rx_rings,
NETMAP_TXRING(nmd->nifp, 0)->num_slots,
NETMAP_RXRING(nmd->nifp, 0)->num_slots);
nm_close(nmd);

return 0;
}



Command line
  # ./info re0

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Bug 206932] Realtek 8111 card stops responding under high load in netmap mode

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206932

hippi-vik...@protonmail.com changed:

   What|Removed |Added

 CC||hippi-vik...@protonmail.com

--- Comment #6 from hippi-vik...@protonmail.com ---
(In reply to Vincenzo Maffione from comment #5)
I still can see this issue under load on FreeBSD 11.2-RELEASE-p9-HBSD with an
integrated Realtek RTL8111/8168/8411 chip:

re0@pci0:1:0:0: class=0x02 card=0x11c01734 chip=0x816810ec rev=0x06
hdr=0x00
vendor = 'Realtek Semiconductor Co., Ltd.'
device = 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller'
class  = network
subclass   = ethernet

351.711973 [1613] netmap_ring_reinitcalled for re0 RX0
002.919057 [1613] netmap_ring_reinitcalled for re0 RX0
908.475302 [1512] nm_txsync_prologuere0 TX1 kring error: head 765 cur
766 tail 765 rhead 766 rcur 766 rtail 765 hwcur 766 hwtail 765
908.488304 [1613] netmap_ring_reinitcalled for re0 TX1
177.477234 [1613] netmap_ring_reinitcalled for re0 RX0
603.209901 [1613] netmap_ring_reinitcalled for re0 RX1
873.183945 [1512] nm_txsync_prologuere0 TX1 kring error: head 5 cur 6
tail 5 rhead 6 rcur 6 rtail 5 hwcur 6 hwtail 5
873.195708 [1613] netmap_ring_reinitcalled for re0 TX1
912.229349 [1613] netmap_ring_reinitcalled for re0 RX1
921.732103 [1512] nm_txsync_prologuere0 TX0 kring error: head 148 cur
150 tail 148 rhead 150 rcur 150 rtail 148 hwcur 150 hwtail 148
921.744464 [1613] netmap_ring_reinitcalled for re0 TX0
931.657772 [1613] netmap_ring_reinitcalled for re0 RX0
931.727492 [1512] nm_txsync_prologuere0 TX1 kring error: head 734 cur
735 tail 734 rhead 735 rcur 735 rtail 734 hwcur 735 hwtail 734
931.746382 [1613] netmap_ring_reinitcalled for re0 TX1
605.399805 [1613] netmap_ring_reinitcalled for re0 RX1
043.944163 [1613] netmap_ring_reinitcalled for re0 RX0
942.272725 [1512] nm_txsync_prologuere0 TX1 kring error: head 771 cur
773 tail 772 rhead 773 rcur 773 rtail 772 hwcur 773 hwtail 772
942.286046 [1613] netmap_ring_reinitcalled for re0 TX1
597.788061 [1512] nm_txsync_prologuere0 TX1 kring error: head 330 cur
331 tail 330 rhead 331 rcur 331 rtail 330 hwcur 331 hwtail 330
597.800517 [1613] netmap_ring_reinitcalled for re0 TX1
921.844295 [1512] nm_txsync_prologuere0 TX1 kring error: head 448 cur
449 tail 448 rhead 449 rcur 449 rtail 448 hwcur 449 hwtail 448
921.857727 [1613] netmap_ring_reinitcalled for re0 TX1
929.117950 [1613] netmap_ring_reinitcalled for re0 RX0
955.503793 [1512] nm_txsync_prologuere0 TX1 kring error: head 694 cur
695 tail 694 rhead 695 rcur 695 rtail 694 hwcur 695 hwtail 694
955.517123 [1613] netmap_ring_reinitcalled for re0 TX1
054.130772 [1613] netmap_ring_reinitcalled for re0 RX0
239.912053 [1613] netmap_ring_reinitcalled for re0 RX1
426.000205 [1613] netmap_ring_reinitcalled for re0 RX0
431.708618 [1512] nm_txsync_prologuere0 TX0 kring error: head 482 cur
483 tail 481 rhead 483 rcur 483 rtail 481 hwcur 483 hwtail 481
431.721230 [1613] netmap_ring_reinitcalled for re0 TX0
466.558021 [1613] netmap_ring_reinitcalled for re0 RX0
505.101756 [1613] netmap_ring_reinitcalled for re0 RX0
626.222834 [1613] netmap_ring_reinitcalled for re0 RX1
745.757168 [1613] netmap_ring_reinitcalled for re0 RX1
128.048022 [1512] nm_txsync_prologuere0 TX0 kring error: head 126 cur
126 tail 125 rhead 127 rcur 127 rtail 125 hwcur 127 hwtail 125
128.060369 [1613] netmap_ring_reinitcalled for re0 TX0
301.754435 [1613] netmap_ring_reinitcalled for re0 RX1
305.059117 [1613] netmap_ring_reinitcalled for re0 RX0
476.838054 [1512] nm_txsync_prologuere0 TX1 kring error: head 1023 cur
0 tail 1023 rhead 0 rcur 0 rtail 1023 hwcur 0 hwtail 1023
476.850660 [1613] netmap_ring_reinitcalled for re0 TX1
810.251369 [1613] netmap_ring_reinitcalled for re0 RX0
973.553584 [1613] netmap_ring_reinitcalled for re0 RX1
991.498730 [1613] netmap_ring_reinitcalled for re0 RX0
016.098769 [1613] netmap_ring_reinitcalled for re0 RX1
068.708621 [1613] netmap_ring_reinitcalled for re0 RX1
072.092164 [1613] netmap_ring_reinitcalled for re0 RX1
128.526783 [1512] nm_txsync_prologuere0 TX0 kring error: head 706 cur
707 tail 705 rhead 707 rcur 707 rtail 705 hwcur 707 hwtail 705
128.539348 [1613] netmap_ring_reinitcalled for re0 TX0
157.709591 [1613] netmap_ring_reinitcalled for re0 RX1
167.110939 [1613] netmap_ring_reinitcalled for re0 RX1
176.127982 [1512] nm_txsync_prologuere0 TX0 kring error: head 155 cur
156 tail 154 rhead 156 rcur 156 rtail 154 hwcur 156 hwtail 154

[Bug 206932] Realtek 8111 card stops responding under high load in netmap mode

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206932

Vincenzo Maffione  changed:

   What|Removed |Added

 CC||vmaffi...@freebsd.org
 Status|New |In Progress

--- Comment #5 from Vincenzo Maffione  ---
Current netmap code in HEAD, stable/11 and stable/12 is aligned to the github
(and code has changed quite a lot since 2016).
I just tried to run pkt-gen (tx or rx) in a VM with a r8169 emulated NIC, and
everything seems to work fine to me.

Can you check if the issue is still there?

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Bug 206932] Realtek 8111 card stops responding under high load in netmap mode

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206932

--- Comment #4 from Olivier - interfaSys sàrl  
---
Using fresh netmap from FreeBSD 11 and a newer pkt-gen, this is what I see.

986.519903 [2163] netmap_ioctl  nr_cmd must be 0 not 12
047.486179 [1481] nm_txsync_prologuefail head < kring->rhead || head >
kring->rtail
047.510386 [1511] nm_txsync_prologuere0 TX0 kring error: head 107 cur
107 tail 106 rhead 52 rcur 52 rtail 106 hwcur 52 hwtail 106
047.534818 [1612] netmap_ring_reinitcalled for re0 TX0
051.945718 [1481] nm_txsync_prologuefail head < kring->rhead || head >
kring->rtail
051.990215 [1511] nm_txsync_prologuere0 TX0 kring error: head 225 cur
225 tail 224 rhead 223 rcur 223 rtail 224 hwcur 223 hwtail 224
052.009143 [1612] netmap_ring_reinitcalled for re0 TX0

At this point pkt-gen exits with error. 

I've also tried using the netmap software emulation and it crashes even
earlier.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Bug 206932] Realtek 8111 card stops responding under high load in netmap mode

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206932

--- Comment #3 from Olivier - interfaSys sàrl  
---
I think this is logged when things start failing
```
231.020147 [2925] netmap_transmit   re0 full hwcur 0 hwtail 0 qlen 255
len 42 m 0xf80052005400
235.997171 [2925] netmap_transmit   re0 full hwcur 0 hwtail 0 qlen 255
len 42 m 0xf80023ec9c00
240.989245 [2925] netmap_transmit   re0 full hwcur 0 hwtail 0 qlen 255
len 42 m 0xf800521ad500
247.887586 [2925] netmap_transmit   re0 full hwcur 0 hwtail 0 qlen 255
len 42 m 0xf80023da9b00
253.069781 [2925] netmap_transmit   re0 full hwcur 0 hwtail 0 qlen 255
len 42 m 0xf80023ec7700
258.110746 [2925] netmap_transmit   re0 full hwcur 0 hwtail 0 qlen 255
len 42 m 0xf800521ade00
263.188076 [2925] netmap_transmit   re0 full hwcur 0 hwtail 0 qlen 255
len 42 m 0xf800237d6900
```

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Bug 206932] Realtek 8111 card stops responding under high load in netmap mode

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206932

--- Comment #2 from Olivier - interfaSys sàrl  
---
Setting re0 to use a MTU of 9000 and the connection stays alive. Instead of
timing out, the packet rate drops drastically once and things go back to
normal. 
The main difference in netstat is that the mbuf clusters are split between
standard and jumbo frames

```
768/2787/3555 mbufs in use (current/cache/total)
256/1524/1780/500200 mbuf clusters in use (current/cache/total/max)
256/1515 mbuf+clusters out of packet secondary zone in use (current/cache)
0/46/46/250099 4k (page size) jumbo clusters in use (current/cache/total/max)
256/65/321/74103 9k jumbo clusters in use (current/cache/total/max)
0/0/0/41683 16k jumbo clusters in use (current/cache/total/max)
3008K/4513K/7521K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
```

The rate in vmstat keeps rising, but that doesn't seem to be a problem

```
interrupt  total   rate
irq16: sdhci_pci0  1  0
cpu0:timer   3008083   1113
irq256: ahci0  10125  3
irq257: xhci0  11363  4
irq258: hdac0  3  0
irq259: re0 13105929   4850
irq260: re1   101440 37
cpu2:timer   1095578405
cpu1:timer   1083354400
cpu3:timer   1123144415
Total   19539020   7231
```

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Bug 206932] Realtek 8111 card stops responding under high load in netmap mode

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206932

Mark Linimon  changed:

   What|Removed |Added

   Assignee|freebsd-b...@freebsd.org|freebsd-net@FreeBSD.org

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"

[Bug 206932] Realtek 8111 card stops responding under high load in netmap mode

[bugzilla-noreply]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206932

Olivier - interfaSys sàrl  changed:

   What|Removed |Added

Version|10.2-STABLE |11.0-CURRENT

--- Comment #1 from Olivier - interfaSys sàrl  
---
I've just tested on 11-CURRENT and got the same results.

-- 
You are receiving this mail because:
You are the assignee for the bug.
___
freebsd-net@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"