Hi All!
I have run into a weird occurrence of total socket buffer drainage with
the FreeBSD network stack (IPv4).
The problem is triggered by MSIE going bezerk while rendering an
Ajax/DOM style, graphics heavy, web page. It goes into a mode of a
series of spastic connnects to the (eCos) HTTP server, request a graphic
object, followed immediately by a RST. Then a new connection, etc. The
browser manages to get the job done, at the expense of TCP connections
and - worse - the FreeBSD stack loosing socket buffers - forever!
When the network stack is void of buffers, exiting the browser only
frees *1* socket buffer. And waiting > 10 minutes does not uncover more
buffers from the depths of the stack.
While I realize that MSIE is acting up - BIG TIME - I hate that it can
cause semi-permanent damage to the operation of my system. Does anybody
have any clues as to how to uncover the leak? I have a workaround to the
browser behavior, but thats dancing around the issue, really. The
browser does dot always behave like this, but I guess I can hack up a
perl script to recreate the problem more reliantly if needed...
I have attached (part of) a Ethereal summary to display the TCP/browser
access pattern. The server is at 10.10.132.15, the client browser at
10.10.130.96. Needless to say, FF works like a champ...
---Lars
The gradual loosing of buffers (I use cyg_kmem_print_stats()):
Network stack mbuf stats:
mbufs 66, clusters 39, free clusters 2
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
VM zone 'ripcb':
Total: 64, Free: 64, Allocs: 0, Frees: 0, Fails: 0
VM zone 'tcpcb':
Total: 64, Free: 60, Allocs: 226, Frees: 222, Fails: 0
VM zone 'udpcb':
Total: 64, Free: 63, Allocs: 4, Frees: 3, Fails: 0
VM zone 'socket':
Total: 64, *Free: 21*, Allocs: 230, Frees: 187, Fails: 0
Misc mpool: total 98304, free 4192, max free block 3748
Mbufs pool: total 81792, free 73216, blocksize 128
Clust pool: total 163840, free 81920, blocksize 2048
...
Network stack mbuf stats:
mbufs 77, clusters 46, free clusters 2
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
VM zone 'ripcb':
Total: 64, Free: 64, Allocs: 0, Frees: 0, Fails: 0
VM zone 'tcpcb':
Total: 64, Free: 60, Allocs: 261, Frees: 257, Fails: 0
VM zone 'udpcb':
Total: 64, Free: 63, Allocs: 4, Frees: 3, Fails: 0
VM zone 'socket':
Total: 64, *Free: 14*, Allocs: 265, Frees: 215, Fails: 0
Misc mpool: total 98304, free 4192, max free block 3748
Mbufs pool: total 81792, free 71808, blocksize 128
Clust pool: total 163840, free 67584, blocksize 2048
...
Network stack mbuf stats:
mbufs 88, clusters 58, free clusters 2
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
VM zone 'ripcb':
Total: 64, Free: 64, Allocs: 0, Frees: 0, Fails: 0
VM zone 'tcpcb':
Total: 64, Free: 60, Allocs: 327, Frees: 323, Fails: 0
VM zone 'udpcb':
Total: 64, Free: 63, Allocs: 4, Frees: 3, Fails: 0
VM zone 'socket':
Total: 64, *Free: 2*, Allocs: 331, Frees: 269, Fails: 0
Misc mpool: total 98304, free 4192, max free block 3748
Mbufs pool: total 81792, free 70400, blocksize 128
Clust pool: total 163840, free 43008, blocksize 2048
...
Network stack mbuf stats:
mbufs 97, clusters 60, free clusters 1
Failed to get 0 times
Waited to get 0 times
Drained queues to get 0 times
VM zone 'ripcb':
Total: 64, Free: 64, Allocs: 0, Frees: 0, Fails: 0
VM zone 'tcpcb':
Total: 64, Free: 61, Allocs: 353, Frees: 350, Fails: 0
VM zone 'udpcb':
Total: 64, Free: 63, Allocs: 4, Frees: 3, Fails: 0
VM zone 'socket':
Total: 64, *Free: 0*, Allocs: 365, Frees: 293, Fails: 8
Misc mpool: total 98304, free 4192, max free block 3748
Mbufs pool: total 81792, free 69248, blocksize 128
Clust pool: total 163840, free 38912, blocksize 2048
No. Time Source Destination Protocol Info
1 0.000000 10.10.130.96 10.10.132.15 TCP 1806 >
http [ACK] Seq=0 Ack=0 Win=64255 Len=0
19 2.995609 10.10.130.96 10.10.132.15 HTTP GET
/stat/portstate HTTP/1.1
20 2.997336 10.10.132.15 10.10.130.96 HTTP
HTTP/1.1 200 OK
21 3.010650 10.10.132.15 10.10.130.96 HTTP
Continuation or non-HTTP traffic
22 3.010675 10.10.130.96 10.10.132.15 TCP 1807 >
http [ACK] Seq=289 Ack=1628 Win=65535 Len=0
23 3.011296 10.10.132.15 10.10.130.96 HTTP
Continuation or non-HTTP traffic
24 3.026878 10.10.132.15 10.10.130.96 HTTP
Continuation or non-HTTP traffic
25 3.026897 10.10.130.96 10.10.132.15 TCP 1807 >
http [ACK] Seq=289 Ack=3180 Win=65535 Len=0
26 3.027497 10.10.132.15 10.10.130.96 HTTP
Continuation or non-HTTP traffic
27 3.210248 10.10.130.96 10.10.132.15 TCP 1807 >
http [ACK] Seq=289 Ack=3196 Win=65519 Len=0
28 3.211159 10.10.132.15 10.10.130.96 HTTP
Continuation or non-HTTP traffic
29 3.221112 10.10.130.96 10.10.132.15 HTTP GET
/images/switch.png HTTP/1.1
30 3.221485 10.10.130.96 10.10.132.15 HTTP GET
/images/jack_copper_down_bottom.png HTTP/1.1
31 3.222173 10.10.130.96 10.10.132.15 TCP 1807 >
http [RST, ACK] Seq=625 Ack=3806 Win=0 Len=0
32 3.222559 10.10.130.96 10.10.132.15 TCP 1808 >
http [SYN] Seq=0 Ack=0 Win=65535 Len=0 MSS=1460
33 3.223179 10.10.132.15 10.10.130.96 TCP http >
1808 [SYN, ACK] Seq=0 Ack=1 Win=17520 Len=0 MSS=1460
34 3.223196 10.10.130.96 10.10.132.15 TCP 1808 >
http [ACK] Seq=1 Ack=1 Win=65535 Len=0
35 3.223405 10.10.130.96 10.10.132.15 HTTP GET
/images/jack_copper_down_bottom.png HTTP/1.1
36 3.223907 10.10.130.96 10.10.132.15 TCP 1808 >
http [RST, ACK] Seq=337 Ack=1 Win=0 Len=0
37 3.224241 10.10.130.96 10.10.132.15 TCP 1809 >
http [SYN] Seq=0 Ack=0 Win=65535 Len=0 MSS=1460
38 3.224586 10.10.132.15 10.10.130.96 HTTP
HTTP/1.1 304 Not Modified
39 3.225056 10.10.132.15 10.10.130.96 TCP http >
1809 [SYN, ACK] Seq=0 Ack=1 Win=17520 Len=0 MSS=1460
40 3.225071 10.10.130.96 10.10.132.15 TCP 1809 >
http [ACK] Seq=1 Ack=1 Win=65535 Len=0
41 3.228341 10.10.130.96 10.10.132.15 HTTP GET
/images/jack_copper_down_bottom.png HTTP/1.1
42 3.230077 10.10.132.15 10.10.130.96 HTTP
HTTP/1.1 304 Not Modified
43 3.241139 10.10.130.96 10.10.132.15 HTTP GET
/images/jack_copper_down_top.png HTTP/1.1
44 3.242073 10.10.130.96 10.10.132.15 TCP 1806 >
http [RST, ACK] Seq=652 Ack=80 Win=0 Len=0
45 3.243023 10.10.130.96 10.10.132.15 HTTP GET
/images/jack_copper_down_top.png HTTP/1.1
46 3.243629 10.10.130.96 10.10.132.15 TCP 1810 >
http [SYN] Seq=0 Ack=0 Win=65535 Len=0 MSS=1460
47 3.243860 10.10.130.96 10.10.132.15 TCP 1809 >
http [RST, ACK] Seq=670 Ack=81 Win=0 Len=0
48 3.244291 10.10.132.15 10.10.130.96 TCP http >
1810 [SYN, ACK] Seq=0 Ack=1 Win=17520 Len=0 MSS=1460
49 3.244308 10.10.130.96 10.10.132.15 TCP 1810 >
http [ACK] Seq=1 Ack=1 Win=65535 Len=0
50 3.244597 10.10.130.96 10.10.132.15 HTTP GET
/images/jack_copper_down_top.png HTTP/1.1
51 3.246103 10.10.130.96 10.10.132.15 TCP 1811 >
http [SYN] Seq=0 Ack=0 Win=65535 Len=0 MSS=1460
52 3.246744 10.10.132.15 10.10.130.96 TCP http >
1811 [SYN, ACK] Seq=0 Ack=1 Win=17520 Len=0 MSS=1460
53 3.246761 10.10.130.96 10.10.132.15 TCP 1811 >
http [ACK] Seq=1 Ack=1 Win=65535 Len=0
54 3.246953 10.10.130.96 10.10.132.15 HTTP GET
/images/jack_copper_down_bottom.png HTTP/1.1
55 3.247596 10.10.130.96 10.10.132.15 TCP 1811 >
http [RST, ACK] Seq=337 Ack=1 Win=0 Len=0
56 3.247950 10.10.130.96 10.10.132.15 TCP 1812 >
http [SYN] Seq=0 Ack=0 Win=65535 Len=0 MSS=1460
57 3.248097 10.10.132.15 10.10.130.96 HTTP
HTTP/1.1 304 Not Modified
58 3.248683 10.10.132.15 10.10.130.96 TCP http >
1812 [SYN, ACK] Seq=0 Ack=1 Win=17520 Len=0 MSS=1460
59 3.248698 10.10.130.96 10.10.132.15 TCP 1812 >
http [ACK] Seq=1 Ack=1 Win=65535 Len=0
60 3.248929 10.10.130.96 10.10.132.15 HTTP GET
/images/jack_copper_down_bottom.png HTTP/1.1
61 3.249787 10.10.130.96 10.10.132.15 TCP 1812 >
http [RST, ACK] Seq=337 Ack=1 Win=0 Len=0
62 3.251914 10.10.130.96 10.10.132.15 HTTP GET
/images/jack_copper_down_bottom.png HTTP/1.1
63 3.253695 10.10.132.15 10.10.130.96 HTTP
HTTP/1.1 304 Not Modified
64 3.255002 10.10.130.96 10.10.132.15 HTTP GET
/images/jack_copper_down_top.png HTTP/1.1
65 3.256753 10.10.130.96 10.10.132.15 TCP 1813 >
http [SYN] Seq=0 Ack=0 Win=65535 Len=0 MSS=1460
66 3.257415 10.10.132.15 10.10.130.96 TCP http >
1813 [SYN, ACK] Seq=0 Ack=1 Win=17520 Len=0 MSS=1460
67 3.257430 10.10.130.96 10.10.132.15 TCP 1813 >
http [RST] Seq=1 Ack=1598354149 Win=0 Len=0
68 3.257890 10.10.132.15 10.10.130.96 HTTP
HTTP/1.1 304 Not Modified
69 3.259839 10.10.130.96 10.10.132.15 HTTP GET
/images/jack_copper_down_top.png HTTP/1.1
70 3.260435 10.10.130.96 10.10.132.15 TCP 1810 >
http [RST, ACK] Seq=1336 Ack=241 Win=0 Len=0
71 3.262212 10.10.130.96 10.10.132.15 TCP 1814 >
http [SYN] Seq=0 Ack=0 Win=65535 Len=0 MSS=1460
72 3.262653 10.10.130.96 10.10.132.15 TCP 1815 >
http [SYN] Seq=0 Ack=0 Win=65535 Len=0 MSS=1460
73 3.262854 10.10.132.15 10.10.130.96 TCP http >
1814 [SYN, ACK] Seq=0 Ack=1 Win=17520 Len=0 MSS=1460
74 3.262868 10.10.130.96 10.10.132.15 TCP 1814 >
http [ACK] Seq=1 Ack=1 Win=65535 Len=0
75 3.263082 10.10.130.96 10.10.132.15 HTTP GET
/images/jack_copper_down_bottom.png HTTP/1.1
76 3.263379 10.10.132.15 10.10.130.96 TCP http >
1815 [SYN, ACK] Seq=0 Ack=1 Win=17520 Len=0 MSS=1460
77 3.263401 10.10.130.96 10.10.132.15 TCP 1815 >
http [ACK] Seq=1 Ack=1 Win=65535 Len=0
78 3.263598 10.10.130.96 10.10.132.15 HTTP GET
/images/jack_copper_down_top.png HTTP/1.1
79 3.265344 10.10.132.15 10.10.130.96 HTTP
HTTP/1.1 304 Not Modified
80 3.266120 10.10.130.96 10.10.132.15 HTTP GET
/images/jack_copper_down_bottom.png HTTP/1.1
81 3.266775 10.10.132.15 10.10.130.96 HTTP
HTTP/1.1 304 Not Modified
82 3.267176 10.10.130.96 10.10.132.15 TCP 1814 >
http [RST, ACK] Seq=673 Ack=81 Win=0 Len=0
83 3.268794 10.10.130.96 10.10.132.15 HTTP GET
/images/jack_copper_down_bottom.png HTTP/1.1
84 3.269492 10.10.130.96 10.10.132.15 TCP 1816 >
http [SYN] Seq=0 Ack=0 Win=65535 Len=0 MSS=1460
85 3.270141 10.10.132.15 10.10.130.96 TCP http >
1816 [SYN, ACK] Seq=0 Ack=1 Win=17520 Len=0 MSS=1460
86 3.270158 10.10.130.96 10.10.132.15 TCP 1816 >
http [ACK] Seq=1 Ack=1 Win=65535 Len=0
87 3.270351 10.10.130.96 10.10.132.15 HTTP GET
/images/jack_copper_link_top.png HTTP/1.1
88 3.270969 10.10.130.96 10.10.132.15 TCP 1816 >
http [RST, ACK] Seq=334 Ack=1 Win=0 Len=0
89 3.271308 10.10.130.96 10.10.132.15 TCP 1817 >
http [SYN] Seq=0 Ack=0 Win=65535 Len=0 MSS=1460
90 3.271925 10.10.132.15 10.10.130.96 TCP http >
1817 [SYN, ACK] Seq=0 Ack=1 Win=17520 Len=0 MSS=1460
91 3.271941 10.10.130.96 10.10.132.15 TCP 1817 >
http [ACK] Seq=1 Ack=1 Win=65535 Len=0
92 3.272131 10.10.130.96 10.10.132.15 HTTP GET
/images/jack_copper_link_top.png HTTP/1.1
93 3.272395 10.10.132.15 10.10.130.96 HTTP
HTTP/1.1 304 Not Modified
94 3.273240 10.10.130.96 10.10.132.15 TCP 1817 >
http [RST, ACK] Seq=334 Ack=1 Win=0 Len=0
96 3.274646 10.10.130.96 10.10.132.15 HTTP GET
/images/jack_copper_link_top.png HTTP/1.1
97 3.275243 10.10.130.96 10.10.132.15 TCP 1818 >
http [SYN] Seq=0 Ack=0 Win=65535 Len=0 MSS=1460
98 3.275870 10.10.132.15 10.10.130.96 TCP http >
1818 [SYN, ACK] Seq=0 Ack=1 Win=17520 Len=0 MSS=1460
99 3.275887 10.10.130.96 10.10.132.15 TCP 1818 >
http [ACK] Seq=1 Ack=1 Win=65535 Len=0
100 3.276077 10.10.130.96 10.10.132.15 HTTP GET
/images/jack_copper_down_bottom.png HTTP/1.1
--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss
