On 27/09/2013 11:30, "Mike Belopuhov" <[email protected]> wrote:
>On 27 September 2013 12:21, Stuart Henderson <[email protected]> wrote: >> On 2013/09/27 10:09, Thomas Greer wrote: >>> Hi All >>> >>> I'm seeing high CPU usage with bgpd session engine, and this was >>>knocking out >>> all my routing. The only way to get routing back is to pill the bgpd >>>and then >>> start it again. sthen suggested I ktraced it and the output is below. >> >> To clarify from an IRC discussion (tgreer please correct me if I'm >>wrong), >> AIUI this was running OK until announcements are made on a particular >>peer >> session, just bringing the session up without making announcements >>doesn't >> trigger it. Same happens when either announcing just a default route or >> when announcing full table. >> >> >>> This is on 5.3 however same issues experienced on 5.2. Different >>>hardware >>> also. >>> >>> Peer is running bird. Versions: 1.3.8 and 1.3.11 >>> >>> Kdump: >>> >>> 6326 bgpd EMUL "native" >>> 6326 bgpd RET sendmsg -1 errno 35 Resource temporarily >>>unavailable >>> 6326 bgpd CALL sendmsg(0x9,0x7f7ffffe7080,0) >> >> EAGAIN - send(2) says "The socket is marked non-blocking and the >>requested >> operation would block." >> >> Anyone have clues? >> >>> 6326 bgpd RET sendmsg -1 errno 35 Resource temporarily >>>unavailable >>> 6326 bgpd CALL sendmsg(0x9,0x7f7ffffe7080,0) >>> 6326 bgpd RET sendmsg -1 errno 35 Resource temporarily >>>unavailable >>> 6326 bgpd CALL sendmsg(0x9,0x7f7ffffe7080,0) >>> 6326 bgpd RET sendmsg -1 errno 35 Resource temporarily >>>unavailable >>> 6326 bgpd CALL sendmsg(0x9,0x7f7ffffe7080,0) >>> 6326 bgpd RET sendmsg -1 errno 35 Resource temporarily >>>unavailable >>> 6326 bgpd CALL sendmsg(0x9,0x7f7ffffe7080,0) >>> 6326 bgpd RET sendmsg -1 errno 35 Resource temporarily >>>unavailable >>> 6326 bgpd CALL sendmsg(0x9,0x7f7ffffe7080,0) >>> 6326 bgpd RET sendmsg -1 errno 35 Resource temporarily >>>unavailable >>> 6326 bgpd CALL sendmsg(0x9,0x7f7ffffe7080,0) >>> 6326 bgpd RET sendmsg -1 errno 35 Resource temporarily >>>unavailable >>> > >how was fd 0x9 obtained? Not sure how to answer thisŠ does the info below help? This is taken during it's 'broken' state -bash-4.2# netstat -m 93 mbufs in use: 79 mbufs allocated to data 7 mbufs allocated to packet headers 7 mbufs allocated to socket names and addresses 68/96/6144 mbuf 2048 byte clusters in use (current/peak/max) 0/8/6144 mbuf 4096 byte clusters in use (current/peak/max) 0/8/6144 mbuf 8192 byte clusters in use (current/peak/max) 0/8/6144 mbuf 9216 byte clusters in use (current/peak/max) 0/8/6144 mbuf 12288 byte clusters in use (current/peak/max) 0/8/6144 mbuf 16384 byte clusters in use (current/peak/max) 0/8/6144 mbuf 65536 byte clusters in use (current/peak/max) 440 Kbytes allocated to network (36% in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines -bash-4.2# fstat | grep bgpd _bgpd bgpd 7250 text /usr 77969 -r-xr-xr-x r 284152 _bgpd bgpd 7250 wd /var 623616 drwxr-xr-x r 512 _bgpd bgpd 7250 root /var 623616 drwxr-xr-x r 512 _bgpd bgpd 7250 0 / 27022 crw-rw-rw- rw null _bgpd bgpd 7250 1 / 27022 crw-rw-rw- rw null _bgpd bgpd 7250 2 / 27022 crw-rw-rw- rw null _bgpd bgpd 7250 3* unix dgram 0xffff800001f81380 <-> 0xffff800000cc7c00 _bgpd bgpd 7250 4* internet stream tcp 0xfffffe807c3dc460 *:179 _bgpd bgpd 7250 5* unix stream 0xffff800001f81300 <-> 0xffff800001f81e00 _bgpd bgpd 7250 6* internet6 stream tcp 0xfffffe807c3dc690 *:179 _bgpd bgpd 7250 7* unix stream 0xffff800001f81800 _bgpd bgpd 7250 8* unix stream 0xffff800000ccfe00 <-> 0xffff800000ccf000 _bgpd bgpd 7250 10* unix stream 0xffff800001f81f80 <-> 0xffff800001f81b00 _bgpd bgpd 7250 12* 30 raw 2 0xfffffe80561133e8 _bgpd bgpd 7250 15* internet stream tcp 0xfffffe807c003470 91.232.181.2:16998 --> 91.232.181.40:179 _bgpd bgpd 7250 54* internet6 stream tcp 0xfffffe807b8bf6b0 [2a02:1648:0:36::102]:41376 --> [2a02:1648:0:36::1]:179 _bgpd bgpd 7250 56* internet stream tcp 0xfffffe807b8bf8e0 188.94.40.75:29107 --> 188.94.40.73:179 _bgpd bgpd 7250 57* internet stream tcp 0xfffffe807b8bfb10 188.94.40.69:37732 --> 188.94.40.65:179 _bgpd bgpd 5255 text /usr 77969 -r-xr-xr-x r 284152 _bgpd bgpd 5255 wd /var 623616 drwxr-xr-x r 512 _bgpd bgpd 5255 root /var 623616 drwxr-xr-x r 512 _bgpd bgpd 5255 0 / 27022 crw-rw-rw- rw null _bgpd bgpd 5255 1 / 27022 crw-rw-rw- rw null _bgpd bgpd 5255 2 / 27022 crw-rw-rw- rw null _bgpd bgpd 5255 3* unix dgram 0xffff800001f81380 <-> 0xffff800000cc7c00 _bgpd bgpd 5255 7* unix stream 0xffff800000ccfc00 <-> 0xffff800000cf8800 _bgpd bgpd 5255 9* unix stream 0xffff800000ccf000 <-> 0xffff800000ccfe00 _bgpd bgpd 5255 11* unix stream 0xffff800001f81b00 <-> 0xffff800001f81f80 root bgpd 13885 text /usr 77969 -r-xr-xr-x r 284152 root bgpd 13885 wd / 77954 drwx------ r 512 root bgpd 13885 0 / 27022 crw-rw-rw- rw null root bgpd 13885 1 / 27022 crw-rw-rw- rw null root bgpd 13885 2 / 27022 crw-rw-rw- rw null root bgpd 13885 3* unix dgram 0xffff800001f81380 <-> 0xffff800000cc7c00 root bgpd 13885 4* unix stream 0xffff800001f81e00 <-> 0xffff800001f81300 root bgpd 13885 5* route raw 0 0xfffffe8056113970 root bgpd 13885 6* unix stream 0xffff800000cf8800 <-> 0xffff800000ccfc00 root bgpd 13885 10* unix stream 0xffff800001f81f80 <-> 0xffff800001f81b00 root bgpd 13885 11* unix stream 0xffff800001f81b00 <-> 0xffff800001f81f80
