Re: FreeBSD: syslog-ng: I/O error occurred while writing; fd='xx', error='No buffer space available (yy)'
On 22 Mar 2012, at 11:40, Traiano Welcome wrote: That's what I thought as well, but it's the details that evade me. Almost all traffic to and from this server is UDP (syslog), the graph I sent earlier shows the kind of volumes and trends that are typical: Peak traffic during the problem periods averages at about 1 Mbps outbound and 200 Kbps inbound to/from the interface. The interface itself is a Embedded Broadcom 5708 NIC on a Dell PowerEdge 1950. Here are a couple of netstat polls during one of the problem periods: [root@syslog2]# date;netstat -p udp -s |egrep -w (received|delivered|dropped) Thu Mar 22 12:11:34 SAST 2012 19969 datagrams received 2 dropped due to no socket 0 dropped due to full socket buffers 19967 delivered . . . [root@syslog2~]# date;netstat -p udp -s |egrep -w (received|delivered|dropped) Thu Mar 22 13:36:46 SAST 2012 662385 datagrams received 118 dropped due to no socket 0 dropped due to full socket buffers 662267 delivered --- Somehow this doesn't strike me as a large volume of throughput … Ok, fair enough. You might try simulating the problem by deliberately overloading the syslog UDP output and confirm the cause. - Mark___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: FreeBSD: syslog-ng: I/O error occurred while writing; fd='xx', error='No buffer space available (yy)'
Hi Mark On 22/03/2012 13:54, Mark Blackman m...@exonetric.com wrote: On 22 Mar 2012, at 11:40, Traiano Welcome wrote: That's what I thought as well, but it's the details that evade me. Almost all traffic to and from this server is UDP (syslog), the graph I sent earlier shows the kind of volumes and trends that are typical: Peak traffic during the problem periods averages at about 1 Mbps outbound and 200 Kbps inbound to/from the interface. The interface itself is a Embedded Broadcom 5708 NIC on a Dell PowerEdge 1950. Here are a couple of netstat polls during one of the problem periods: [root@syslog2]# date;netstat -p udp -s |egrep -w (received|delivered|dropped) Thu Mar 22 12:11:34 SAST 2012 19969 datagrams received 2 dropped due to no socket 0 dropped due to full socket buffers 19967 delivered . . . [root@syslog2~]# date;netstat -p udp -s |egrep -w (received|delivered|dropped) Thu Mar 22 13:36:46 SAST 2012 662385 datagrams received 118 dropped due to no socket 0 dropped due to full socket buffers 662267 delivered --- Somehow this doesn't strike me as a large volume of throughput Š Ok, fair enough. You might try simulating the problem by deliberately overloading the syslog UDP output and confirm the cause. Apparently this means that the network driver has filled up with packets. John Baldwin over at freebsd-net@ advises I up the number of descriptors assigned to igb to the maximum to workaround this using the hw.igb.maxtxd tunable you would set. So I've rebooted with the following in loader.conf: hw.igb.rxd=4096 hw.igb.txd=4096 This seems to be working so far. What I've noticed is that the system is using far less RAM than previously, and CPU utilisation is up to 100% of one core, load average is 1, which I would guess means that the system is now processing a lot more syslog data now that more packets are making it through the network driver. I'll keep monitoring over a 24 hour period though, to see how effective this is. - Mark ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: FreeBSD: syslog-ng: I/O error occurred while writing; fd='xx', error='No buffer space available (yy)'
On 23 Mar 2012, at 08:58, Traiano Welcome wrote: Hi Mark On 22/03/2012 13:54, Mark Blackman m...@exonetric.com wrote: On 22 Mar 2012, at 11:40, Traiano Welcome wrote: Somehow this doesn't strike me as a large volume of throughput Š Ok, fair enough. You might try simulating the problem by deliberately overloading the syslog UDP output and confirm the cause. Apparently this means that the network driver has filled up with packets. John Baldwin over at freebsd-net@ advises I up the number of descriptors assigned to igb to the maximum to workaround this using the hw.igb.maxtxd tunable you would set. So I've rebooted with the following in loader.conf: hw.igb.rxd=4096 hw.igb.txd=4096 This seems to be working so far. What I've noticed is that the system is using far less RAM than previously, and CPU utilisation is up to 100% of one core, load average is 1, which I would guess means that the system is now processing a lot more syslog data now that more packets are making it through the network driver. I'll keep monitoring over a 24 hour period though, to see how effective this is. Right, good news. Interesting that you need to tweak network drivers. - Mark ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
FreeBSD: syslog-ng: I/O error occurred while writing; fd='xx', error='No buffer space available (yy)'
Hi List I've been seeing the following in the messages log of my freebsd syslog server for quite some time now: --- Mar 20 12:19:12 syslog2 syslog-ng[35313]: I/O error occurred while writing; fd='12', error='No buffer space available (55)' Mar 20 12:19:12 syslog2 syslog-ng[35313]: Connection broken; time_reopen='60' Mar 20 12:19:12 syslog2 syslog-ng[35313]: I/O error occurred while writing; fd='13', error='No buffer space available (55)' Mar 20 12:19:12 syslog2 syslog-ng[35313]: Connection broken; time_reopen='60' --- These happen at a frequency of about 7 per minute on average. See attached trend graphs for an idea of the volume of traffic we're doing, as well as the memory and cpu utilisation trends on this server during this period. As can be seen from the graphs, load does not seem to be the issue. Occasionally during the week, the system freezes and requires a reboot, I think it's related to the above message, though I'm not sure. My question is: What does this error mean, and how can I resolve it? I have tried to frame this as an operating system kernel resource issue, and experimented with increasing the freebsd kernel sysctls for UDP performance: --- [root@syslog2 mailto:r...@syslog2.ops.mtnbusiness.co.za /var/log]# sysctl kern.ipc.nmbclusters=102400 kern.ipc.nmbclusters: 25600 - 102400 [root@syslog2 mailto:r...@syslog2.ops.mtnbusiness.co.za /var/log]# sysctl kern.ipc.maxsockbuf=201326592 kern.ipc.maxsockbuf: 100663296 - 201326592 [root@syslog2 mailto:r...@syslog2.ops.mtnbusiness.co.za /var/log]# sysctl net.inet.udp.recvspace=33554432 net.inet.udp.recvspace: 16777216 - 33554432 --- This has reduced the frequency of the errors a little, but in general the problem still remains. Syslog version: -- [root@syslog2]# syslog-ng -V syslog-ng 2.0.10 -- FreeBSD version: -- FreeBSD 7.2-STABLE #0 -- Any help would be much appreciated! Traiano ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: FreeBSD: syslog-ng: I/O error occurred while writing; fd='xx', error='No buffer space available (yy)'
On 03/22/12 19:00, Traiano Welcome wrote: Hi List I've been seeing the following in the messages log of my freebsd syslog server for quite some time now: --- Mar 20 12:19:12 syslog2 syslog-ng[35313]: I/O error occurred while writing; fd='12', error='No buffer space available (55)' Mar 20 12:19:12 syslog2 syslog-ng[35313]: Connection broken; time_reopen='60' Mar 20 12:19:12 syslog2 syslog-ng[35313]: I/O error occurred while writing; fd='13', error='No buffer space available (55)' Mar 20 12:19:12 syslog2 syslog-ng[35313]: Connection broken; time_reopen='60' --- These happen at a frequency of about 7 per minute on average. See attached trend graphs for an idea of the volume of traffic we're doing, as well as the memory and cpu utilisation trends on this server during this period. As can be seen from the graphs, load does not seem to be the issue. Occasionally during the week, the system freezes and requires a reboot, I think it's related to the above message, though I'm not sure. My question is: What does this error mean, and how can I resolve it? I have tried to frame this as an operating system kernel resource issue, and experimented with increasing the freebsd kernel sysctls for UDP performance: --- [root@syslog2mailto:r...@syslog2.ops.mtnbusiness.co.za /var/log]# sysctl kern.ipc.nmbclusters=102400 kern.ipc.nmbclusters: 25600 - 102400 [root@syslog2mailto:r...@syslog2.ops.mtnbusiness.co.za /var/log]# sysctl kern.ipc.maxsockbuf=201326592 kern.ipc.maxsockbuf: 100663296 - 201326592 [root@syslog2mailto:r...@syslog2.ops.mtnbusiness.co.za /var/log]# sysctl net.inet.udp.recvspace=33554432 net.inet.udp.recvspace: 16777216 - 33554432 --- This has reduced the frequency of the errors a little, but in general the problem still remains. Syslog version: -- [root@syslog2]# syslog-ng -V syslog-ng 2.0.10 -- FreeBSD version: -- FreeBSD 7.2-STABLE #0 -- Any help would be much appreciated! I'm sorry I can't shed some light on a solution, but this happens on ping and some other network related apps and tools for me too; just not often enough for me concern with atm due to higher priorities. Perhaps net@ might be a better resource for an answer to this one? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: FreeBSD: syslog-ng: I/O error occurred while writing; fd='xx', error='No buffer space available (yy)'
On 22 Mar 2012, at 09:00, Traiano Welcome wrote: My question is: What does this error mean, and how can I resolve it? From a very casual inspection of the problem, I'd say you're pushing out syslog messages faster than the kernel can get them out the interface. How many syslog messages are going in (per second) and what kind of network interface are you trying to send them out through? I have tried to frame this as an operating system kernel resource issue, and experimented with increasing the freebsd kernel sysctls for UDP performance: I think you can push nmbclusters up to about 600k, but if your input is running faster than your output, no amount of buffering will permanently stave off this problem. - Mark ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: FreeBSD: syslog-ng: I/O error occurred while writing; fd='xx', error='No buffer space available (yy)'
Hi Mark On 22/03/2012 11:52, Mark Blackman m...@exonetric.com wrote: On 22 Mar 2012, at 09:00, Traiano Welcome wrote: My question is: What does this error mean, and how can I resolve it? From a very casual inspection of the problem, I'd say you're pushing out syslog messages faster than the kernel can get them out the interface. How many syslog messages are going in (per second) and what kind of network interface are you trying to send them out through? That's what I thought as well, but it's the details that evade me. Almost all traffic to and from this server is UDP (syslog), the graph I sent earlier shows the kind of volumes and trends that are typical: Peak traffic during the problem periods averages at about 1 Mbps outbound and 200 Kbps inbound to/from the interface. The interface itself is a Embedded Broadcom 5708 NIC on a Dell PowerEdge 1950. Here are a couple of netstat polls during one of the problem periods: [root@syslog2]# date;netstat -p udp -s |egrep -w (received|delivered|dropped) Thu Mar 22 12:11:34 SAST 2012 19969 datagrams received 2 dropped due to no socket 0 dropped due to full socket buffers 19967 delivered . . . [root@syslog2~]# date;netstat -p udp -s |egrep -w (received|delivered|dropped) Thu Mar 22 13:36:46 SAST 2012 662385 datagrams received 118 dropped due to no socket 0 dropped due to full socket buffers 662267 delivered --- Somehow this doesn't strike me as a large volume of throughput ... I have tried to frame this as an operating system kernel resource issue, and experimented with increasing the freebsd kernel sysctls for UDP performance: I think you can push nmbclusters up to about 600k, but if your input is running faster than your output, no amount of buffering will permanently stave off this problem. I've done that just in the last 2 hours, though I agree with you that this is probably a (very) temporary imrovement. - Mark ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
No buffer space available on igb0
Hi # uname -a FreeBSD 9.0-STABLE FreeBSD 9.0-STABLE #7: Sat Jan 7 00:24:06 EET 2012 @:/usr/obj/usr/src/sys/KES_KERN_v9 i386 igb stops to work. saing 'No bufferspace available' no messages in /var/log/messages.log, /var/log/console.log, dmesg is empty tcpdump -n -i igb0 shows nothing. pull out LAN cable from igb0 card: # ifconfig igb0 igb0: flags=8d43UP,BROADCAST,RUNNING,PROMISC,OACTIVE,SIMPLEX,MULTICAST metric 0 mtu 1500 options=401bbRXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO ether 00:1b:21:45:da:b8 inet6 fe80::21b:21ff:fe45:dab8%igb0 prefixlen 64 scopeid 0x1 nd6 options=29PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL media: Ethernet autoselect (1000baseT full-duplex) status: active but expected: 'status: no carrier' other interface in system re0 continue to work fine. CAn you help me what is wrong? -- С уважением, Коньков mailto:kes-...@yandex.ru ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: No buffer space available on igb0
additional info sysctl -a | grep igb IGB Core Lock,igb0:tx(0) igb0:tx(0),bpf interface lock igb0:tx(0),system map igb0:tx(0),UMA zone IGB Core Lock,igb0:tx(1) igb0:tx(1),system map igb0:tx(1),UMA zone IGB Core Lock,igb0:tx(2) igb0:tx(2),bpf interface lock igb0:tx(2),UMA zone IGB Core Lock,igb0:tx(3) igb0:tx(3),UMA zone IGB Core Lock,igb0:rx(3) igb0:rx(3),system map igb0:rx(3),UMA zone igb0:rx(3),UMA boot pages IGB Core Lock,igb1:tx(0) igb1:tx(0),bpf interface lock IGB Core Lock,igb1:tx(1) IGB Core Lock,igb1:tx(2) IGB Core Lock,igb1:tx(3) IGB Core Lock,igb1:rx(3) igb1:rx(3),system map igb1:rx(3),UMA zone hw.igb.rx_process_limit: 100 hw.igb.num_queues: 0 hw.igb.header_split: 0 hw.igb.max_interrupt_rate: 8000 hw.igb.enable_msix: 1 hw.igb.enable_aim: 1 hw.igb.txd: 1024 hw.igb.rxd: 1024 dev.igb.0.%desc: Intel(R) PRO/1000 Network Connection version - 2.2.5 dev.igb.0.%driver: igb dev.igb.0.%location: slot=0 function=0 dev.igb.0.%pnpinfo: vendor=0x8086 device=0x10c9 subvendor=0x8086 subdevice=0xa01c class=0x02 dev.igb.0.%parent: pci1 dev.igb.0.nvm: -1 dev.igb.0.enable_aim: 1 dev.igb.0.fc: 65536003 dev.igb.0.rx_processing_limit: 100 dev.igb.0.link_irq: 2 dev.igb.0.dropped: 0 dev.igb.0.tx_dma_fail: 0 dev.igb.0.rx_overruns: 0 dev.igb.0.watchdog_timeouts: 0 dev.igb.0.device_control: 1086325313 dev.igb.0.rx_control: 67141634 dev.igb.0.interrupt_mask: 4 dev.igb.0.extended_int_mask: 2147483648 dev.igb.0.tx_buf_alloc: 0 dev.igb.0.rx_buf_alloc: 0 dev.igb.0.fc_high_water: 58976 dev.igb.0.fc_low_water: 58960 dev.igb.0.queue0.no_desc_avail: 0 dev.igb.0.queue0.tx_packets: 25405982 dev.igb.0.queue0.rx_packets: 6059861 dev.igb.0.queue0.rx_bytes: 0 dev.igb.0.queue0.lro_queued: 0 dev.igb.0.queue0.lro_flushed: 0 dev.igb.0.queue1.no_desc_avail: 0 dev.igb.0.queue1.tx_packets: 115976 dev.igb.0.queue1.rx_packets: 6541438 dev.igb.0.queue1.rx_bytes: 0 dev.igb.0.queue1.lro_queued: 0 dev.igb.0.queue1.lro_flushed: 0 dev.igb.0.queue2.no_desc_avail: 0 dev.igb.0.queue2.tx_packets: 97730 dev.igb.0.queue2.rx_packets: 13955306 dev.igb.0.queue2.rx_bytes: 0 dev.igb.0.queue2.lro_queued: 0 dev.igb.0.queue2.lro_flushed: 0 dev.igb.0.queue3.no_desc_avail: 0 dev.igb.0.queue3.tx_packets: 402421 dev.igb.0.queue3.rx_packets: 11549500 dev.igb.0.queue3.rx_bytes: 0 dev.igb.0.queue3.lro_queued: 0 dev.igb.0.queue3.lro_flushed: 0 dev.igb.0.mac_stats.excess_coll: 0 dev.igb.0.mac_stats.single_coll: 0 dev.igb.0.mac_stats.multiple_coll: 0 dev.igb.0.mac_stats.late_coll: 0 dev.igb.0.mac_stats.collision_count: 0 dev.igb.0.mac_stats.symbol_errors: 0 dev.igb.0.mac_stats.sequence_errors: 0 dev.igb.0.mac_stats.defer_count: 0 dev.igb.0.mac_stats.missed_packets: 4434 dev.igb.0.mac_stats.recv_no_buff: 0 dev.igb.0.mac_stats.recv_undersize: 0 dev.igb.0.mac_stats.recv_fragmented: 0 dev.igb.0.mac_stats.recv_oversize: 0 dev.igb.0.mac_stats.recv_jabber: 0 dev.igb.0.mac_stats.recv_errs: 0 dev.igb.0.mac_stats.crc_errs: 0 dev.igb.0.mac_stats.alignment_errs: 0 dev.igb.0.mac_stats.coll_ext_errs: 0 dev.igb.0.mac_stats.xon_recvd: 0 dev.igb.0.mac_stats.xon_txd: 0 dev.igb.0.mac_stats.xoff_recvd: 0 dev.igb.0.mac_stats.xoff_txd: 0 dev.igb.0.mac_stats.total_pkts_recvd: 38126667 dev.igb.0.mac_stats.good_pkts_recvd: 38106904 dev.igb.0.mac_stats.bcast_pkts_recvd: 10354 dev.igb.0.mac_stats.mcast_pkts_recvd: 11 dev.igb.0.mac_stats.rx_frames_64: 510017 dev.igb.0.mac_stats.rx_frames_65_127: 3263713 dev.igb.0.mac_stats.rx_frames_128_255: 870429 dev.igb.0.mac_stats.rx_frames_256_511: 313001 dev.igb.0.mac_stats.rx_frames_512_1023: 459884 dev.igb.0.mac_stats.rx_frames_1024_1522: 32689860 dev.igb.0.mac_stats.good_octets_recvd: 48232959266 dev.igb.0.mac_stats.good_octets_txd: 8584384267 dev.igb.0.mac_stats.total_pkts_txd: 26021552 dev.igb.0.mac_stats.good_pkts_txd: 26021552 dev.igb.0.mac_stats.bcast_pkts_txd: 34 dev.igb.0.mac_stats.mcast_pkts_txd: 0 dev.igb.0.mac_stats.tx_frames_64: 9280722 dev.igb.0.mac_stats.tx_frames_65_127: 10937932 dev.igb.0.mac_stats.tx_frames_128_255: 701935 dev.igb.0.mac_stats.tx_frames_256_511: 122992 dev.igb.0.mac_stats.tx_frames_512_1023: 242761 dev.igb.0.mac_stats.tx_frames_1024_1522: 4735210 dev.igb.0.mac_stats.tso_txd: 449 dev.igb.0.mac_stats.tso_ctx_fail: 0 dev.igb.0.interrupts.asserts: 50371217 dev.igb.0.interrupts.rx_pkt_timer: 38106551 dev.igb.0.interrupts.rx_abs_timer: 0 dev.igb.0.interrupts.tx_pkt_timer: 0 dev.igb.0.interrupts.tx_abs_timer: 38106110 dev.igb.0.interrupts.tx_queue_empty: 26019865 dev.igb.0.interrupts.tx_queue_min_thresh: 0 dev.igb.0.interrupts.rx_desc_min_thresh: 0 dev.igb.0.interrupts.rx_overrun: 0 dev.igb.0.host.breaker_tx_pkt: 0 dev.igb.0.host.host_tx_pkt_discard: 0 dev.igb.0.host.rx_pkt: 353 dev.igb.0.host.breaker_rx_pkts: 0 dev.igb.0.host.breaker_rx_pkt_drop: 0 dev.igb.0.host.tx_good_pkt: 1687 dev.igb.0.host.breaker_tx_pkt_drop: 0 dev.igb.0.host.rx_good_bytes: 48232960754 dev.igb.0.host.tx_good_bytes: 8584384267 dev.igb.0.host.length_errors: 0 dev.igb.0.host.serdes_violation_pkt: 0 dev.igb.0.host.header_redir_missed: 0
reexec socketpair: No buffer space available
Hi all, Has anyone come across this error before: sshd[20861]: error: reexec socketpair: No buffer space available It stops remote users/services to connect to the machine remotely. I have 17 jails running on the machine. CPU load on the machine is low, and RAM usage is also low (16GB avail, 5GB Active) System info: FreeBSD 8.0-RELEASE-p6 FreeBSD 8.0-RELEASE-p6 $ netstat -mb 1029/4101/5130 mbufs in use (current/cache/total) 514/3446/3960/65536 mbuf clusters in use (current/cache/total/max) 0/1536 mbuf+clusters out of packet secondary zone in use (current/cache) 0/1582/1582/12800 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 1285K/14245K/15530K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 373686 requests for I/O initiated by sendfile 0 calls to protocol drain routines What else should I be looking for to help me trouble this? I know I'm hitting some limit, but not sure which one. Any help would be much appreciated. Thanks, Henry ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: reexec socketpair: No buffer space available
On Tue, Nov 22, 2011 at 11:50 AM, Henry M henr...@gmail.com wrote: Hi all, Has anyone come across this error before: sshd[20861]: error: reexec socketpair: No buffer space available It stops remote users/services to connect to the machine remotely. I have 17 jails running on the machine. CPU load on the machine is low, and RAM usage is also low (16GB avail, 5GB Active) System info: FreeBSD 8.0-RELEASE-p6 FreeBSD 8.0-RELEASE-p6 $ netstat -mb 1029/4101/5130 mbufs in use (current/cache/total) 514/3446/3960/65536 mbuf clusters in use (current/cache/total/max) 0/1536 mbuf+clusters out of packet secondary zone in use (current/cache) 0/1582/1582/12800 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 1285K/14245K/15530K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 373686 requests for I/O initiated by sendfile 0 calls to protocol drain routines What else should I be looking for to help me trouble this? I know I'm hitting some limit, but not sure which one. Any help would be much appreciated. http://unix.derkeiler.com/Mailing-Lists/FreeBSD/stable/2007-03/msg00764.html So sysctl's like net.inet.tcp.recvbuf_max might play role here, but I would be curious as to what is the reason this comes up in the first place. Is the traffic to system valid, and hence raising the limits would be the correct course of action. Or do you have some deeper problem which limit raising would only mask or temporarily alleviate the issue. -- Adam Vande More ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: ping: sendto: No buffer space available
Hi, Erik. You can not find out what buffer you have overflow. This error message cover many network buffers, sadly In my case I have some fortune I try and get that error disappeared by trying these: kern.ipc.maxsockbuf=1048576 net.graph.maxdgram=524288 net.graph.recvspace=524288 I do not know what that mean, but that work. beleave me Another sad thing there is no man, no documentations, no any FAQ for all sysctl variables, but you can find some info in mail lists Good luck. Вы писали 24 апреля 2010 г., 14:06:37: EN Hi! EN I'm running FreeBSD 8.0. Some times my network just go down without EN leaving any errors behind, now this morning it went down but didn't cut EN my ssh connection to the box and I got this error: EN ping: sendto: No buffer space available EN From what I have found this relates to protocols like udp and icmp, I EN assume this can occur with p2p but also vpn protocols like l2tp. EN Is there some way that I can set limits on these protocols such that EN they will not use up all available buffer space? Or some way to increase EN buffer? EN Or is the problem something completely different? I've got two vr EN interfaces on a VIA Nehemiah ITX. EN Thanks, Erik -- С уважением, Eugen Konkov mailto:kes-...@yandex.ru http://kes.net.ua ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
ping: sendto: No buffer space available
Hi! I'm running FreeBSD 8.0. Some times my network just go down without leaving any errors behind, now this morning it went down but didn't cut my ssh connection to the box and I got this error: ping: sendto: No buffer space available From what I have found this relates to protocols like udp and icmp, I assume this can occur with p2p but also vpn protocols like l2tp. Is there some way that I can set limits on these protocols such that they will not use up all available buffer space? Or some way to increase buffer? Or is the problem something completely different? I've got two vr interfaces on a VIA Nehemiah ITX. Thanks, Erik -- Erik Nørgaard Ph: +34.666334818/+34.915211157 http://www.locolomo.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: ping: sendto: No buffer space available
I almost forgot! And if you find out the reason for shortage you can tweak it with the appropiate sysctl value. At the moment I'm not sure which value you should tweak, but if you search for this issue, maybe you can find the appropiate net. values. Regards, MB. On 24 April 2010 22:35, Balázs Mátéffy repcs...@gmail.com wrote: Hello, I had a similar problem sometimes on one or two of my machines, look up netstat -m, usually if you run out of buffer space you have to tweak the mbuf memory size. You can see the memory usage current / cache / total, if the current or cache is the same value as the total, you have memory shortage. You can search for it, there are plenty of mail list archives about issue like this. Hope this helps! Best Regards, MB. On 24 April 2010 13:06, Erik Norgaard norga...@locolomo.org wrote: Hi! I'm running FreeBSD 8.0. Some times my network just go down without leaving any errors behind, now this morning it went down but didn't cut my ssh connection to the box and I got this error: ping: sendto: No buffer space available From what I have found this relates to protocols like udp and icmp, I assume this can occur with p2p but also vpn protocols like l2tp. Is there some way that I can set limits on these protocols such that they will not use up all available buffer space? Or some way to increase buffer? Or is the problem something completely different? I've got two vr interfaces on a VIA Nehemiah ITX. Thanks, Erik -- Erik Nørgaard Ph: +34.666334818/+34.915211157 http://www.locolomo.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: ping: sendto: No buffer space available
Hello, I had a similar problem sometimes on one or two of my machines, look up netstat -m, usually if you run out of buffer space you have to tweak the mbuf memory size. You can see the memory usage current / cache / total, if the current or cache is the same value as the total, you have memory shortage. You can search for it, there are plenty of mail list archives about issue like this. Hope this helps! Best Regards, MB. On 24 April 2010 13:06, Erik Norgaard norga...@locolomo.org wrote: Hi! I'm running FreeBSD 8.0. Some times my network just go down without leaving any errors behind, now this morning it went down but didn't cut my ssh connection to the box and I got this error: ping: sendto: No buffer space available From what I have found this relates to protocols like udp and icmp, I assume this can occur with p2p but also vpn protocols like l2tp. Is there some way that I can set limits on these protocols such that they will not use up all available buffer space? Or some way to increase buffer? Or is the problem something completely different? I've got two vr interfaces on a VIA Nehemiah ITX. Thanks, Erik -- Erik Nørgaard Ph: +34.666334818/+34.915211157 http://www.locolomo.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: No buffer space available
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Thiago ... What version of kernel did you end up going back to? - --On Wednesday, April 04, 2007 10:15:48 -0300 Marc G. Fournier free...@hub.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I'm seeing the same effect (haven't tried older kernel, mind you) almost like clockwork, every 72 hours after reboot ... at least now I don't feel so crazy, knowing it isn't just me ... - --On Sunday, April 01, 2007 17:07:08 -0300 Thiago Esteves de Oliveira thi...@lamce.coppe.ufrj.edu.br wrote: I've tried to increase the kern.ipc.nmbclusters value but it worked only when I changed the kernel to an older one. netstat -m (Now it's working with the same values.) - 515/850/1365 mbufs in use (current/cache/total) 512/390/902/65024 mbuf clusters in use (current/cache/total/max) 512/243 mbuf+clusters out of packet secondary zone in use (current/cache) 0/0/0/0 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/0 9k jumbo clusters in use (current/cache/total/max) 0/0/0/0 16k jumbo clusters in use (current/cache/total/max) 1152K/992K/2145K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 2759 requests for I/O initiated by sendfile 2982 calls to protocol drain routines Ethernet adapters - em0: Intel(R) PRO/1000 Network Connection Version - 6.0.5 port 0xec80-0xecbf m em 0xfebe-0xfebf irq 10 at device 4.0 on pci7 em0: Ethernet address: 00:04:23:c3:06:78 em0: [FAST] skc0: 3Com 3C940 Gigabit Ethernet port 0xe800-0xe8ff mem 0xfebd8000-0xfebdbfff irq 15 at device 6.0 on pci7 skc0: 3Com Gigabit NIC (3C2000) rev. (0x1) sk0: Marvell Semiconductor, Inc. Yukon on skc0 sk0: Ethernet address: 00:0a:5e:65:ad:c3 miibus0: MII bus on sk0 e1000phy0: Marvell 88E1000 Gigabit PHY on miibus0 e1000phy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX-FDX, auto P.S.: I am using the FreeBSD/amd64. Brian A. Seklecki wrote: Show us netstat -m on the broken kernel? Show us your dmesg(8) for em(4). TIA, ~BAS On Fri, 2007-03-30 at 11:13 -0300, Thiago Esteves de Oliveira wrote: Hello, I've had a problem with one of my FreeBSD servers, the machine has stopped its network services and then sent these messages: -Mar 27 13:00:03 anubis dhcpd: send_packet: No buffer space available -Mar 27 13:00:26 anubis routed[431]: Send bcast sendto(em0, 146.164.92.255.520): No buffer space available The messages were repeated a lot of times before a temporary solution. I've changed the kernel(FreeBSD 6.2) to an older one(FreeBSD 6.1) and since then it's been working well. What happened? P.S.: I can give more informations if necessary. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . scra...@hub.org MSN . scra...@hub.org Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (FreeBSD) iD8DBQFGE6UE4QvfyHIvDvMRAlutAJ0WzVTYq99hmx1km2mdXE7pdUC8IgCgt4O1 eG6kXgqHveumXjkL0t+Q8Q8= =sieE -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . scra...@hub.org MSN . scra...@hub.org Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (FreeBSD) iD8DBQFGE/ZC4QvfyHIvDvMRAsWoAJwJpD8nCtG0iv5U6LY8ISyyDKxgegCg1eti SezStun7CLDA9pgfrp8GloM= =UwSU -END PGP SIGNATURE- ___ freebsd-sta...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: vde qemu write: No buffer space available
On Friday 26 June 2009 10:04:13 Adam Vande More wrote: I'm having issues with high network throughput with vde and qemu. There are two qemu vm's each running debian lenny and they are configured for drbd. The vm's work fine until drbd is started then the networking fails. The only message I get(on the host side) is write: No buffer space available It can be a driver issue, but the error message is somewhat misleading as it can be the result of an ill-configured firewall rule. Typically this happens when no state exists for the outgoing connection. which is echoed to console, and nothing appears in the logs. I believe this to be an issue with vde, but I can't be certain because I can't seem to find any way to turn on more extensive logging. Anyone have an idea how to resolve this? You should check netstat -m to make sure there are mbufs available and if there is check your firewall. If all seems ok, try freebsd-net list for any known issues, since you didn't get any me too's here. You may want to specify a bit more info, like pciconf -lv for the vde device, vmstat -i at the time of the errors, ifconfig vde0 output and any firewall information (including I don't have one or error persists if firewall is disabled). -- Mel ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
vde qemu write: No buffer space available
I'm having issues with high network throughput with vde and qemu. There are two qemu vm's each running debian lenny and they are configured for drbd. The vm's work fine until drbd is started then the networking fails. The only message I get(on the host side) is write: No buffer space available which is echoed to console, and nothing appears in the logs. I believe this to be an issue with vde, but I can't be certain because I can't seem to find any way to turn on more extensive logging. Anyone have an idea how to resolve this? Thanks, -- Adam Vande More ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
kern.ipc.maxsockets and FIN_WAIT_2: No buffer space available
Hi list, I've got some problems with full sockets on one FreeBSD 6.2 system acting as a loadbalancer for a webfarm. Sometimes I get some errors like these from different daemons: haproxy[46932]: Proxy my_proxy reached system memory limit at 83 sockets. Please check system tunables. stunnel: LOG3[45738:139512832]: remote socket: No buffer space available (55) netstat -m looks fine: 491/874/1365 mbufs in use (current/cache/total) 450/618/1068/25600 mbuf clusters in use (current/cache/total/max) 450/490 mbuf+clusters out of packet secondary zone in use (current/cache) 0/0/0/0 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/0 9k jumbo clusters in use (current/cache/total/max) 0/0/0/0 16k jumbo clusters in use (current/cache/total/max) 1022K/1454K/2477K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/8/6656 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 7696 calls to protocol drain routines But this looks bad: # sysctl kern.ipc.numopensockets kern.ipc.numopensockets: 11301 # sysctl kern.ipc.maxsockets kern.ipc.maxsockets: 12328 After raising kern.ipc.maxsockets up to 16384 the errors disappeared, for now. Some further research gave me the following result: # netstat -n | grep -c FIN_WAIT_2 11156 Hmm, strange. All the connections go to (Debian Linux)-HTTP-Nodes. But I don't know why the connections don't close. On the Debian Linux system there are lots of sockets in LAST_ACK state. Any ideas what could cause these problems and how I could solve them? Can I set a timeout for the FIN_WAIT_2 state on the FreeBSD system, so the sockets won't fill up with unused connections waiting for termination? I also looked at all tcp4 sockets in netstat -n output. The number of these sockets is higher than kern.ipc.numopensockets at the same time. I think the number should be lower than kern.ipc.numopensockets because all tcp4 sockets are only a part of all sockets, right? Thanks, Matthias ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
No buffer space available
Hello, Since moving over ftp traffic to a 6-STABLE from 9/20/2007 to a machine of ours, we've been getting the above errors in the logs. Obviously the machine becomes unresponsive from the network and requires a console to log in and reboot. I generally can fix these types of problems rather quickly (or thought I did), as I've handled these problems before in the past quite frequently. However, this particular machine is giving me a really hard time. I have to reboot the machine every 2ish weeks due to the above. It's my hopes that after reading through the output that follows, someone can point out a crucial piece that I am missing...cause I am stumped. With the above said, and while looking through tons of output, I came across what I believe to 'be the gem'. I'm hoping that this can be either confirmed or denied: ITEM SIZE LIMIT USED FREE REQUESTS FAILURES mbuf_cluster: 2048, 64000, 1024, 10, 1024, 0## While machine is borked mbuf_cluster: 2048, 64000, 1532, 246, 1823214, 0## While the machine is not borked The above is output from vmstat -z obviously trimmed just to show the specific lines. The first line quite frankly makes no sense to me whatsoever. In fact, it's ?artifically? stuck at 1024 for both the 'used' and 'requests' fields. Formatting bug? or integer overflow of some kind? Maybe..but it's ironic that the network is locking up at the same time. That and the values simply don't add up. Additionally, I have netstat -m output that follows which again shows strange values for requests for I/O initiated by sendfile and calls to protocol drain routines: --- 522/888/1410 mbufs in use (current/cache/total) 516/518/1034/64000 mbuf clusters in use (current/cache/total/max) 516/508 mbuf+clusters out of packet secondary zone in use (current/cache) 0/0/0/0 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/0 9k jumbo clusters in use (current/cache/total/max) 0/0/0/0 16k jumbo clusters in use (current/cache/total/max) 1162K/1258K/2420K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/5/8704 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines --- 0 ?? Those items mentioned above are counters which increment extremely slowly. I can't imagine this ever being an integer rollover type of problem. Something is weird here as well. --- Lastly, em0 shows no errors to speak of: NameMtu Network Address Ipkts IerrsOpkts Oerrs Coll em01500 Link#1 00:0e:0c:b1:a7:0e23104 027905 0 0 01:00:5e:00:00:01 744 0 --- Would certainly appreciate any help whether in the form of links, patches, or other non aggressive types of responses ;) Thanks, Paul ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Syslog warnings: 15 x No buffer space available
Probably you shoul look to: netstat -m 4/1421/1425 mbufs in use (current/cache/total) 0/614/614/25600 mbuf clusters in use (current/cache/total/max) netstat -Lan give you a link to good article. It`s in russian, but you can see it for commands and sysctl variables which may help you. http://www.opennet.ru/base/net/tune_freebsd.txt.html Anyone who could point me to how to remedy this? Thanks, --per ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Syslog warnings: 15 x No buffer space available
Anyone who could point me to how to remedy this? Thanks, --per ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: No buffer space available error
Nejc Škoberne-2 wrote: Hello, I've been trying to solve this problem by myself for a long time now, but no luck. I run a few dozens of FreeBSD 5.3/5.4 machines, which serve as routers, NAT boxes, Apache, Postfix, OpenVPN, ... servers. Most of them are low-cost PC machines since they are usually deployed to SOHO environments and the loads are rather low. I am having problems with the No buffer space available error like this: Jul 18 08:49:36 Router openvpn[661]: write UDPv4: No buffer space available (code=55) so this is obviously when OpenVPN tries to send UDP packets. And also like this: Jun 23 06:27:38 Router pdns[2182]: Unable to send a packet to our recursing backend: No buffer space available when PowerDNS DNS server tries to do some recursive work. I have been searching Google for a solution and I found out that the error should appear when the mbuf (or sfbuf?) is full and that I can print the current buffer status with 'netstat -m'. Because the error would show up (and not only show up, but also block the network operability for that server) at random times, I set up the swatch daemon on all those servers, so that as soon as the error is logged in messages, I run this command: #!/usr/local/bin/bash LOG=/var/log/swatch.log datum=`date` echo == $datum === sockstat $LOG echo $LOG netstat -n -a $LOG echo $LOG netstat -m $LOG echo $LOG ps ax $LOG echo $LOG Even though the log was growing as I assumed, I couldn't find anything particulary interesting, because the netstat -m command issued by swatch (at the time of the error) still shows something like this: 2 mbufs in use 1/17088 mbuf clusters in use (current/max) 0/6/4528 sfbufs in use (current/peak/max) 2 KBytes allocated to network 0 requests for sfbufs denied 0 requests for sfbufs delayed 1819 requests for I/O initiated by sendfile 7578 calls to protocol drain routines I am not sure, but as I understand it, this means that the buffers are quite OK. What would be the proper way to debug this problem? This is happening on machines with various hardware, from good old Pentium I with 32 MB RAM up to P4 3GHz, 1GB RAM, various network cards (mostly rtl8139), with ADSL or VDSL, although the errors are very rare at the VDSL boxes (where the upstream bandwidth is substantially greater). So, usually the errors appear but the users don't bother really, so it looks like the problems goes away sometimes (the connection is restored), but sometimes reboot is needed. Thanks for your ideas. P.S.: If the output of the script above could be helpful, let me know, I can publish it somewhere. Cheers, Nejc Hello Nejc! Have you managed to solve this? I've just been having the same issue - I've set up multiple OpenVPN connections with TAP-device (FreeBSD-5.3 as server, 5.4 as client, multiple 4.x as clients etc.) and the server gives the same error regularly - I can't restart the server, the error occures every 30 minutes or so.. Do you know what's the deal with these buffers? My netstat -m shows: 520 mbufs in use 515/128000 mbuf clusters in use (current/max) 0/5/6656 sfbufs in use (current/peak/max) 1160 KBytes allocated to network 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 9814 calls to protocol drain routines -- View this message in context: http://www.nabble.com/No-buffer-space-available-error-tf1960830.html#a13644937 Sent from the freebsd-questions mailing list archive at Nabble.com. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: No buffer space available
On Sat, Apr 07, 2007 at 08:20:58PM -0300, Marc G. Fournier wrote: - --On Saturday, April 07, 2007 20:12:00 +0100 Chris [EMAIL PROTECTED] wrote: Also to add I now have a 2nd box using 6.2 STABLE few days old code, had to use it because of broadcom 5755 nic card, I plan to use large tcp window sizes so will be interesting to see if this also suffers from the problem. I've got 8 servers on the same network, 3 are almost identical, but one of them (the one with the problem) is using software RAID vs hardware ... but, if you are seeing it without using software RAID, then that is obviously not the culprit :( May be a red herring... I'm able to reproduce the No buffer space available message when setting net.inet.tcp.(send|recv)space to non-default values. All I've tried is the following, with a kernel dated 2007/04/22: # sysctl net.inet.tcp.sendspace=131072 # sysctl net.inet.tcp.recvspace=262144 Example session: $ su2 # sysctl net.inet.tcp.sendspace=131072 net.inet.tcp.sendspace: 32768 - 131072 # sysctl net.inet.tcp.recvspace=262144 net.inet.tcp.recvspace: 65536 - 262144 # logout $ ssh medusa socket: No buffer space available ssh: connect to host medusa port 22: No buffer space available $ su2 # sysctl net.inet.tcp.sendspace=32768 net.inet.tcp.sendspace: 131072 - 32768 # sysctl net.inet.tcp.recvspace=65536 net.inet.tcp.recvspace: 262144 - 65536 # logout $ ssh medusa Last login: Mon Apr 23 03:45:45 2007 from ... I assume this is because the maximum size of a TCP datagram is 65536 bytes, but as I'm not familiar enough with TCP on such a low level, this may be speculation on my part. Just something worth checking/tinkering with. -- | Jeremy Chadwickjdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: No buffer space available
On Monday, 23 April 2007 at 4:06:10 -0700, Jeremy Chadwick wrote: On Sat, Apr 07, 2007 at 08:20:58PM -0300, Marc G. Fournier wrote: - --On Saturday, April 07, 2007 20:12:00 +0100 Chris [EMAIL PROTECTED] wrote: Also to add I now have a 2nd box using 6.2 STABLE few days old code, had to use it because of broadcom 5755 nic card, I plan to use large tcp window sizes so will be interesting to see if this also suffers from the problem. I've got 8 servers on the same network, 3 are almost identical, but one of them (the one with the problem) is using software RAID vs hardware ... but, if you are seeing it without using software RAID, then that is obviously not the culprit :( May be a red herring... I'm able to reproduce the No buffer space available message when setting net.inet.tcp.(send|recv)space to non-default values. All I've tried is the following, with a kernel dated 2007/04/22: # sysctl net.inet.tcp.sendspace=131072 # sysctl net.inet.tcp.recvspace=262144 Example session: $ su2 # sysctl net.inet.tcp.sendspace=131072 net.inet.tcp.sendspace: 32768 - 131072 # sysctl net.inet.tcp.recvspace=262144 net.inet.tcp.recvspace: 65536 - 262144 # logout $ ssh medusa socket: No buffer space available ssh: connect to host medusa port 22: No buffer space available $ su2 # sysctl net.inet.tcp.sendspace=32768 net.inet.tcp.sendspace: 131072 - 32768 # sysctl net.inet.tcp.recvspace=65536 net.inet.tcp.recvspace: 262144 - 65536 # logout $ ssh medusa Last login: Mon Apr 23 03:45:45 2007 from ... I assume this is because the maximum size of a TCP datagram is 65536 bytes, but as I'm not familiar enough with TCP on such a low level, this may be speculation on my part. Just something worth checking/tinkering with. Try to adjust kern.ipc.maxsockbuf value. -- == - Best regards, Nikolay Pavlov. --- == ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: No buffer space available
On 06/04/07, Marc G. Fournier [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Friday, April 06, 2007 06:17:04 +0100 Chris [EMAIL PROTECTED] wrote: I am seeing the no buffer space error on a machine running 6.2 STABLE feb 24 code, the machine isn't using gmirror. I had to recude recvspace and sendspace to lower values then I want to get round the problem. 67/1163/1230 mbufs in use (current/cache/total) 65/275/340/65536 mbuf clusters in use (current/cache/total/max) 65/255 mbuf+clusters out of packet secondary zone in use (current/cache) 0/0/0/0 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/0 9k jumbo clusters in use (current/cache/total/max) 0/0/0/0 16k jumbo clusters in use (current/cache/total/max) 146K/840K/987K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/56/8704 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 20233 requests for I/O initiated by sendfile 7740 calls to protocol drain routines What ethernet driver are you using? In my case, its an fxp device ... trying to see if there is *some* sort of common denominator here :( I just upgraded to the latest kernel last night, to see if maybe a recent commit had a side-effect of fixing it, but won't know anything for another 48 hours or so ... - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (FreeBSD) iD8DBQFGFpJ44QvfyHIvDvMRAny4AKCOVStyCBOi5Pwt5uyelgze3ML/kQCgxqCp 6VZ/f9U4ibx/zahMLWu+Fs0= =U8Y1 -END PGP SIGNATURE- its a re0 Chris ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: No buffer space available
On 06/04/07, Marc G. Fournier [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Friday, April 06, 2007 06:17:04 +0100 Chris [EMAIL PROTECTED] wrote: I am seeing the no buffer space error on a machine running 6.2 STABLE feb 24 code, the machine isn't using gmirror. I had to recude recvspace and sendspace to lower values then I want to get round the problem. 67/1163/1230 mbufs in use (current/cache/total) 65/275/340/65536 mbuf clusters in use (current/cache/total/max) 65/255 mbuf+clusters out of packet secondary zone in use (current/cache) 0/0/0/0 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/0 9k jumbo clusters in use (current/cache/total/max) 0/0/0/0 16k jumbo clusters in use (current/cache/total/max) 146K/840K/987K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/56/8704 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 20233 requests for I/O initiated by sendfile 7740 calls to protocol drain routines What ethernet driver are you using? In my case, its an fxp device ... trying to see if there is *some* sort of common denominator here :( I just upgraded to the latest kernel last night, to see if maybe a recent commit had a side-effect of fixing it, but won't know anything for another 48 hours or so ... - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (FreeBSD) iD8DBQFGFpJ44QvfyHIvDvMRAny4AKCOVStyCBOi5Pwt5uyelgze3ML/kQCgxqCp 6VZ/f9U4ibx/zahMLWu+Fs0= =U8Y1 -END PGP SIGNATURE- Also to add I now have a 2nd box using 6.2 STABLE few days old code, had to use it because of broadcom 5755 nic card, I plan to use large tcp window sizes so will be interesting to see if this also suffers from the problem. Chris ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: No buffer space available
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Saturday, April 07, 2007 20:12:00 +0100 Chris [EMAIL PROTECTED] wrote: Also to add I now have a 2nd box using 6.2 STABLE few days old code, had to use it because of broadcom 5755 nic card, I plan to use large tcp window sizes so will be interesting to see if this also suffers from the problem. I've got 8 servers on the same network, 3 are almost identical, but one of them (the one with the problem) is using software RAID vs hardware ... but, if you are seeing it without using software RAID, then that is obviously not the culprit :( - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (FreeBSD) iD8DBQFGGCda4QvfyHIvDvMRAshzAJ47nHUdu2Xlxy8odBbaCxufhfV9igCgjQTw xNFG2VFQmGPNhjToZJ6HDNk= =6BN+ -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: No buffer space available
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Friday, April 06, 2007 06:17:04 +0100 Chris [EMAIL PROTECTED] wrote: I am seeing the no buffer space error on a machine running 6.2 STABLE feb 24 code, the machine isn't using gmirror. I had to recude recvspace and sendspace to lower values then I want to get round the problem. 67/1163/1230 mbufs in use (current/cache/total) 65/275/340/65536 mbuf clusters in use (current/cache/total/max) 65/255 mbuf+clusters out of packet secondary zone in use (current/cache) 0/0/0/0 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/0 9k jumbo clusters in use (current/cache/total/max) 0/0/0/0 16k jumbo clusters in use (current/cache/total/max) 146K/840K/987K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/56/8704 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 20233 requests for I/O initiated by sendfile 7740 calls to protocol drain routines What ethernet driver are you using? In my case, its an fxp device ... trying to see if there is *some* sort of common denominator here :( I just upgraded to the latest kernel last night, to see if maybe a recent commit had a side-effect of fixing it, but won't know anything for another 48 hours or so ... - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (FreeBSD) iD8DBQFGFpJ44QvfyHIvDvMRAny4AKCOVStyCBOi5Pwt5uyelgze3ML/kQCgxqCp 6VZ/f9U4ibx/zahMLWu+Fs0= =U8Y1 -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: No buffer space available
Marc, My machine is working with the kernel that comes with 6.1-STABLE (I archived this good kernel before upgrade the OS to 6.2). No, I'm not using geom. Can you send your dmesg.boot and sysctl -a kern output? Marc G. Fournier wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Thiago ... I'm just curious here, but are you by any chance using geom at all? The only machine I have that seems to be affected like this (where netstat -m doesn't seem to indicate a problem with mbufs) is using gmirror ... the rest all use hardware RAID controllers ... Its a long shot, but so far, its the only one I seem to be able to draw :( - --On Sunday, April 01, 2007 17:07:08 -0300 Thiago Esteves de Oliveira [EMAIL PROTECTED] wrote: I've tried to increase the kern.ipc.nmbclusters value but it worked only when I changed the kernel to an older one. netstat -m (Now it's working with the same values.) - 515/850/1365 mbufs in use (current/cache/total) 512/390/902/65024 mbuf clusters in use (current/cache/total/max) 512/243 mbuf+clusters out of packet secondary zone in use (current/cache) 0/0/0/0 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/0 9k jumbo clusters in use (current/cache/total/max) 0/0/0/0 16k jumbo clusters in use (current/cache/total/max) 1152K/992K/2145K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 2759 requests for I/O initiated by sendfile 2982 calls to protocol drain routines Ethernet adapters - em0: Intel(R) PRO/1000 Network Connection Version - 6.0.5 port 0xec80-0xecbf m em 0xfebe-0xfebf irq 10 at device 4.0 on pci7 em0: Ethernet address: 00:04:23:c3:06:78 em0: [FAST] skc0: 3Com 3C940 Gigabit Ethernet port 0xe800-0xe8ff mem 0xfebd8000-0xfebdbfff irq 15 at device 6.0 on pci7 skc0: 3Com Gigabit NIC (3C2000) rev. (0x1) sk0: Marvell Semiconductor, Inc. Yukon on skc0 sk0: Ethernet address: 00:0a:5e:65:ad:c3 miibus0: MII bus on sk0 e1000phy0: Marvell 88E1000 Gigabit PHY on miibus0 e1000phy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX-FDX, auto P.S.: I am using the FreeBSD/amd64. Brian A. Seklecki wrote: Show us netstat -m on the broken kernel? Show us your dmesg(8) for em(4). TIA, ~BAS On Fri, 2007-03-30 at 11:13 -0300, Thiago Esteves de Oliveira wrote: Hello, I've had a problem with one of my FreeBSD servers, the machine has stopped its network services and then sent these messages: -Mar 27 13:00:03 anubis dhcpd: send_packet: No buffer space available -Mar 27 13:00:26 anubis routed[431]: Send bcast sendto(em0, 146.164.92.255.520): No buffer space available The messages were repeated a lot of times before a temporary solution. I've changed the kernel(FreeBSD 6.2) to an older one(FreeBSD 6.1) and since then it's been working well. What happened? P.S.: I can give more informations if necessary. -- Thiago Esteves de Oliveira ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: No buffer space available
Marc, My machine is working with the kernel that comes with 6.1-STABLE (I archived this good kernel before upgrade the OS to 6.2). No, I'm not using geom. Can you send your dmesg.boot and sysctl -a kern output? Marc G. Fournier wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Thiago ... I'm just curious here, but are you by any chance using geom at all? The only machine I have that seems to be affected like this (where netstat -m doesn't seem to indicate a problem with mbufs) is using gmirror ... the rest all use hardware RAID controllers ... Its a long shot, but so far, its the only one I seem to be able to draw :( - --On Sunday, April 01, 2007 17:07:08 -0300 Thiago Esteves de Oliveira [EMAIL PROTECTED] wrote: I've tried to increase the kern.ipc.nmbclusters value but it worked only when I changed the kernel to an older one. netstat -m (Now it's working with the same values.) - 515/850/1365 mbufs in use (current/cache/total) 512/390/902/65024 mbuf clusters in use (current/cache/total/max) 512/243 mbuf+clusters out of packet secondary zone in use (current/cache) 0/0/0/0 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/0 9k jumbo clusters in use (current/cache/total/max) 0/0/0/0 16k jumbo clusters in use (current/cache/total/max) 1152K/992K/2145K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 2759 requests for I/O initiated by sendfile 2982 calls to protocol drain routines Ethernet adapters - em0: Intel(R) PRO/1000 Network Connection Version - 6.0.5 port 0xec80-0xecbf m em 0xfebe-0xfebf irq 10 at device 4.0 on pci7 em0: Ethernet address: 00:04:23:c3:06:78 em0: [FAST] skc0: 3Com 3C940 Gigabit Ethernet port 0xe800-0xe8ff mem 0xfebd8000-0xfebdbfff irq 15 at device 6.0 on pci7 skc0: 3Com Gigabit NIC (3C2000) rev. (0x1) sk0: Marvell Semiconductor, Inc. Yukon on skc0 sk0: Ethernet address: 00:0a:5e:65:ad:c3 miibus0: MII bus on sk0 e1000phy0: Marvell 88E1000 Gigabit PHY on miibus0 e1000phy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX-FDX, auto P.S.: I am using the FreeBSD/amd64. Brian A. Seklecki wrote: Show us netstat -m on the broken kernel? Show us your dmesg(8) for em(4). TIA, ~BAS On Fri, 2007-03-30 at 11:13 -0300, Thiago Esteves de Oliveira wrote: Hello, I've had a problem with one of my FreeBSD servers, the machine has stopped its network services and then sent these messages: -Mar 27 13:00:03 anubis dhcpd: send_packet: No buffer space available -Mar 27 13:00:26 anubis routed[431]: Send bcast sendto(em0, 146.164.92.255.520): No buffer space available The messages were repeated a lot of times before a temporary solution. I've changed the kernel(FreeBSD 6.2) to an older one(FreeBSD 6.1) and since then it's been working well. What happened? P.S.: I can give more informations if necessary. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: No buffer space available
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Thursday, April 05, 2007 11:06:30 + Thiago Esteves de Oliveira [EMAIL PROTECTED] wrote: Marc, My machine is working with the kernel that comes with 6.1-STABLE (I archived this good kernel before upgrade the OS to 6.2). No, I'm not using geom. Can you send your dmesg.boot and sysctl -a kern output? pid 8 (sendmail), uid 25 inumber 8577 on /var: filesystem full pid 84023 (sendmail), uid 25 inumber 8587 on /var: filesystem full pid 81719 (sendmail), uid 25 inumber 8591 on /var: filesystem full pid 85247 (sendmail), uid 25 inumber 8593 on /var: filesystem full pid 86881 (dd), uid 2 inumber 8596 on /var: filesystem full pid 90114 (dd), uid 2 inumber 8580 on /var: filesystem full fxp0: promiscuous mode disabled fxp0: promiscuous mode enabled pid 92861 (dd), uid 2 inumber 8365 on /var: filesystem full pid 96933 (dd), uid 2 inumber 8439 on /var: filesystem full pid 2289 (dd), uid 2 inumber 8480 on /var: filesystem full fxp0: promiscuous mode disabled fxp0: promiscuous mode enabled pid 4664 (sendmail), uid 25 inumber 8572 on /var: filesystem full pid 4965 (dd), uid 2 inumber 8581 on /var: filesystem full pid 5228 (sendmail), uid 25 inumber 8588 on /var: filesystem full pid 5970 (sendmail), uid 25 inumber 8592 on /var: filesystem full pid 8960 (dd), uid 2 inumber 8595 on /var: filesystem full pid 11981 (dd), uid 2 inumber 8565 on /var: filesystem full fxp0: promiscuous mode disabled fxp0: promiscuous mode enabled pid 14065 (sendmail), uid 25 inumber 8597 on /var: filesystem full pid 15495 (dd), uid 2 inumber 8600 on /var: filesystem full pid 15532 (kiplingOuija.cgi), uid 80: exited on signal 11 pid 15909 (sendmail), uid 25 inumber 8601 on /var: filesystem full pid 18424 (dd), uid 2 inumber 8510 on /var: filesystem full pid 21371 (dd), uid 2 inumber 8559 on /var: filesystem full fxp0: promiscuous mode disabled fxp0: promiscuous mode enabled pid 23625 (sendmail), uid 25 inumber 8572 on /var: filesystem full pid 24260 (sendmail), uid 25 inumber 8588 on /var: filesystem full pid 25003 (sendmail), uid 25 inumber 8592 on /var: filesystem full pid 27135 (dd), uid 2 inumber 8578 on /var: filesystem full pid 29086 (sendmail), uid 25 inumber 8596 on /var: filesystem full pid 30176 (dd), uid 2 inumber 8597 on /var: filesystem full fxp0: promiscuous mode disabled fxp0: promiscuous mode enabled pid 33170 (dd), uid 2 inumber 8581 on /var: filesystem full pid 35631 (dd), uid 2 inumber 8365 on /var: filesystem full pid 39216 (dd), uid 2 inumber 8439 on /var: filesystem full fxp0: promiscuous mode disabled fxp0: promiscuous mode enabled pid 41773 (dd), uid 2 inumber 8480 on /var: filesystem full pid 42005 (sendmail), uid 25 inumber 8572 on /var: filesystem full pid 44505 (dd), uid 2 inumber 8510 on /var: filesystem full pid 45410 (sendmail), uid 25 inumber 8580 on /var: filesystem full pid 47630 (dd), uid 2 inumber 8582 on /var: filesystem full fxp0: promiscuous mode disabled fxp0: promiscuous mode enabled pid 49712 (sendmail), uid 25 inumber 8588 on /var: filesystem full pid 51003 (dd), uid 2 inumber 8593 on /var: filesystem full pid 51467 (sendmail), uid 25 inumber 8594 on /var: filesystem full pid 51804 (kiplingOuija.cgi), uid 80: exited on signal 11 pid 53577 (dd), uid 2 inumber 8559 on /var: filesystem full pid 54476 (sendmail), uid 25 inumber 8579 on /var: filesystem full pid 55549 (sendmail), uid 25 inumber 8601 on /var: filesystem full pid 56090 (bigOuija.cgi), uid 80: exited on signal 11 pid 57137 (dd), uid 2 inumber 8604 on /var: filesystem full pid 58015 (kiplingOuija.cgi), uid 80: exited on signal 11 fxp0: promiscuous mode disabled fxp0: promiscuous mode enabled fxp0: promiscuous mode disabled GEOM_MIRROR: Device vm: provider mirror/vm destroyed. GEOM_MIRROR: Device vm destroyed. Waiting (max 60 seconds) for system process `vnlru' to stop...done Waiting (max 60 seconds) for system process `bufdaemon' to stop...done Waiting (max 60 seconds) for system process `syncer' to stop... Syncing disks, vnodes remaining...4 4 4 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 done All buffers synced. /vm: unmount pending error: blocks -64 files 0 GEOM_MIRROR: Device md2: provider mirror/md2 destroyed. GEOM_MIRROR: Device md2 destroyed. GEOM_STRIPE: Disk mirror/md2 removed from md0. GEOM_STRIPE: Device md0 removed. GEOM_MIRROR: Device md1: provider mirror/md1 destroyed. GEOM_MIRROR: Device md1 destroyed. GEOM_STRIPE: Disk mirror/md1 removed from md0. GEOM_STRIPE: Device md0 destroyed. Uptime: 3d8h23m45s Rebooting... cpu_reset: Stopping other CPUs Copyright (c) 1992-2007 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 6.2-STABLE #5: Tue Mar 13 02:29:37 ADT 2007 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/kernel Timecounter i8254 frequency 1193182 Hz quality 0 CPU: Intel(R)
Re: No buffer space available
On 05/04/07, Marc G. Fournier [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Thursday, April 05, 2007 11:06:30 + Thiago Esteves de Oliveira [EMAIL PROTECTED] wrote: Marc, My machine is working with the kernel that comes with 6.1-STABLE (I archived this good kernel before upgrade the OS to 6.2). No, I'm not using geom. Can you send your dmesg.boot and sysctl -a kern output? pid 8 (sendmail), uid 25 inumber 8577 on /var: filesystem full pid 84023 (sendmail), uid 25 inumber 8587 on /var: filesystem full pid 81719 (sendmail), uid 25 inumber 8591 on /var: filesystem full pid 85247 (sendmail), uid 25 inumber 8593 on /var: filesystem full pid 86881 (dd), uid 2 inumber 8596 on /var: filesystem full pid 90114 (dd), uid 2 inumber 8580 on /var: filesystem full fxp0: promiscuous mode disabled fxp0: promiscuous mode enabled pid 92861 (dd), uid 2 inumber 8365 on /var: filesystem full pid 96933 (dd), uid 2 inumber 8439 on /var: filesystem full pid 2289 (dd), uid 2 inumber 8480 on /var: filesystem full fxp0: promiscuous mode disabled fxp0: promiscuous mode enabled pid 4664 (sendmail), uid 25 inumber 8572 on /var: filesystem full pid 4965 (dd), uid 2 inumber 8581 on /var: filesystem full pid 5228 (sendmail), uid 25 inumber 8588 on /var: filesystem full pid 5970 (sendmail), uid 25 inumber 8592 on /var: filesystem full pid 8960 (dd), uid 2 inumber 8595 on /var: filesystem full pid 11981 (dd), uid 2 inumber 8565 on /var: filesystem full fxp0: promiscuous mode disabled fxp0: promiscuous mode enabled pid 14065 (sendmail), uid 25 inumber 8597 on /var: filesystem full pid 15495 (dd), uid 2 inumber 8600 on /var: filesystem full pid 15532 (kiplingOuija.cgi), uid 80: exited on signal 11 pid 15909 (sendmail), uid 25 inumber 8601 on /var: filesystem full pid 18424 (dd), uid 2 inumber 8510 on /var: filesystem full pid 21371 (dd), uid 2 inumber 8559 on /var: filesystem full fxp0: promiscuous mode disabled fxp0: promiscuous mode enabled pid 23625 (sendmail), uid 25 inumber 8572 on /var: filesystem full pid 24260 (sendmail), uid 25 inumber 8588 on /var: filesystem full pid 25003 (sendmail), uid 25 inumber 8592 on /var: filesystem full pid 27135 (dd), uid 2 inumber 8578 on /var: filesystem full pid 29086 (sendmail), uid 25 inumber 8596 on /var: filesystem full pid 30176 (dd), uid 2 inumber 8597 on /var: filesystem full fxp0: promiscuous mode disabled fxp0: promiscuous mode enabled pid 33170 (dd), uid 2 inumber 8581 on /var: filesystem full pid 35631 (dd), uid 2 inumber 8365 on /var: filesystem full pid 39216 (dd), uid 2 inumber 8439 on /var: filesystem full fxp0: promiscuous mode disabled fxp0: promiscuous mode enabled pid 41773 (dd), uid 2 inumber 8480 on /var: filesystem full pid 42005 (sendmail), uid 25 inumber 8572 on /var: filesystem full pid 44505 (dd), uid 2 inumber 8510 on /var: filesystem full pid 45410 (sendmail), uid 25 inumber 8580 on /var: filesystem full pid 47630 (dd), uid 2 inumber 8582 on /var: filesystem full fxp0: promiscuous mode disabled fxp0: promiscuous mode enabled pid 49712 (sendmail), uid 25 inumber 8588 on /var: filesystem full pid 51003 (dd), uid 2 inumber 8593 on /var: filesystem full pid 51467 (sendmail), uid 25 inumber 8594 on /var: filesystem full pid 51804 (kiplingOuija.cgi), uid 80: exited on signal 11 pid 53577 (dd), uid 2 inumber 8559 on /var: filesystem full pid 54476 (sendmail), uid 25 inumber 8579 on /var: filesystem full pid 55549 (sendmail), uid 25 inumber 8601 on /var: filesystem full pid 56090 (bigOuija.cgi), uid 80: exited on signal 11 pid 57137 (dd), uid 2 inumber 8604 on /var: filesystem full pid 58015 (kiplingOuija.cgi), uid 80: exited on signal 11 fxp0: promiscuous mode disabled fxp0: promiscuous mode enabled fxp0: promiscuous mode disabled GEOM_MIRROR: Device vm: provider mirror/vm destroyed. GEOM_MIRROR: Device vm destroyed. Waiting (max 60 seconds) for system process `vnlru' to stop...done Waiting (max 60 seconds) for system process `bufdaemon' to stop...done Waiting (max 60 seconds) for system process `syncer' to stop... Syncing disks, vnodes remaining...4 4 4 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 done All buffers synced. /vm: unmount pending error: blocks -64 files 0 GEOM_MIRROR: Device md2: provider mirror/md2 destroyed. GEOM_MIRROR: Device md2 destroyed. GEOM_STRIPE: Disk mirror/md2 removed from md0. GEOM_STRIPE: Device md0 removed. GEOM_MIRROR: Device md1: provider mirror/md1 destroyed. GEOM_MIRROR: Device md1 destroyed. GEOM_STRIPE: Disk mirror/md1 removed from md0. GEOM_STRIPE: Device md0 destroyed. Uptime: 3d8h23m45s Rebooting... cpu_reset: Stopping other CPUs Copyright (c) 1992-2007 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 6.2-STABLE #5: Tue Mar 13 02:29:37 ADT 2007 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/kernel Timecounter
Re: No buffer space available
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I'm seeing the same effect (haven't tried older kernel, mind you) almost like clockwork, every 72 hours after reboot ... at least now I don't feel so crazy, knowing it isn't just me ... - --On Sunday, April 01, 2007 17:07:08 -0300 Thiago Esteves de Oliveira [EMAIL PROTECTED] wrote: I've tried to increase the kern.ipc.nmbclusters value but it worked only when I changed the kernel to an older one. netstat -m (Now it's working with the same values.) - 515/850/1365 mbufs in use (current/cache/total) 512/390/902/65024 mbuf clusters in use (current/cache/total/max) 512/243 mbuf+clusters out of packet secondary zone in use (current/cache) 0/0/0/0 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/0 9k jumbo clusters in use (current/cache/total/max) 0/0/0/0 16k jumbo clusters in use (current/cache/total/max) 1152K/992K/2145K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 2759 requests for I/O initiated by sendfile 2982 calls to protocol drain routines Ethernet adapters - em0: Intel(R) PRO/1000 Network Connection Version - 6.0.5 port 0xec80-0xecbf m em 0xfebe-0xfebf irq 10 at device 4.0 on pci7 em0: Ethernet address: 00:04:23:c3:06:78 em0: [FAST] skc0: 3Com 3C940 Gigabit Ethernet port 0xe800-0xe8ff mem 0xfebd8000-0xfebdbfff irq 15 at device 6.0 on pci7 skc0: 3Com Gigabit NIC (3C2000) rev. (0x1) sk0: Marvell Semiconductor, Inc. Yukon on skc0 sk0: Ethernet address: 00:0a:5e:65:ad:c3 miibus0: MII bus on sk0 e1000phy0: Marvell 88E1000 Gigabit PHY on miibus0 e1000phy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX-FDX, auto P.S.: I am using the FreeBSD/amd64. Brian A. Seklecki wrote: Show us netstat -m on the broken kernel? Show us your dmesg(8) for em(4). TIA, ~BAS On Fri, 2007-03-30 at 11:13 -0300, Thiago Esteves de Oliveira wrote: Hello, I've had a problem with one of my FreeBSD servers, the machine has stopped its network services and then sent these messages: -Mar 27 13:00:03 anubis dhcpd: send_packet: No buffer space available -Mar 27 13:00:26 anubis routed[431]: Send bcast sendto(em0, 146.164.92.255.520): No buffer space available The messages were repeated a lot of times before a temporary solution. I've changed the kernel(FreeBSD 6.2) to an older one(FreeBSD 6.1) and since then it's been working well. What happened? P.S.: I can give more informations if necessary. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (FreeBSD) iD8DBQFGE6UE4QvfyHIvDvMRAlutAJ0WzVTYq99hmx1km2mdXE7pdUC8IgCgt4O1 eG6kXgqHveumXjkL0t+Q8Q8= =sieE -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: No buffer space available
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Thiago ... What version of kernel did you end up going back to? - --On Wednesday, April 04, 2007 10:15:48 -0300 Marc G. Fournier [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I'm seeing the same effect (haven't tried older kernel, mind you) almost like clockwork, every 72 hours after reboot ... at least now I don't feel so crazy, knowing it isn't just me ... - --On Sunday, April 01, 2007 17:07:08 -0300 Thiago Esteves de Oliveira [EMAIL PROTECTED] wrote: I've tried to increase the kern.ipc.nmbclusters value but it worked only when I changed the kernel to an older one. netstat -m (Now it's working with the same values.) - 515/850/1365 mbufs in use (current/cache/total) 512/390/902/65024 mbuf clusters in use (current/cache/total/max) 512/243 mbuf+clusters out of packet secondary zone in use (current/cache) 0/0/0/0 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/0 9k jumbo clusters in use (current/cache/total/max) 0/0/0/0 16k jumbo clusters in use (current/cache/total/max) 1152K/992K/2145K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 2759 requests for I/O initiated by sendfile 2982 calls to protocol drain routines Ethernet adapters - em0: Intel(R) PRO/1000 Network Connection Version - 6.0.5 port 0xec80-0xecbf m em 0xfebe-0xfebf irq 10 at device 4.0 on pci7 em0: Ethernet address: 00:04:23:c3:06:78 em0: [FAST] skc0: 3Com 3C940 Gigabit Ethernet port 0xe800-0xe8ff mem 0xfebd8000-0xfebdbfff irq 15 at device 6.0 on pci7 skc0: 3Com Gigabit NIC (3C2000) rev. (0x1) sk0: Marvell Semiconductor, Inc. Yukon on skc0 sk0: Ethernet address: 00:0a:5e:65:ad:c3 miibus0: MII bus on sk0 e1000phy0: Marvell 88E1000 Gigabit PHY on miibus0 e1000phy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX-FDX, auto P.S.: I am using the FreeBSD/amd64. Brian A. Seklecki wrote: Show us netstat -m on the broken kernel? Show us your dmesg(8) for em(4). TIA, ~BAS On Fri, 2007-03-30 at 11:13 -0300, Thiago Esteves de Oliveira wrote: Hello, I've had a problem with one of my FreeBSD servers, the machine has stopped its network services and then sent these messages: -Mar 27 13:00:03 anubis dhcpd: send_packet: No buffer space available -Mar 27 13:00:26 anubis routed[431]: Send bcast sendto(em0, 146.164.92.255.520): No buffer space available The messages were repeated a lot of times before a temporary solution. I've changed the kernel(FreeBSD 6.2) to an older one(FreeBSD 6.1) and since then it's been working well. What happened? P.S.: I can give more informations if necessary. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (FreeBSD) iD8DBQFGE6UE4QvfyHIvDvMRAlutAJ0WzVTYq99hmx1km2mdXE7pdUC8IgCgt4O1 eG6kXgqHveumXjkL0t+Q8Q8= =sieE -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] - Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . [EMAIL PROTECTED] MSN . [EMAIL PROTECTED] Yahoo . yscrappy Skype: hub.orgICQ . 7615664 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (FreeBSD) iD8DBQFGE/ZC4QvfyHIvDvMRAsWoAJwJpD8nCtG0iv5U6LY8ISyyDKxgegCg1eti SezStun7CLDA9pgfrp8GloM= =UwSU -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: No buffer space available
I've tried to increase the kern.ipc.nmbclusters value but it worked only when I changed the kernel to an older one. netstat -m (Now it's working with the same values.) - 515/850/1365 mbufs in use (current/cache/total) 512/390/902/65024 mbuf clusters in use (current/cache/total/max) 512/243 mbuf+clusters out of packet secondary zone in use (current/cache) 0/0/0/0 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/0 9k jumbo clusters in use (current/cache/total/max) 0/0/0/0 16k jumbo clusters in use (current/cache/total/max) 1152K/992K/2145K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/0/0 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 2759 requests for I/O initiated by sendfile 2982 calls to protocol drain routines Ethernet adapters - em0: Intel(R) PRO/1000 Network Connection Version - 6.0.5 port 0xec80-0xecbf m em 0xfebe-0xfebf irq 10 at device 4.0 on pci7 em0: Ethernet address: 00:04:23:c3:06:78 em0: [FAST] skc0: 3Com 3C940 Gigabit Ethernet port 0xe800-0xe8ff mem 0xfebd8000-0xfebdbfff irq 15 at device 6.0 on pci7 skc0: 3Com Gigabit NIC (3C2000) rev. (0x1) sk0: Marvell Semiconductor, Inc. Yukon on skc0 sk0: Ethernet address: 00:0a:5e:65:ad:c3 miibus0: MII bus on sk0 e1000phy0: Marvell 88E1000 Gigabit PHY on miibus0 e1000phy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX-FDX, auto P.S.: I am using the FreeBSD/amd64. Brian A. Seklecki wrote: Show us netstat -m on the broken kernel? Show us your dmesg(8) for em(4). TIA, ~BAS On Fri, 2007-03-30 at 11:13 -0300, Thiago Esteves de Oliveira wrote: Hello, I've had a problem with one of my FreeBSD servers, the machine has stopped its network services and then sent these messages: -Mar 27 13:00:03 anubis dhcpd: send_packet: No buffer space available -Mar 27 13:00:26 anubis routed[431]: Send bcast sendto(em0, 146.164.92.255.520): No buffer space available The messages were repeated a lot of times before a temporary solution. I've changed the kernel(FreeBSD 6.2) to an older one(FreeBSD 6.1) and since then it's been working well. What happened? P.S.: I can give more informations if necessary. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: No buffer space available
Show us netstat -m on the broken kernel? Show us your dmesg(8) for em(4). TIA, ~BAS On Fri, 2007-03-30 at 11:13 -0300, Thiago Esteves de Oliveira wrote: Hello, I've had a problem with one of my FreeBSD servers, the machine has stopped its network services and then sent these messages: -Mar 27 13:00:03 anubis dhcpd: send_packet: No buffer space available -Mar 27 13:00:26 anubis routed[431]: Send bcast sendto(em0, 146.164.92.255.520): No buffer space available The messages were repeated a lot of times before a temporary solution. I've changed the kernel(FreeBSD 6.2) to an older one(FreeBSD 6.1) and since then it's been working well. What happened? P.S.: I can give more informations if necessary. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
No buffer space available
Hello, I've had a problem with one of my FreeBSD servers, the machine has stopped its network services and then sent these messages: -Mar 27 13:00:03 anubis dhcpd: send_packet: No buffer space available -Mar 27 13:00:26 anubis routed[431]: Send bcast sendto(em0, 146.164.92.255.520): No buffer space available The messages were repeated a lot of times before a temporary solution. I've changed the kernel(FreeBSD 6.2) to an older one(FreeBSD 6.1) and since then it's been working well. What happened? P.S.: I can give more informations if necessary. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
no buffer space available PROBLEM
all possible values bumped higher, netstat -m shows big reserves but i still sometimes get this message. using nmap for whole subnet is quite likely to trigger this, but other programs too. ifconfig interface down and then up fixes the problem for some time. any idea? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
squid: no buffer space available - after tuning!
average loaded site (about 1000 users), fast machine, lots of ram, fxp interfaces (no realteks), squid reports 32768 filedescriptors available [EMAIL PROTECTED] ~]# limits -U squid Resource limits for class squid: cputime infinity secs filesize infinity kB datasize infinity kB stacksizeinfinity kB coredumpsize0 kB memoryuseinfinity kB memorylocked infinity kB maxprocesses 64 openfiles 32768 sbsize infinity bytes vmemoryuse infinity kB [ /etc/sysctl.conf: kern.ipc.somaxconn=65535 kern.ipc.nmbclusters=32768 net.inet.tcp.delayed_ack=0 net.inet.icmp.icmplim=500 vm.defer_swapspace_pageouts=1 net.inet.tcp.sendspace=65536 net.inet.tcp.recvspace=65536 kern.ipc.shmseg=128 kern.ipc.shmall=16384 kern.maxfiles=65536 kern.maxfilesperproc=32768 kern.ipc.nmbclusters=131072 net.inet.ip.portrange.last=65535 vfs.lorunningspace=3145728 vfs.hirunningspace=6291456 net.inet.tcp.msl=5000 vfs.root.mountfrom=ufs:ad0a kern.cam.scsi_delay=1000 kern.ipc.msgseg=1024 kern.ipc.msgssz=128 kern.ipc.msgtql=8192 kern.ipc.msgmnb=65536 kern.ipc.msgmni=100 kern.ipc.msgmax=8192 kern.maxproc=1000 kern.maxbcache=134217728 kern.dfldsiz=2147483648 kern.maxdsiz=2147483648 in dmesg i found lots of ipfw: pullup failed CPU load is always 10%, it's P4 machine with 2GB ram (much more than squid uses) running FreeBSD 6.2-RC1, 3 interfaces - out output, 2 for different connections, ipfw is used with only 1 line. any more ideas? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: squid: no buffer space available - after tuning!
On Dec 11, 2006, at 2:37 PM, Wojciech Puchar wrote: in dmesg i found lots of ipfw: pullup failed CPU load is always 10%, it's P4 machine with 2GB ram (much more than squid uses) running FreeBSD 6.2-RC1, 3 interfaces - out output, 2 for different connections, ipfw is used with only 1 line. The IPFW message implies that you are seeing low-level network problems with truncated packets. What does netstat -i and -s reveal, and, if possible, do you have any switch-based statistics...? -- -Chuck ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
No buffer space available error
Hello, I've been trying to solve this problem by myself for a long time now, but no luck. I run a few dozens of FreeBSD 5.3/5.4 machines, which serve as routers, NAT boxes, Apache, Postfix, OpenVPN, ... servers. Most of them are low-cost PC machines since they are usually deployed to SOHO environments and the loads are rather low. I am having problems with the No buffer space available error like this: Jul 18 08:49:36 Router openvpn[661]: write UDPv4: No buffer space available (code=55) so this is obviously when OpenVPN tries to send UDP packets. And also like this: Jun 23 06:27:38 Router pdns[2182]: Unable to send a packet to our recursing backend: No buffer space available when PowerDNS DNS server tries to do some recursive work. I have been searching Google for a solution and I found out that the error should appear when the mbuf (or sfbuf?) is full and that I can print the current buffer status with 'netstat -m'. Because the error would show up (and not only show up, but also block the network operability for that server) at random times, I set up the swatch daemon on all those servers, so that as soon as the error is logged in messages, I run this command: #!/usr/local/bin/bash LOG=/var/log/swatch.log datum=`date` echo == $datum === sockstat $LOG echo $LOG netstat -n -a $LOG echo $LOG netstat -m $LOG echo $LOG ps ax $LOG echo $LOG Even though the log was growing as I assumed, I couldn't find anything particulary interesting, because the netstat -m command issued by swatch (at the time of the error) still shows something like this: 2 mbufs in use 1/17088 mbuf clusters in use (current/max) 0/6/4528 sfbufs in use (current/peak/max) 2 KBytes allocated to network 0 requests for sfbufs denied 0 requests for sfbufs delayed 1819 requests for I/O initiated by sendfile 7578 calls to protocol drain routines I am not sure, but as I understand it, this means that the buffers are quite OK. What would be the proper way to debug this problem? This is happening on machines with various hardware, from good old Pentium I with 32 MB RAM up to P4 3GHz, 1GB RAM, various network cards (mostly rtl8139), with ADSL or VDSL, although the errors are very rare at the VDSL boxes (where the upstream bandwidth is substantially greater). So, usually the errors appear but the users don't bother really, so it looks like the problems goes away sometimes (the connection is restored), but sometimes reboot is needed. Thanks for your ideas. P.S.: If the output of the script above could be helpful, let me know, I can publish it somewhere. Cheers, Nejc ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: No Buffer Space Available
Your trying to run too many memory hungry applications at same time. Tweaking the kernel is not going to help you. Adding more ram will. Better to only run single network monitoring application at a time. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Yousef Raffah Sent: Monday, April 24, 2006 3:33 AM To: freebsd-questions@freebsd.org Subject: No Buffer Space Available Hello.. Please forgive me for being quite new to FreeBSD... I noticed while I'm trying to monitor my network from my laptop while running fragrouter -B1 and trying to monitor the connections coming to and going from another machine on the same network through ettercap or ethereal that I get a lot of No buffer space available messages as following: SEND L3 ERROR: 1500 byte packet (0800:06) destined to 192.168.1.4 was not forwarded (libnet_write_raw_ipv4(): -1 bytes written (No buffer space available) ) I even was not able to nmap the other machine. I was trying to run these test over my iwi0 card and I'm on FreeBSD 6.1-RC While googling I found several posts about setting certain kernel parameters with sysctl and stuff can help but I didn't really get the clear picture of the problem and how it can be resolved, if it is considered a problem. Or is it the iwi0 doesn't handle much load? Thanks in advance for any input -- Sincerely, Yousef Raffah Senior Systems Administrator -- Aren't you using Firefox? Get it at http://www.getfirefox.com ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
No Buffer Space Available
Hello.. Please forgive me for being quite new to FreeBSD... I noticed while I'm trying to monitor my network from my laptop while running fragrouter -B1 and trying to monitor the connections coming to and going from another machine on the same network through ettercap or ethereal that I get a lot of No buffer space available messages as following: SEND L3 ERROR: 1500 byte packet (0800:06) destined to 192.168.1.4 was not forwarded (libnet_write_raw_ipv4(): -1 bytes written (No buffer space available) ) I even was not able to nmap the other machine. I was trying to run these test over my iwi0 card and I'm on FreeBSD 6.1-RC While googling I found several posts about setting certain kernel parameters with sysctl and stuff can help but I didn't really get the clear picture of the problem and how it can be resolved, if it is considered a problem. Or is it the iwi0 doesn't handle much load? Thanks in advance for any input -- Sincerely, Yousef Raffah Senior Systems Administrator -- Aren't you using Firefox? Get it at http://www.getfirefox.com signature.asc Description: This is a digitally signed message part
NET_SendPacket ERROR: No buffer space available problem
Hello, I've just installed a 5.4 STABLE - Release freebsd machine and have setup a counter-strike server on it. Now I'm constantly seeing things like this in my server logs: NET_SendPacket ERROR: No buffer space available I have stumbled upon a thread on a forum that was telling me to increase: net.inet.udp.maxdgram=57344 kern.ipc.maxsockbuf=8388608 net.inet.udp.recvspace=84160 So I have done this, but I still get that error. Another forum was telling me to look at the output netstat -m generates: 4294743666 mbufs in use 6235/25600 mbuf clusters in use (current/max) 0/4/6656 sfbufs in use (current/peak/max) 4150866 KBytes allocated to network 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 200 calls to protocol drain routines but as far as I see I only have 6235 out of 25600 mbufs used. Has anyone ever stumbled upon this problem and if so do you have a solution for it ? Thanks, Mihai ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
[Fwd: Re: [Openvpn-users] Problems on FreeBSD and MacOSX (buffer space available)]
Forwarding to freebsd-questions: There are 3 of us know that are having this same issue with OpenVPN and FreeBSD. The thread starts here: http://openvpn.net/archive/openvpn-users/2005-08/msg00356.html Anyone else having this issue, or know of a fix? Forwarded Message From: Derrick MacPherson [EMAIL PROTECTED] To: Sean Leach [EMAIL PROTECTED] Cc: Michael Scheidell [EMAIL PROTECTED], openvpn- [EMAIL PROTECTED] Subject: Re: [Openvpn-users] Problems on FreeBSD and MacOSX (buffer space available) Date: Wed, 21 Sep 2005 16:49:29 -0700 I'm having this same issue, and after updating my OS from 5.3 to latest just to see if that would help. no go. any solution? Since we now have 3 instances of this, can we get any others in FreeBSD to take a look ? On Fri, 2005-08-26 at 15:18 -0700, Sean Leach wrote: As I mention, I am using openvpn 2.0.2 on the FreeBSD machine. Same problem. openvpn --version OpenVPN 2.0.2 i386-unknown-freebsd5.3 [SSL] [LZO] built on Aug 26 2005 Developed by James Yonan Copyright (C) 2002-2005 OpenVPN Solutions LLC [EMAIL PROTECTED] On Aug 26, 2005, at 2:45 PM, Michael Scheidell wrote: See notes on openvpn release 2.0.2 . Fixes network buffer problems in *BSD systems. --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/ bsce5sf ___ Openvpn-users mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/openvpn-users --- SF.Net email is Sponsored by the Better Software Conference EXPO September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices Agile Plan-Driven Development * Managing Projects Teams * Testing QA Security * Process Improvement Measurement * http://www.sqe.com/bsce5sf ___ Openvpn-users mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/openvpn-users --- SF.Net email is sponsored by: Tame your development challenges with Apache's Geronimo App Server. Download it for free - -and be entered to win a 42 plasma tv or your very own Sony(tm)PSP. Click here to play: http://sourceforge.net/geronimo.php ___ Openvpn-users mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/openvpn-users ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
No buffer space available?
Recently my FreeBSD 4.10 server has been experiencing various connection problems under heavy loads-- HTTP requests don't return, connections to the MySQL server fail, and /var/log/messages contains errors like: Aug 31 19:19:16 main sendmail[77493]: j810JGQD077493: SYSERR(www): makeconnection: cannot create socket: No buffer space available Aug 31 19:25:57 main sshd[77700]: error: PAM: failed create sockets: No buffer space available Aug 31 19:53:02 main named[81]: socket(SOCK_RAW): No buffer space available I did some web searching, and read that I should do a netstat -m to see if I am running out of mbufs. But if I understand it correctly, the output of netstat seems to indicate that I am not running out of mbufs? 258/2320/26624 mbufs in use (current/peak/max): 156 mbufs allocated to data 102 mbufs allocated to packet headers 131/938/6656 mbuf clusters in use (current/peak/max) 2456 Kbytes allocated to network (12% of mb_map in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines Meanwhile, output of 'top' and 'vmstat' shows that my load average is usually under 1.0 (even under heavy loads) and that my system is hardly paging anything to disk (pi and po are usually 0)... although I admit I'm not 100% certain how to interpret the rest of vmstat's output or how it might apply. Also, when I do netstat -s -p tcp during heavy traffic, listen queue overflows increases at around 50-70 per second. I tried increasing net.inet.ip.intr_queue_maxlen (from 50 to 1024 to 2048) but that doesn't seem to make any difference. I'm not sure what else to look for, or where to go from here. Should I increase NMBCLUSTERS and/or NMBUFS anyway? Do I need more memory? Or perhaps the hardware simply can't handle the number of connections it's handling now? My system is: FreeBSD 4.10-SECURITY FreeBSD 4.10-SECURITY #0: Wed Jun 29 20:49:39 GMT 2005 Hardware: AMD Sempron 2600, 1G RAM, 80GB Any tips, pointers would be greatly appreciated. Thank you very much! wayne a. lee ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
No buffer space available arplookup 0.0.0.0 failed: host is not on load network
Hi folks, anybody know about the message below? Abandoning IP Address 200.x.x.x: pinged before offer No buffer space available arplookup 0.0.0.0 failed: host is not on load network Fatal trap 12: page fault while in kernel mode fault virtual address = 0x8a Syncing disks, buffers remaining.1347 1347 1347 I have two traffic shapers that are working with ipfw and dummynet. Thanks in advance, Ricardo Pichler ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: No buffer space available arplookup 0.0.0.0 failed: host is not onload network
Always in the past with my hardware this has been due to indifferent network adapters cards. Post a dmesg please. Ted -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Ricardo Pichler Sent: Monday, May 30, 2005 6:47 PM To: freebsd-questions@freebsd.org Subject: No buffer space available arplookup 0.0.0.0 failed: host is not onload network Hi folks, anybody know about the message below? Abandoning IP Address 200.x.x.x: pinged before offer No buffer space available arplookup 0.0.0.0 failed: host is not on load network Fatal trap 12: page fault while in kernel mode fault virtual address = 0x8a Syncing disks, buffers remaining.1347 1347 1347 I have two traffic shapers that are working with ipfw and dummynet. Thanks in advance, Ricardo Pichler ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: NDIS: no buffer space available
I had a similar issue with the ath0 driver for my Dlink card. I was trying to get the NDISultaor to work to remedy this though. I found a post that you might want to try, can't remember where, search for ping: sendto: No buffer space available on google groups. Deepak Jain responded to set our kern.ipc.maxsocbuf to a bit higher: sysctl -w kern.ipc.maxsockbuf=384000 now if only i could get my ndis0 device to appear... --- Jorge Mario G. [EMAIL PROTECTED] wrote: Hi there after using emule for a like 30mins I start getting that message I used netstat -m to track the mbuf clusters but when I start getting the message there is only like 356 used out of 32768 - I'm using the NDIS module to load my wifi card - I can send and recive all the info I want via http, ftp, etc it's the p2p software what makes it fail - When I'm connected via ethernet it seems to work fine Thanks Jorge = _ Do You Yahoo!? Información de Estados Unidos y América Latina, en Yahoo! Noticias. Visítanos en http://noticias.espanol.yahoo.com ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] __ Do you Yahoo!? Read only the mail you want - Yahoo! Mail SpamGuard. http://promotions.yahoo.com/new_mail ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
NDIS: no buffer space available
Hi there after using emule for a like 30mins I start getting that message I used netstat -m to track the mbuf clusters but when I start getting the message there is only like 356 used out of 32768 - I'm using the NDIS module to load my wifi card - I can send and recive all the info I want via http, ftp, etc it's the p2p software what makes it fail - When I'm connected via ethernet it seems to work fine Thanks Jorge = _ Do You Yahoo!? Información de Estados Unidos y América Latina, en Yahoo! Noticias. Visítanos en http://noticias.espanol.yahoo.com ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
ndis0: No buffer space available
Hi there I'm using FreeBSD mosca.doom 5.2.1-RELEASE-p9 FreeBSD 5.2.1-RELEASE-p9 #0: Thu Aug 26 10:47:42 GMT 2004 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/MOSCA i386 I'm using the NDISulator to load my Dell TrueMobile 1300 WLAN Mini-PCI Card, it works really good but after some have traffic I alway get No buffer space availablecat E = _ Do You Yahoo!? Información de Estados Unidos y América Latina, en Yahoo! Noticias. Visítanos en http://noticias.espanol.yahoo.com ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
ndis0: No buffer space available
Hi there I'm using FreeBSD mosca.doom 5.2.1-RELEASE-p9 FreeBSD 5.2.1-RELEASE-p9 #0: Thu Aug 26 10:47:42 GMT 2004 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/MOSCA i386 I'm using the NDISulator to load my Dell TrueMobile 1300 WLAN Mini-PCI Card, it works really good but after some have traffic I alway get No buffer space available bash-2.05b$ ping a PING a (192.168.0.1): 56 data bytes ping: sendto: No buffer space available How do I get around this??? thaks = _ Do You Yahoo!? Información de Estados Unidos y América Latina, en Yahoo! Noticias. Visítanos en http://noticias.espanol.yahoo.com ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
[gateway 4.9] sendto: no buffer space available
Hello, Hardware: PC, 128 Mo, 2 interfaces 3Com 3c905-TX Software: 4.9, generic kernel; from the initial distribution no special configuration Suddenly, this PC acting as a gateway stops forwarding packets, I was in a hurry so I just noticed that ICMP ping packets failed. Message on the console: ping: sendto: no buffer space available Question 1: is it supposed to be fixed automatically within a few minutes? This gateway is important in my network... Question 2: is there some tuning I can do? Here is some output: root# sysctl -a | grep space | grep net net.local.stream.sendspace: 8192 net.local.stream.recvspace: 8192 net.local.dgram.recvspace: 4096 net.inet.tcp.sendspace: 32768 net.inet.tcp.recvspace: 57344 net.inet.udp.recvspace: 42080 net.inet.raw.recvspace: 8192 Thanks, -- Jacques Beigbeder| [EMAIL PROTECTED] Service de Prestations Informatiques | http://www.spi.ens.fr Ecole normale supérieure | 45 rue d'Ulm |Tel : (+33 1)1 44 32 37 96 F75230 Paris cedex 05|Fax : (+33 1)1 44 32 20 75 ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: socket: no buffer space available
On Mon, 2003-12-15 at 16:15, Andrew Thomson wrote: I've got a perl script doing some p5-sybase stuff for me.. However after a while, it fails with the following error message: ..socket: No buffer space available... I've seen other reports from other uses getting this problem however no clear responses on a fix. I had a similar problem on a (5.1-RELEASE) box with a: bge0: Broadcom BCM5704C Dual Gigabit Ethernet, ASIC rev. 0x2002 mem 0xfcd2 -0xfcd2,0xfcd3-0xfcd3 irq 5 at device 0.0 on pci2 As well as the no buffer space problem, there were watchguard timeout measures. This may not be your problem, but its worth ruling out. The fix was a kernel src patch for the bge driver. Jason RHYBUDD: Mae'r wybodaeth sydd yn y neges E-Bost yma yn gyfrinachol ac mae'n bosib y bydd yn gyfreithiol freintiedig. Os nad y sawl a fydd yn darllen y neges yma yw'r sawl y bwriadwyd yr e-bost ar ei gyfer, fe'ch hysbysir drwy hyn y gwaherddir defnyddio'r neges, ei rhannu, ei dosbarthu neu ei hatgynhyrchu gennych chi eich hun neu ar eich anogaeth. Os cawsoch y neges yma trwy amryfusedd, a fyddech cystal â rhoi gwybod i ni ar unwaith a dychwelyd y neges wreiddiol. Nid yw'r wybodaeth sydd yn y neges yma o reidrwydd yn cynrychioli polisi a threfn weithredu'r Cyngor. Os cewch gan y Cyngor hwn ddeunydd heb i chi ofyn amdano neu ddeunydd tramgwyddus rhowch wybod i ni ar unwaith. Diolch CAUTION: The information in this E-Mail message is confidential and may be legally privileged. If the reader of this message is not the intended recipient you are hereby notified that any use, dissemination, distribution or reproduction of this message by you or at your instigation is prohibited. If you have received this message in error please notify us immediately and return the original message to us. The information in this message does not necessarily represent Council policy or procedure. In the instance of receiving unsolicited or offensive mail from this Council please notify us immediately. Where information of a sensitive or highly confidential nature is to be communicated, you will be informed that this information will be sent to you in writing and not by E-Mail, unless secure arrangements are made. Thank you. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: socket: no buffer space available
Just for the record, I fixed this problem by recompiling my kernel with MAXUSERS 512 :) ajt. On Mon, 2003-12-15 at 16:15, Andrew Thomson wrote: I've got a perl script doing some p5-sybase stuff for me.. However after a while, it fails with the following error message: ..socket: No buffer space available... I've seen other reports from other uses getting this problem however no clear responses on a fix. This script used to work find on my 5.0-RELEASE box now I'm trying it on a 5.1-RELEASE box. I admit the new box is a lower spec - less cpu and less memory - so that may affect some of the default sysctl values??? I've tried tweaking a couple of sysctl entries however nothing has gotten me over this hurdle. Below are some relevant(??) sysctls. kern.ipc.maxsockbuf: 1048576 kern.ipc.sockbuf_waste_factor: 8 kern.ipc.nmbufs: 17920 kern.ipc.nsfbufs: 2496 kern.ipc.mbuf_wait: 64 kern.ipc.mbuf_hiwm: 512 kern.ipc.mbuf_lowm: 128 kern.ipc.numopensockets: 94 kern.ipc.maxsockets: 4008 Any suggested tweaks appreciated. Regards, ajt. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re[2]: socket: no buffer space available
AT Just for the record, I fixed this problem by recompiling my kernel with AT MAXUSERS 512 actualy you don't need to recompile kernel. you can achieve that by doing: echo kern.maxusers=0 /boot/loader.conf It is better to use 0, since them freeBSD can dynamicaly change you need (as i heard) AT :) AT ajt. AT On Mon, 2003-12-15 at 16:15, Andrew Thomson wrote: I've got a perl script doing some p5-sybase stuff for me.. However after a while, it fails with the following error message: ..socket: No buffer space available... I've seen other reports from other uses getting this problem however no clear responses on a fix. This script used to work find on my 5.0-RELEASE box now I'm trying it on a 5.1-RELEASE box. I admit the new box is a lower spec - less cpu and less memory - so that may affect some of the default sysctl values??? I've tried tweaking a couple of sysctl entries however nothing has gotten me over this hurdle. Below are some relevant(??) sysctls. kern.ipc.maxsockbuf: 1048576 kern.ipc.sockbuf_waste_factor: 8 kern.ipc.nmbufs: 17920 kern.ipc.nsfbufs: 2496 kern.ipc.mbuf_wait: 64 kern.ipc.mbuf_hiwm: 512 kern.ipc.mbuf_lowm: 128 kern.ipc.numopensockets: 94 kern.ipc.maxsockets: 4008 Any suggested tweaks appreciated. Regards, ajt. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] AT ___ AT [EMAIL PROTECTED] mailing list AT http://lists.freebsd.org/mailman/listinfo/freebsd-questions AT To unsubscribe, send any mail to AT [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
sendto: No buffer space available
Hello, We have a FreeBSD box here that we use to route some GRE tunnels and ipv6 gif tunnels. We use zebra for dynamic routing running zebra, bgpd, ospfd, and ospf6d. We have about 12 FreeBSD boxes with exact same configuration, the only difference is just the IP address of each interface. None of them fail but this one box... Everyday, this box stops all networking. I can still console in and stuff.. When I typed 'ping 127.0.0.1' at the console after networking locked up, it says: ping: sendto: No buffer space available The only solution seems to be rebooting it everyday... It happens every 12 hours or so... This is not related with mbuf, etc either, as netstat -m doesn't show any issues. The box has one IP address and IPv6 address in addition to 127.0.0.1 on lo0 interface. It also has a ds0 interface with 10.5.5.5/30 assigned to ds0. This is exact same configuration on all other boxes, and none of them fail but this one. I've swapped out NICs with different vendors 3 times (tried, xl, dc, and now rl) I've also swapped out the whole box, and also swapped out the whole hard drive and did full reinstall. And problem still persists and it's definately not hardware as I swapped everything out... (unless the 3 NIC vendors above are all exhibiting same issue) I tried to look on Google but nothing useful that corelates to this particular issue.. Any help would be very appreciated :) Thanks, -hc The box is running FreeBSD 4.8-STABLE FreeBSD necsis 4.8-STABLE FreeBSD 4.8-STABLE #0: Tue Jul 29 13:10:11 GMT 2003 [EMAIL PROTECTED]:/usr/src/sys/compile/router i386 Following is output of netstat -s AFTER the networking locks up with no buffer space available error: tcp: 30115 packets sent 17167 data packets (1232152 bytes) 301 data packets (54320 bytes) retransmitted 0 resends initiated by MTU discovery 12416 ack-only packets (10931 delayed) 0 URG only packets 0 window probe packets 41 window update packets 280 control packets 28010 packets received 16762 acks (for 1236693 bytes) 140 duplicate acks 0 acks for unsent data 13205 packets (567038 bytes) received in-sequence 43 completely duplicate packets (818 bytes) 0 old duplicate packets 2 packets with some dup. data (38 bytes duped) 9 out-of-order packets (240 bytes) 0 packets (0 bytes) of data after window 0 window probes 31 window update packets 0 packets received after close 0 discarded for bad checksums 0 discarded for bad header offset fields 0 discarded because packet too short 252 connection requests 18 connection accepts 6 bad connection attempts 0 listen queue overflows 30 connections established (including accepts) 288 connections closed (including 10 drops) 23 connections updated cached RTT on close 23 connections updated cached RTT variance on close 11 connections updated cached ssthresh on close 164 embryonic connections dropped 16643 segments updated rtt (of 16929 attempts) 1566 retransmit timeouts 10 connections dropped by rexmit timeout 0 persist timeouts 0 connections dropped by persist timeout 161 keepalive timeouts 0 keepalive probes sent 161 connections dropped by keepalive 96 correct ACK header predictions 10392 correct data packet header predictions 19 syncache entries added 6 retransmitted 2 dupsyn 0 dropped 18 completed 0 bucket overflow 0 cache overflow 0 reset 0 stale 0 aborted 0 badack 1 unreach 0 zone failures 0 cookies sent 0 cookies received udp: 196 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 1 with no checksum 61 dropped due to no socket 3 broadcast/multicast datagrams dropped due to no socket 0 dropped due to full socket buffers 0 not for hashed pcb 132 delivered 132 datagrams output ip: 2154646 total packets received 0 bad header checksums 0 with size smaller than minimum 0 with data size data length 0 with ip length max ip packet size 0 with header length data size 0 with data length header length 0 with bad options 0 with incorrect version number 366 fragments received 0 fragments
Re: sendto: No buffer space available
I use a freebsd box as a gateway for my home network. It uses a dialup internet connection. When my ISP is having network problems, I will get the precise same issue. I have also had the modem crash, and also got the same problem. I could fix it by killing ppp and restarting it.That clears the tcp buffers for ppp. I suspect you possibly have a bad NIC or perhaps some other network issue that's intermittent. Taking down the interface might clear the buffers... NEOFAST.NET North East Oregon FAST Net mark(at)neofast.net - Original Message - From: Haesu [EMAIL PROTECTED] To: [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Wednesday, July 30, 2003 9:32 AM Subject: sendto: No buffer space available Hello, We have a FreeBSD box here that we use to route some GRE tunnels and ipv6 gif tunnels. We use zebra for dynamic routing running zebra, bgpd, ospfd, and ospf6d. We have about 12 FreeBSD boxes with exact same configuration, the only difference is just the IP address of each interface. None of them fail but this one box... Everyday, this box stops all networking. I can still console in and stuff.. When I typed 'ping 127.0.0.1' at the console after networking locked up, it says: ping: sendto: No buffer space available --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.504 / Virus Database: 302 - Release Date: 7/24/03 ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: sendto: No buffer space available
I did ifconfig down/up on all interfaces, and that didn't help... The only way to clear it up seems like rebooting the whole box.. This one isn't related to any ppp, it has gre tunnels which are kernel based... This is bazzarre problem.. none of the other boxes exhibit this problem ever.. Thanks, -hc -- Sincerely, Haesu C. TowardEX Technologies, Inc. WWW: http://www.towardex.com E-mail: [EMAIL PROTECTED] Cell: (978) 394-2867 On Wed, Jul 30, 2003 at 10:02:02AM -0700, Mark Koskenmaki wrote: I use a freebsd box as a gateway for my home network. It uses a dialup internet connection. When my ISP is having network problems, I will get the precise same issue. I have also had the modem crash, and also got the same problem. I could fix it by killing ppp and restarting it.That clears the tcp buffers for ppp. I suspect you possibly have a bad NIC or perhaps some other network issue that's intermittent. Taking down the interface might clear the buffers... NEOFAST.NET North East Oregon FAST Net mark(at)neofast.net - Original Message - From: Haesu [EMAIL PROTECTED] To: [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Wednesday, July 30, 2003 9:32 AM Subject: sendto: No buffer space available Hello, We have a FreeBSD box here that we use to route some GRE tunnels and ipv6 gif tunnels. We use zebra for dynamic routing running zebra, bgpd, ospfd, and ospf6d. We have about 12 FreeBSD boxes with exact same configuration, the only difference is just the IP address of each interface. None of them fail but this one box... Everyday, this box stops all networking. I can still console in and stuff.. When I typed 'ping 127.0.0.1' at the console after networking locked up, it says: ping: sendto: No buffer space available --- Outgoing mail is certified Virus Free. Checked by AVG anti-virus system (http://www.grisoft.com). Version: 6.0.504 / Virus Database: 302 - Release Date: 7/24/03 ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: sendto: No buffer space available
I had the same exact problem. I traced it to be a bug in some software
that opened a domain socket(2) but could not connect(2) and never closed
the descriptor returned.
something like:
sd = socket(AF_UNIX, SOCK_STREAM, 0);
...
if(connect(sd, (struct sockaddr *)saddr, sizeof(saddr)) 0) {
return -1;
}
where close(2) was skipped on sd if the connect failed.
over a period of time (12-18hrs), these unconnected sockets would fill
up the available buffer space, and exhibit the same symptoms you are
having. even running ifconfig would fail with No buffer space available.
from intro(2):
55 ENOBUFS No buffer space available. An operation on a socket or pipe
was not performed because the system lacked sufficient buffer space or
because a queue was full.
Fixing that bug fixed the problem. I doubt you have a hardware problem,
I would try narrowing down what software is causing the lockup. truss(1)
might help you out here.
Dave
On Wed, 2003-07-30 at 09:32, Haesu wrote:
Hello,
We have a FreeBSD box here that we use to route some GRE tunnels and ipv6 gif
tunnels. We use zebra for dynamic routing running zebra, bgpd, ospfd, and ospf6d.
We have about 12 FreeBSD boxes with exact same configuration, the only
difference is just the IP address of each interface.
None of them fail but this one box...
Everyday, this box stops all networking. I can still console in and stuff.. When
I typed 'ping 127.0.0.1' at the console after networking locked up, it says:
ping: sendto: No buffer space available
The only solution seems to be rebooting it everyday... It happens every 12 hours
or so...
This is not related with mbuf, etc either, as netstat -m doesn't show any
issues.
The box has one IP address and IPv6 address in addition to 127.0.0.1 on lo0
interface. It also has a ds0 interface with 10.5.5.5/30 assigned to ds0.
This is exact same configuration on all other boxes, and none of them fail but
this one.
I've swapped out NICs with different vendors 3 times (tried, xl, dc, and now rl)
I've also swapped out the whole box, and also swapped out the whole hard drive
and did full reinstall. And problem still persists and it's definately not
hardware as I swapped everything out... (unless the 3 NIC vendors above are all
exhibiting same issue)
I tried to look on Google but nothing useful that corelates to this particular
issue..
Any help would be very appreciated :)
Thanks,
-hc
The box is running FreeBSD 4.8-STABLE
FreeBSD necsis 4.8-STABLE FreeBSD 4.8-STABLE #0: Tue Jul 29 13:10:11 GMT 2003
[EMAIL PROTECTED]:/usr/src/sys/compile/router i386
Following is output of netstat -s AFTER the networking locks up with no buffer
space available error:
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: sendto: No buffer space available
Hmmm... i had truss running but the moment it died it was running gettimeoftheday() so
i am not sure :-/
I tried different ports on the switch.. It's a cisco switch btw, and other freebsd
boxes on that switch
are not exhibiting similar problem
I'll try putting this behind a hub or something other than cisco just for kicks but if
anyone has any further ideas/suggestions, i'd really appreciate it.
Thank you!
-hc
--
Sincerely,
Haesu C.
TowardEX Technologies, Inc.
WWW: http://www.towardex.com
E-mail: [EMAIL PROTECTED]
Cell: (978) 394-2867
On Wed, Jul 30, 2003 at 11:36:35AM -0700, Dave Byrne wrote:
I had the same exact problem. I traced it to be a bug in some software
that opened a domain socket(2) but could not connect(2) and never closed
the descriptor returned.
something like:
sd = socket(AF_UNIX, SOCK_STREAM, 0);
...
if(connect(sd, (struct sockaddr *)saddr, sizeof(saddr)) 0) {
return -1;
}
where close(2) was skipped on sd if the connect failed.
over a period of time (12-18hrs), these unconnected sockets would fill
up the available buffer space, and exhibit the same symptoms you are
having. even running ifconfig would fail with No buffer space available.
from intro(2):
55 ENOBUFS No buffer space available. An operation on a socket or pipe
was not performed because the system lacked sufficient buffer space or
because a queue was full.
Fixing that bug fixed the problem. I doubt you have a hardware problem,
I would try narrowing down what software is causing the lockup. truss(1)
might help you out here.
Dave
On Wed, 2003-07-30 at 09:32, Haesu wrote:
Hello,
We have a FreeBSD box here that we use to route some GRE tunnels and ipv6 gif
tunnels. We use zebra for dynamic routing running zebra, bgpd, ospfd, and ospf6d.
We have about 12 FreeBSD boxes with exact same configuration, the only
difference is just the IP address of each interface.
None of them fail but this one box...
Everyday, this box stops all networking. I can still console in and stuff.. When
I typed 'ping 127.0.0.1' at the console after networking locked up, it says:
ping: sendto: No buffer space available
The only solution seems to be rebooting it everyday... It happens every 12 hours
or so...
This is not related with mbuf, etc either, as netstat -m doesn't show any
issues.
The box has one IP address and IPv6 address in addition to 127.0.0.1 on lo0
interface. It also has a ds0 interface with 10.5.5.5/30 assigned to ds0.
This is exact same configuration on all other boxes, and none of them fail but
this one.
I've swapped out NICs with different vendors 3 times (tried, xl, dc, and now rl)
I've also swapped out the whole box, and also swapped out the whole hard drive
and did full reinstall. And problem still persists and it's definately not
hardware as I swapped everything out... (unless the 3 NIC vendors above are all
exhibiting same issue)
I tried to look on Google but nothing useful that corelates to this particular
issue..
Any help would be very appreciated :)
Thanks,
-hc
The box is running FreeBSD 4.8-STABLE
FreeBSD necsis 4.8-STABLE FreeBSD 4.8-STABLE #0: Tue Jul 29 13:10:11 GMT 2003
[EMAIL PROTECTED]:/usr/src/sys/compile/router i386
Following is output of netstat -s AFTER the networking locks up with no buffer
space available error:
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-isp
To unsubscribe, send any mail to [EMAIL PROTECTED]
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: sendto: No buffer space available
I think heading down the path of switchign out network
gear is a bad idea, this is definitely something in
the software.
I have had this error a few times when messing with
the TCP window sizes, net.inet.tcp.sendspace and
net.inet.tcp.recvspace. When I set them to something
over 128000 I would get the error. The solution was
set the number of mbufs to 128000. Doing so allowed
me to make the window sizes 256000 and eliminated
the error.
-good luck
-mtl
--
Michael Lapinski
Computer Scientist
GE Research
I think there is a world market for maybe five computers.
- IBM Chairman Thomas Watson, 1943
--Original Message-
-From: Haesu [mailto:[EMAIL PROTECTED]
-Sent: Wednesday, July 30, 2003 4:26 PM
-To: Dave Byrne; [EMAIL PROTECTED]; [EMAIL PROTECTED]
-Subject: Re: sendto: No buffer space available
-
-
-Hmmm... i had truss running but the moment it died it was
-running gettimeoftheday() so i am not sure :-/
-
-I tried different ports on the switch.. It's a cisco switch
-btw, and other freebsd boxes on that switch
-are not exhibiting similar problem
-
-I'll try putting this behind a hub or something other than
-cisco just for kicks but if anyone has any further
-ideas/suggestions, i'd really appreciate it.
-
-Thank you!
-
--hc
-
---
-Sincerely,
- Haesu C.
- TowardEX Technologies, Inc.
- WWW: http://www.towardex.com
- E-mail: [EMAIL PROTECTED]
- Cell: (978) 394-2867
-
-On Wed, Jul 30, 2003 at 11:36:35AM -0700, Dave Byrne wrote:
- I had the same exact problem. I traced it to be a bug in
-some software
- that opened a domain socket(2) but could not connect(2) and
-never closed
- the descriptor returned.
-
- something like:
-
- sd = socket(AF_UNIX, SOCK_STREAM, 0);
- ...
- if(connect(sd, (struct sockaddr *)saddr, sizeof(saddr)) 0) {
-return -1;
- }
-
- where close(2) was skipped on sd if the connect failed.
-
- over a period of time (12-18hrs), these unconnected sockets
-would fill
- up the available buffer space, and exhibit the same symptoms you are
- having. even running ifconfig would fail with No buffer
-space available.
-
- from intro(2):
- 55 ENOBUFS No buffer space available. An operation on a
-socket or pipe
- was not performed because the system lacked sufficient
-buffer space or
- because a queue was full.
-
- Fixing that bug fixed the problem. I doubt you have a
-hardware problem,
- I would try narrowing down what software is causing the
-lockup. truss(1)
- might help you out here.
-
-
-
- Dave
-
-
-
-
- On Wed, 2003-07-30 at 09:32, Haesu wrote:
- Hello,
-
- We have a FreeBSD box here that we use to route some GRE
-tunnels and ipv6 gif tunnels. We use zebra for dynamic
-routing running zebra, bgpd, ospfd, and ospf6d.
-
- We have about 12 FreeBSD boxes with exact same
-configuration, the only
- difference is just the IP address of each interface.
-
- None of them fail but this one box...
-
- Everyday, this box stops all networking. I can still
-console in and stuff.. When
- I typed 'ping 127.0.0.1' at the console after networking
-locked up, it says:
- ping: sendto: No buffer space available
-
- The only solution seems to be rebooting it everyday... It
-happens every 12 hours
- or so...
-
- This is not related with mbuf, etc either, as netstat -m
-doesn't show any
- issues.
-
- The box has one IP address and IPv6 address in addition
-to 127.0.0.1 on lo0
- interface. It also has a ds0 interface with 10.5.5.5/30
-assigned to ds0.
- This is exact same configuration on all other boxes, and
-none of them fail but
- this one.
-
- I've swapped out NICs with different vendors 3 times
-(tried, xl, dc, and now rl)
-
- I've also swapped out the whole box, and also swapped out
-the whole hard drive
- and did full reinstall. And problem still persists and
-it's definately not
- hardware as I swapped everything out... (unless the 3 NIC
-vendors above are all
- exhibiting same issue)
-
- I tried to look on Google but nothing useful that
-corelates to this particular
- issue..
-
- Any help would be very appreciated :)
-
- Thanks,
- -hc
-
- The box is running FreeBSD 4.8-STABLE
- FreeBSD necsis 4.8-STABLE FreeBSD 4.8-STABLE #0: Tue Jul
-29 13:10:11 GMT 2003 [EMAIL PROTECTED]:/usr/src/sys/compile/router i386
-
- Following is output of netstat -s AFTER the networking
-locks up with no buffer
- space available error:
-
-
- ___
- [EMAIL PROTECTED] mailing list
- http://lists.freebsd.org/mailman/listinfo/freebsd-isp
- To unsubscribe, send any mail to
-[EMAIL PROTECTED]
-
-___
-[EMAIL PROTECTED] mailing list
-http://lists.freebsd.org/mailman/listinfo/freebsd-isp
-To unsubscribe, send any mail to [EMAIL PROTECTED]
-
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo
Re: sendto: No buffer space available
On Thu, 31 Jul 2003 06:25 am, Haesu wrote: Hmmm... i had truss running but the moment it died it was running gettimeoftheday() so i am not sure :-/ If it is software, the other thing to try might be sockstat if your not already aware of it (it lists all the sockets being used by which programs). JacobRhoden - http://rhoden.id.au/ ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ping: sendto: No buffer space available
* Jaime [EMAIL PROTECTED] [2003-06-18 12:28]: On Wednesday, June 18, 2003, at 07:17 AM, Loz wrote: Sounds familiar - a friend had a Linux box cracked over the weekend... apparently russian script kiddies using a php gallery exploit. Sorry I don't have any more details, but I do know that in his case at least nothing else was compromised. He found all the answers he needed on Google. So only his Gallery install was compomised? Or was there a more direct effect, e.g. a backdoor or rootkit install? No other damage apart from a little trojan ping flooding the network and filling up log files. More details on the Gallery exploit at http://www.linuxadvisory.com/articles.php?articleId=35page=3 HTH /loz. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ping: sendto: No buffer space available
* Jaime [EMAIL PROTECTED] [2003-06-18 00:49]: The clues to a crack are evident, too. A process /usr/sbin/nscd is running on the box according to top and ps, but the file does not exist. Further more, I never told such a process to execute. Shortly after a reboot, a netstat command showed a connection to 37303 on a remote host. I was the only person logged in and I did not initiate that connection. Sounds familiar - a friend had a Linux box cracked over the weekend... apparently russian script kiddies using a php gallery exploit. Sorry I don't have any more details, but I do know that in his case at least nothing else was compromised. He found all the answers he needed on Google. good luck, /loz. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ping: sendto: No buffer space available
On Wednesday, June 18, 2003, at 07:17 AM, Loz wrote: Sounds familiar - a friend had a Linux box cracked over the weekend... apparently russian script kiddies using a php gallery exploit. Sorry I don't have any more details, but I do know that in his case at least nothing else was compromised. He found all the answers he needed on Google. So only his Gallery install was compomised? Or was there a more direct effect, e.g. a backdoor or rootkit install? Thanks in advance, Jaime ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
ping: sendto: No buffer space available
I've been noticing for a few days that my network's performance is less than good. When I checked on it, I found that the firewall attempting to ping the ISP's DNS resolver would have hiccups. The ISP claims that there is nothing wrong on the T-1 line and that there is a problem on the ethernet interface of the router (which leads to the firewall). The pings will run just fine for several minutes at a time and then begin to output this: ping: sendto: No buffer space available This will go on for anywhere from 15 seconds to 5 minutes, during which we're effectively not connected to the Internet at all. An occasional ping will work, but only about 1 in 20 and it seems random. Then, just as suddenly, the connection will work again. I'm not completely sure what this means, but I found the following command in the mailing list archives: cerberus# sysctl -a | grep intr_qu net.inet.ip.intr_queue_maxlen: 50 net.inet.ip.intr_queue_drops: 6987 Does anyone have any suggestions or tips? Thanks in advance, Jaime -- To affect the quality of the day, that is the highest of arts. - Henry David Thoreau, _Where_I_Live_ ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ping: sendto: No buffer space available
On Tue, 17 Jun 2003, lbland wrote: I don't know, but it may be a router loop problem in the ISP router tables. Those tables can change dynamically and can cause intermittent issues like you explained. I had three pings going at the same time. One to the ISP's DNS resolver, one to the far end of the T-1, and one to the ethernet interface on the router at my site. The router and firewall are on opposite ends of the same cable. When the pings to the DNS resolver gave the No buffer space message, so did the other two pings. This means that the break down is not any further up stream than the router. I'm now running pings to a host on the same LAN as the firewall. The next time that the No buffer space message appears on the pings to the DNS resolver, I'll check the pings to the internal host. If they have the same problem, then I'm experiencing an OS level issue of some kind. OK, it happened while I was typing this. :) Results: internal host remained ping-able while the other three pings were all giving No buffer space messages. This is starting to sound like some kind of packet over-load on the public side of my FreeBSD/ipfw based firewall. Does anyone have any advice on how to confirm this? bash-2.05b$ uname -a FreeBSD cerberus.cairodurham.org. 4.7-STABLE FreeBSD 4.7-STABLE #0: Sat Oct 12 12:54:03 EDT 2002 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/CERBERUS i386 Thanks in advance, Jaime ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ping: sendto: No buffer space available
Jaime wrote: I've been noticing for a few days that my network's performance is less than good. When I checked on it, I found that the firewall attempting to ping the ISP's DNS resolver would have hiccups. The ISP claims that there is nothing wrong on the T-1 line and that there is a problem on the ethernet interface of the router (which leads to the firewall). The pings will run just fine for several minutes at a time and then begin to output this: ping: sendto: No buffer space available This will go on for anywhere from 15 seconds to 5 minutes, during which we're effectively not connected to the Internet at all. An occasional ping will work, but only about 1 in 20 and it seems random. Then, just as suddenly, the connection will work again. I'm not completely sure what this means, but I found the following command in the mailing list archives: cerberus# sysctl -a | grep intr_qu net.inet.ip.intr_queue_maxlen: 50 net.inet.ip.intr_queue_drops: 6987 Does anyone have any suggestions or tips? What make/model of NIC are you using? The only time I've ever seen this, the only thing that solved the problem was swapping the network card out for a better one. That's not to say it isn't a driver problem, as the new network card used a different driver as well. -- Bill Moran Potential Technologies http://www.potentialtech.com ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ping: sendto: No buffer space available
On Tue, 17 Jun 2003, Bill Moran wrote: What make/model of NIC are you using? cerberus# ifconfig -a fxp0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500 inet 10.0.3.2 netmask 0xff00 broadcast 10.0.3.255 ether 00:e0:81:21:45:8c media: Ethernet autoselect (10baseT/UTP) status: active fxp1: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500 inet 10.1.1.1 netmask 0x broadcast 10.1.255.255 ether 00:e0:81:21:45:8d media: Ethernet autoselect (100baseTX full-duplex) status: active lp0: flags=8810POINTOPOINT,SIMPLEX,MULTICAST mtu 1500 ppp0: flags=8010POINTOPOINT,MULTICAST mtu 1500 sl0: flags=c010POINTOPOINT,LINK2,MULTICAST mtu 552 lo0: flags=8049UP,LOOPBACK,RUNNING,MULTICAST mtu 16384 inet 127.0.0.1 netmask 0xff00 The interface in question is 10.0.3.2. That interface has worked fine for over a year. That driver is in use on several other systems for several years each. No problems until now. The only time I've ever seen this, the only thing that solved the problem was swapping the network card out for a better one. That's not to say it isn't a driver problem, as the new network card used a different driver as well. I think that the NIC is on the logic board. I can try to install a PCI card and use that in its place to see if the problem goes away. Should I bother? FWIW, a reboot of the system did not help. Jaime ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ping: sendto: No buffer space available
[EMAIL PROTECTED] wrote: On Tue, 17 Jun 2003, Bill Moran wrote: What make/model of NIC are you using? cerberus# ifconfig -a fxp0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500 inet 10.0.3.2 netmask 0xff00 broadcast 10.0.3.255 ether 00:e0:81:21:45:8c media: Ethernet autoselect (10baseT/UTP) status: active SNIP The interface in question is 10.0.3.2. That interface has worked fine for over a year. That driver is in use on several other systems for several years each. No problems until now. I, too, have use Intel cards with the fxp driver quite often, with no problems. The only time I've ever seen this, the only thing that solved the problem was swapping the network card out for a better one. That's not to say it isn't a driver problem, as the new network card used a different driver as well. I think that the NIC is on the logic board. I can try to install a PCI card and use that in its place to see if the problem goes away. Should I bother? I would. There are two possibilities that I would consider here: a) The NIC has gone flaky with age b) Newer drivers don't talk to that particular NIC as well as the old Did you notice this starting to happen after a particular upgrade? You may be able to correlate this with a particular update to the driver by looking at dates in the cvs logs. This is hearsay, and I have no personal experience with it, but I've seen lots of complaints across the lists about onboard cards that use the fxp driver not being very good. I've never had (nor heard of) any problems with the PCI versions. Another possibility is hardware ... have you added any hardware or changed any BIOS settings? There's the possibility of interrupt problems. I'm just shooting out ideas for you to work with. Please distill everything I've said through your own experience. i.e. take it with a grain of salt, as I don't _know_ what your problem is. FWIW, a reboot of the system did not help. Never helped for me either. You may want to check, but in my experience the output of 'netstat -m' will also tell you that you have plenty of network buffers available. -- Bill Moran Potential Technologies http://www.potentialtech.com ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ping: sendto: No buffer space available
On Tue, 17 Jun 2003, Bill Moran wrote: I think that the NIC is on the logic board. I can try to install a PCI card and use that in its place to see if the problem goes away. Should I bother? I would. There are two possibilities that I would consider here: a) The NIC has gone flaky with age b) Newer drivers don't talk to that particular NIC as well as the old Did you notice this starting to happen after a particular upgrade? You may be able to correlate this with a particular update to the driver by looking at dates in the cvs logs. Nope. The problem is only a few days old and the OS is 4.7-Stable. I think that the last update was in February or so. This is hearsay, and I have no personal experience with it, but I've seen lots of complaints across the lists about onboard cards that use the fxp driver not being very good. I've never had (nor heard of) any problems with the PCI versions. Hrm An interesting thought Another possibility is hardware ... have you added any hardware or changed any BIOS settings? There's the possibility of interrupt problems. No. The system was up for more than 2 months before the problems began. I'm just shooting out ideas for you to work with. Please distill everything I've said through your own experience. i.e. take it with a grain of salt, as I don't _know_ what your problem is. I always try to take email list advice this way. :) Never helped for me either. You may want to check, but in my experience the output of 'netstat -m' will also tell you that you have plenty of network buffers available. bash-2.05b$ netstat -m 144/768/26624 mbufs in use (current/peak/max): 139 mbufs allocated to data 5 mbufs allocated to packet headers 138/572/6656 mbuf clusters in use (current/peak/max) 1336 Kbytes allocated to network (6% of mb_map in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines That was durring normal operation. The following are at the tail end of one of the outages: bash-2.05b$ netstat -m 477/768/26624 mbufs in use (current/peak/max): 386 mbufs allocated to data 91 mbufs allocated to packet headers 384/572/6656 mbuf clusters in use (current/peak/max) 1336 Kbytes allocated to network (6% of mb_map in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines bash-2.05b$ netstat -m 476/768/26624 mbufs in use (current/peak/max): 387 mbufs allocated to data 89 mbufs allocated to packet headers 385/572/6656 mbuf clusters in use (current/peak/max) 1336 Kbytes allocated to network (6% of mb_map in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines bash-2.05b$ netstat -m 182/768/26624 mbufs in use (current/peak/max): 149 mbufs allocated to data 33 mbufs allocated to packet headers 147/572/6656 mbuf clusters in use (current/peak/max) 1336 Kbytes allocated to network (6% of mb_map in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines bash-2.05b$ netstat -m 156/768/26624 mbufs in use (current/peak/max): 153 mbufs allocated to data 3 mbufs allocated to packet headers 151/572/6656 mbuf clusters in use (current/peak/max) 1336 Kbytes allocated to network (6% of mb_map in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines bash-2.05b$ netstat -m 135/768/26624 mbufs in use (current/peak/max): 134 mbufs allocated to data 1 mbufs allocated to packet headers 132/572/6656 mbuf clusters in use (current/peak/max) 1336 Kbytes allocated to network (6% of mb_map in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines bash-2.05b$ netstat -m 144/768/26624 mbufs in use (current/peak/max): 139 mbufs allocated to data 5 mbufs allocated to packet headers 136/572/6656 mbuf clusters in use (current/peak/max) 1336 Kbytes allocated to network (6% of mb_map in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines It looks like something is causing it to pile up packets in the buffers temporarily. Any thoughts? In the mean time, I will see if I can dig up a PCI ethernet card. Thanks, Jaime ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ping: sendto: No buffer space available
[EMAIL PROTECTED] wrote: On Tue, 17 Jun 2003, Bill Moran wrote: I think that the NIC is on the logic board. I can try to install a PCI card and use that in its place to see if the problem goes away. Should I bother? I would. There are two possibilities that I would consider here: a) The NIC has gone flaky with age b) Newer drivers don't talk to that particular NIC as well as the old Another possibility that bites me in the ass when I'm not looking is link-level problems. Occasionally I've had weird issues that were resolved by replacing a switch or patch cable, or by moving to a different port on a switch. As usual ... just throwing ideas at you. Never helped for me either. You may want to check, but in my experience the output of 'netstat -m' will also tell you that you have plenty of network buffers available. bash-2.05b$ netstat -m 144/768/26624 mbufs in use (current/peak/max): 139 mbufs allocated to data 5 mbufs allocated to packet headers 138/572/6656 mbuf clusters in use (current/peak/max) 1336 Kbytes allocated to network (6% of mb_map in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines That was durring normal operation. The following are at the tail end of one of the outages: bash-2.05b$ netstat -m 477/768/26624 mbufs in use (current/peak/max): 386 mbufs allocated to data 91 mbufs allocated to packet headers 384/572/6656 mbuf clusters in use (current/peak/max) 1336 Kbytes allocated to network (6% of mb_map in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines snip additional netstat -m output 144/768/26624 mbufs in use (current/peak/max): 139 mbufs allocated to data 5 mbufs allocated to packet headers 136/572/6656 mbuf clusters in use (current/peak/max) 1336 Kbytes allocated to network (6% of mb_map in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines It looks like something is causing it to pile up packets in the buffers temporarily. Any thoughts? In the mean time, I will see if I can dig up a PCI ethernet card. Yes, but it doesn't look like the pile is deep enough that it should have run out of buffer space. This one is a bit of a shot in the dark, but try using rndcontrol to increase the entropy collection. I'm not sure why I think this might help, but I have some vague memory of it helping somewhere. -- Bill Moran Potential Technologies http://www.potentialtech.com ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ping: sendto: No buffer space available
Bill Moran writes: [EMAIL PROTECTED] wrote: It looks like something is causing it to pile up packets in the buffers temporarily. Any thoughts? In the mean time, I will see if I can dig up a PCI ethernet card. Yes, but it doesn't look like the pile is deep enough that it should have run out of buffer space. The ``No buffer space available'' message generally has _nothing at all_ to do with whether there are enough mbufs available. It really means that the send queue in the driver is full and no further packets can be added to it until it drains soemwhat. The message indicates that, for some reason, the driver can't send out any packets on the wire. For some reason most people think that this message means they've run out of mbufs. Examination of the source would quickly disabuse them of this idea. --- Gary Jennejohn / garyj[at]jennejohn.org gj[at]freebsd.org gj[at]denx.de ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ping: sendto: No buffer space available
Gary Jennejohn wrote: Bill Moran writes: [EMAIL PROTECTED] wrote: It looks like something is causing it to pile up packets in the buffers temporarily. Any thoughts? In the mean time, I will see if I can dig up a PCI ethernet card. Yes, but it doesn't look like the pile is deep enough that it should have run out of buffer space. The ``No buffer space available'' message generally has _nothing at all_ to do with whether there are enough mbufs available. It really means that the send queue in the driver is full and no further packets can be added to it until it drains soemwhat. The message indicates that, for some reason, the driver can't send out any packets on the wire. So the comment that I made that it was either a driver, NIC, or link-level problem was near the mark? For some reason most people think that this message means they've run out of mbufs. Examination of the source would quickly disabuse them of this idea. My original comment was meant to say that. But apparently I didn't communicate well enough. I spent a while looking through the source to get a better idea of where that error originates, and only got frustrated. As a favor, can you point me to the area of the source from which I can learn more of this? -- Bill Moran Potential Technologies http://www.potentialtech.com ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ping: sendto: No buffer space available
Bill Moran writes: So the comment that I made that it was either a driver, NIC, or link-level problem was near the mark? Seems to me that it is. I'd suspect a link level problem myself, based on the description of the problem. I spent a while looking through the source to get a better idea of where that error originates, and only got frustrated. As a favor, can you point me to the area of the source from which I can learn more of this? In -current things have changed, so I could now be wrong. Most of the drivers now return ENOBUFS if they really can't get an mbuf, but I haven't examined all the drivers to see under just which circumstances they return ENOBUFS. However, I was thinking of the macro IF_HANDOFF(), which uses if_handoff() which uses _IF_QFULL(), all of which are defined in /sys/net/if_var.h. _IF_QFULL() actually checks whether the send queue is full. IF_HANDOFF() is invoked (among other places) in /sys/net/netisr.c and /sys/net/if_ethersubr.c. I remember running into this situation while debugging ISDN drivers, many of which use IF_HANDOFF(). Most moden ethernet drivers don't seem to use it any more. --- Gary Jennejohn / garyj[at]jennejohn.org gj[at]freebsd.org gj[at]denx.de ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ping: sendto: No buffer space available
FWIW, I think that I found the problem. With the help of our ISP, we've found that one of my servers has been dumping so many packets out to the Internet that our router was dropping packets. I've unplugged it at this point and we do not have the same symptoms at this time. The clues to a crack are evident, too. A process /usr/sbin/nscd is running on the box according to top and ps, but the file does not exist. Further more, I never told such a process to execute. Shortly after a reboot, a netstat command showed a connection to 37303 on a remote host. I was the only person logged in and I did not initiate that connection. Obviously, I'll be taking steps to find the crack and remote it. :) If anyone wants to suggest something to check, I'd appreciate it. Jaime -- To affect the quality of the day, that is the highest of arts. - Henry David Thoreau, _Where_I_Live_ ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ping: sendto: No buffer space available
Jaime wrote: FWIW, I think that I found the problem. With the help of our ISP, we've found that one of my servers has been dumping so many packets out to the Internet that our router was dropping packets. I've unplugged it at this point and we do not have the same symptoms at this time. The clues to a crack are evident, too. A process /usr/sbin/nscd is running on the box according to top and ps, but the file does not exist. Further more, I never told such a process to execute. Shortly after a reboot, a netstat command showed a connection to 37303 on a remote host. I was the only person logged in and I did not initiate that connection. Obviously, I'll be taking steps to find the crack and remote it. :) If anyone wants to suggest something to check, I'd appreciate it. I found a web page that claims that nscd is a Debian program called name service cache daemon. (Cache only DNS server?) So if it's connecting to any port other than DNS, it's probably a trojan pretending to be nscd. -- Bill Moran Potential Technologies http://www.potentialtech.com ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: ping: sendto: No buffer space available
On Tuesday, June 17, 2003, at 09:36 PM, Bill Moran wrote: I found a web page that claims that nscd is a Debian program called name service cache daemon. (Cache only DNS server?) So if it's connecting to any port other than DNS, it's probably a trojan pretending to be nscd. I think that I found the same page. I agree with your assessment. The IP address that it is attempting to connect to is not found via traceroute and is registered to what appears to be a Russian ISP. How odd I'll be grabbing new source code and recompiling everything tomorrow. The box was running 4.7-Stable anyway. :) The troubling part is that the process claims to be /usr/sbin/nscd, but that file doesn't exist. I'll have to see how they did that with lsof, mergemaster, etc. Jaime ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: No buffer space available
Eric Anderson [EMAIL PROTECTED] writes: 131/32768/32768 mbuf clusters in use (current/peak/max) You ran out of mbuf clusters at some point. How can I fix this? Add kern.ipc.nmbclusters=65536 to /boot/loader.conf and reboot. BTW, -chat is not the appropriate forum for this kind of question. DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
Re: No buffer space available
Dag-Erling Smorgrav wrote: Eric Anderson [EMAIL PROTECTED] writes: 131/32768/32768 mbuf clusters in use (current/peak/max) You ran out of mbuf clusters at some point. What part of the netstat -m indicated this? How can I fix this? Add kern.ipc.nmbclusters=65536 to /boot/loader.conf and reboot. netstat -m shows I only used around 190 clusters out of the 32768 I have set up. How would 65536 help me? BTW, -chat is not the appropriate forum for this kind of question. First, I already posted my question to -questions, and had no responses. Second, -chat is open to pretty much all discussions - at least, I would assume it isn't a problem to chat about FreeBSD problems when there has been very long threads on religion, etc. If FreeBSD discussions are not appropriate for the freebsd-chat list, let me know, and by all means I'll start posting them to freebsd-religion. ;) Eric -- -- Eric Anderson Systems Administrator Centaur Technology Beware the fury of a patient man. -- To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
Re: No buffer space available
Eric Anderson [EMAIL PROTECTED] writes: Dag-Erling Smorgrav wrote: Eric Anderson [EMAIL PROTECTED] writes: 131/32768/32768 mbuf clusters in use (current/peak/max) You ran out of mbuf clusters at some point. What part of the netstat -m indicated this? Well, duh. The part I quoted. DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
Re: No buffer space available
Dag-Erling Smorgrav wrote: Eric Anderson [EMAIL PROTECTED] writes: Dag-Erling Smorgrav wrote: Eric Anderson [EMAIL PROTECTED] writes: 131/32768/32768 mbuf clusters in use (current/peak/max) You ran out of mbuf clusters at some point. What part of the netstat -m indicated this? Well, duh. The part I quoted. It's going to be one of those days, isn't it? :) Sorry. I'm a dumbass. Eric -- -- Eric Anderson Systems Administrator Centaur Technology Beware the fury of a patient man. -- To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
Re: No buffer space available
Eric Anderson [EMAIL PROTECTED] writes: Sorry. I'm a dumbass. We all have our moments :) DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
No buffer space available
I'm running a heavily loaded NFS server, and I am now seeing (occasionally) things like: yp_next: clnt_call: RPC: Unable to send; errno = No buffer space available yp_next: clnt_call: RPC: Unable to send; errno = No buffer space available yp_next: clnt_call: RPC: Unable to send; errno = No buffer space available My NIS client software is complaining No buffer space available. Here is output from netstat -m: 178/38224/131072 mbufs in use (current/peak/max): 174 mbufs allocated to data 4 mbufs allocated to packet headers 131/32768/32768 mbuf clusters in use (current/peak/max) 75092 Kbytes allocated to network (33% of mb_map in use) 581 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines Notice the memory denied numbers.. /etc/sysctl.conf: vfs.nfs.gatherdelay=0 vfs.nfs.async=1 vfs.vmiodirenable=1 kern.ipc.maxsockbuf=2097152 kern.ipc.somaxconn=8192 kern.ipc.maxsockets=16424 kern.maxfiles=65536 kern.maxfilesperproc=32768 net.inet.tcp.rfc1323=1 net.inet.tcp.delayed_ack=0 net.inet.tcp.sendspace=65535 net.inet.tcp.recvspace=65535 net.inet.udp.recvspace=65535 net.inet.udp.maxdgram=57344 net.local.stream.recvspace=65535 net.local.stream.sendspace=65535 net.inet.ip.forwarding=1 Anything else I'm missing? How can I fix this? -- -- Eric Anderson Systems Administrator Centaur Technology Beware the fury of a patient man. -- To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-questions in the body of the message
