Re: [pfSense] High-latency when traffic reaches 80% wirespeed
run "top -SH" to find the top cpu consuming tasks On Thu, Oct 5, 2017 at 8:44 AM, Christoph Haas wrote: > Am Mittwoch, den 04.10.2017, 15:05 -0400 schrieb ED Fochler: > > I have a similar situation and I solved it with limiters. I'm also a > fan of limiters to ensure fair sharing of uplink bandwidth by internal > users. I haven't tried changing system tunables though, so that solution > may be better. > > So far the situation was better this morning. But the web interface > became unresponsive and the OpenVPN daemon died. So I'm still scared. > > > > Nothing is sent through the limiter until you create a rule that catches > the traffic and routes it through the limiter, so you're not going to > accidentally slow everything down just by creating a rule. > > I will try that. > > > > The behavior you're speaking of sounds like your machine is getting maxed > out by interrupts or some internal bandwidth. Setting up a limiter sounds > like a better solution than pushing the hardware to the point of unrefined > behavior. > > Yes, I suspect something like that, too. The system load is going up > heavily (Load >=5) sometimes. However the web interface claims that the > load is around 30%. RAM and state tables look fine, too. > > On Linux-based systems I regularly use iptables rules and often go near > wire speed. But the system load rarely goes up noticably. So I wonder > what part is really causing that load. > > I ran "top" this morning and saw that the "filterlog" process was at > the top of the list. My firewall rules though do not do any logging at > the moment. Could that still be a problem? > > Thanks for your suggestions so far. I'll try them all. > > …Christoph > > ___ > pfSense mailing list > https://lists.pfsense.org/mailman/listinfo/list > Support the project with Gold! https://pfsense.org/gold > -- Ivo R. Tonev +55 61 98409-2642 i...@tonev.com.br ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] High-latency when traffic reaches 80% wirespeed
Am Mittwoch, den 04.10.2017, 15:05 -0400 schrieb ED Fochler: > I have a similar situation and I solved it with limiters. I'm also a fan of > limiters to ensure fair sharing of uplink bandwidth by internal users. I > haven't tried changing system tunables though, so that solution may be better. So far the situation was better this morning. But the web interface became unresponsive and the OpenVPN daemon died. So I'm still scared. > Nothing is sent through the limiter until you create a rule that catches the traffic and routes it through the limiter, so you're not going to accidentally slow everything down just by creating a rule. I will try that. > The behavior you're speaking of sounds like your machine is getting maxed out by interrupts or some internal bandwidth. Setting up a limiter sounds like a better solution than pushing the hardware to the point of unrefined behavior. Yes, I suspect something like that, too. The system load is going up heavily (Load >=5) sometimes. However the web interface claims that the load is around 30%. RAM and state tables look fine, too. On Linux-based systems I regularly use iptables rules and often go near wire speed. But the system load rarely goes up noticably. So I wonder what part is really causing that load. I ran "top" this morning and saw that the "filterlog" process was at the top of the list. My firewall rules though do not do any logging at the moment. Could that still be a problem? Thanks for your suggestions so far. I'll try them all. …Christoph ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] High-latency when traffic reaches 80% wirespeed
Am Mittwoch, den 04.10.2017, 19:13 + schrieb Steve Yates: > Christoph, if you are using CARP/HA for your two routers, see > https://redmine.pfsense.org/issues/4310 "Limiters + HA results in hangs on > secondary." Not yet but I'll look out to that. Thanks. > Alternatively if the overnight traffic is due to an rsync, rsync can limit > its own bandwidth also. I suspect some kind of backup job. There are many different data transfers going on in the network. So a general solution like limiting sounds better. ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] High-latency when traffic reaches 80% wirespeed
Christoph, if you are using CARP/HA for your two routers, see https://redmine.pfsense.org/issues/4310 "Limiters + HA results in hangs on secondary." Alternatively if the overnight traffic is due to an rsync, rsync can limit its own bandwidth also. -- Steve Yates ITS, Inc. -Original Message- From: List [mailto:list-boun...@lists.pfsense.org] On Behalf Of ED Fochler Sent: Wednesday, October 4, 2017 2:05 PM To: pfSense Support and Discussion Mailing List Subject: Re: [pfSense] High-latency when traffic reaches 80% wirespeed I have a similar situation and I solved it with limiters. I'm also a fan of limiters to ensure fair sharing of uplink bandwidth by internal users. I haven't tried changing system tunables though, so that solution may be better. Nothing is sent through the limiter until you create a rule that catches the traffic and routes it through the limiter, so you're not going to accidentally slow everything down just by creating a rule. The behavior you're speaking of sounds like your machine is getting maxed out by interrupts or some internal bandwidth. Setting up a limiter sounds like a better solution than pushing the hardware to the point of unrefined behavior. ED. > On 2017, Oct 4, at 4:08 AM, Christoph Haas wrote: > > Dear list, > > I have become a huge fan of pfSense and managed to replace our old > routers at work by two nifty Netgate SG-4860 gateways. They work nearly > perfectly. I just have a few seperate internal VLANs (e.g. for > administration, monitoring and backup) that give me a headache. Every > day at the same time(s) there are spikes in traffic (I can see in the > dashboard) between two VLANs. Traffic goes up to pretty much 800 Mbps > for 1-2 minutes. > > During that time our monitoring system goes wild. High latencies and > even ping losses. CPU load of the router is shown at around 50%. Once > the traffic goes below 800 Mbps all is instantly fine again. > > I tried to simplify the firewall rules (e.g. let through all the > traffic) but that did not help. Is there anything I can do? Any hidden > switches? Anything to find and fix the situation? Traffic shaping for > ICMP? Unicorn dust? > > Thanks in advance for your hints. > > …Christoph > ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] High-latency when traffic reaches 80% wirespeed
I have a similar situation and I solved it with limiters. I'm also a fan of limiters to ensure fair sharing of uplink bandwidth by internal users. I haven't tried changing system tunables though, so that solution may be better. Nothing is sent through the limiter until you create a rule that catches the traffic and routes it through the limiter, so you're not going to accidentally slow everything down just by creating a rule. The behavior you're speaking of sounds like your machine is getting maxed out by interrupts or some internal bandwidth. Setting up a limiter sounds like a better solution than pushing the hardware to the point of unrefined behavior. ED. > On 2017, Oct 4, at 4:08 AM, Christoph Haas wrote: > > Dear list, > > I have become a huge fan of pfSense and managed to replace our old > routers at work by two nifty Netgate SG-4860 gateways. They work nearly > perfectly. I just have a few seperate internal VLANs (e.g. for > administration, monitoring and backup) that give me a headache. Every > day at the same time(s) there are spikes in traffic (I can see in the > dashboard) between two VLANs. Traffic goes up to pretty much 800 Mbps > for 1-2 minutes. > > During that time our monitoring system goes wild. High latencies and > even ping losses. CPU load of the router is shown at around 50%. Once > the traffic goes below 800 Mbps all is instantly fine again. > > I tried to simplify the firewall rules (e.g. let through all the > traffic) but that did not help. Is there anything I can do? Any hidden > switches? Anything to find and fix the situation? Traffic shaping for > ICMP? Unicorn dust? > > Thanks in advance for your hints. > > …Christoph > ___ > pfSense mailing list > https://lists.pfsense.org/mailman/listinfo/list > Support the project with Gold! https://pfsense.org/gold ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] High-latency when traffic reaches 80% wirespeed
You can try rise some "System tunables" net.inet.tcp.recvspace 524288 net.inet.tcp.sendspace 524288 net.raw.recvspace 524288 net.inet.raw.recvspace 524288 net.raw.sendspace 524288 net.inet.raw.maxdgram 524288 net.link.ifqmaxlen 2048 net.inet.tcp.recvbuf_inc 65536 net.inet.udp.recvspace 524288 net.inet.tcp.sendbuf_inc 65536 net.inet.tcp.mssdflt 1460 net.inet.tcp.minmss 536 On Wed, Oct 4, 2017 at 5:08 AM, Christoph Haas wrote: > Dear list, > > I have become a huge fan of pfSense and managed to replace our old > routers at work by two nifty Netgate SG-4860 gateways. They work nearly > perfectly. I just have a few seperate internal VLANs (e.g. for > administration, monitoring and backup) that give me a headache. Every > day at the same time(s) there are spikes in traffic (I can see in the > dashboard) between two VLANs. Traffic goes up to pretty much 800 Mbps > for 1-2 minutes. > > During that time our monitoring system goes wild. High latencies and > even ping losses. CPU load of the router is shown at around 50%. Once > the traffic goes below 800 Mbps all is instantly fine again. > > I tried to simplify the firewall rules (e.g. let through all the > traffic) but that did not help. Is there anything I can do? Any hidden > switches? Anything to find and fix the situation? Traffic shaping for > ICMP? Unicorn dust? > > Thanks in advance for your hints. > > …Christoph > ___ > pfSense mailing list > https://lists.pfsense.org/mailman/listinfo/list > Support the project with Gold! https://pfsense.org/gold -- Ivo R. Tonev +55 61 98409-2642 i...@tonev.com.br ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
[pfSense] High-latency when traffic reaches 80% wirespeed
Dear list, I have become a huge fan of pfSense and managed to replace our old routers at work by two nifty Netgate SG-4860 gateways. They work nearly perfectly. I just have a few seperate internal VLANs (e.g. for administration, monitoring and backup) that give me a headache. Every day at the same time(s) there are spikes in traffic (I can see in the dashboard) between two VLANs. Traffic goes up to pretty much 800 Mbps for 1-2 minutes. During that time our monitoring system goes wild. High latencies and even ping losses. CPU load of the router is shown at around 50%. Once the traffic goes below 800 Mbps all is instantly fine again. I tried to simplify the firewall rules (e.g. let through all the traffic) but that did not help. Is there anything I can do? Any hidden switches? Anything to find and fix the situation? Traffic shaping for ICMP? Unicorn dust? Thanks in advance for your hints. …Christoph ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold