SV: [pfSense Support] Running out of memory
Hi again It seems like its tcpdump that is causing the problem. Both machines are running 1.2.2. One is upgraded from 1.0.1 - 1.2 - 1.2.2. The other is a fresh install. They were bootet 12 days ago. Just after a reboot they use app. 8% of memory, and that has now increased to app. 50%. USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND root 554 3.3 27.7 288320 286384 d0- S 4Sep09 240:11.90 /usr/sbin/tcpdum any ideas? Kind regards Anders -Oprindelig meddelelse- Fra: a_subscribti...@fiberby.dk [mailto:a_subscribti...@fiberby.dk] Sendt: 4. september 2009 09:48 Til: support@pfsense.com Emne: SV: [pfSense Support] Running out of memory Thanks for the quick response. I surely do not have any packages installed. I didn't know of 'ps aux', but now I'll keep an eye on it. Kind regards Anders -Oprindelig meddelelse- Fra: cbuech...@gmail.com [mailto:cbuech...@gmail.com] På vegne af Chris Buechler Sendt: 4. september 2009 09:25 Til: support@pfsense.com Emne: Re: [pfSense Support] Running out of memory On Fri, Sep 4, 2009 at 3:20 AM, a_subscribti...@fiberby.dk wrote: Hi I have two pfsense running for approximately 1300 costumers each. They hold between 10-15 vlans each, and is acting as dhcp servers. All traffic is being natted. It's a full install with no packages installed. The platform they are running on is supermicro servers, with 1GB ram. These are the services that is running: DNS Forwarder (all users are given real dns-server addresses) NTP clock sync DHCP Service SNMP Service UPnP Service My problem is related to the use of memory. Over time they use more and more of the memory, and after approximately two month the memory is full (90+%), and they start to use the swap file. After that I restart them, because I can lower the amount of cpu used. You must have some package installed that has a memory leak. Nothing I'm aware of in the base system will do that. Check 'ps aux' in Diagnostics - Command for what process is leaking memory. - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense Support] Running out of memory
I'd say just run kill 554 but that wouldn't fix the underlying issue. Curtis LaMasters http://www.curtis-lamasters.com http://www.builtnetworks.com On Wed, Sep 16, 2009 at 2:36 AM, a_subscribti...@fiberby.dk wrote: Hi again It seems like its tcpdump that is causing the problem. Both machines are running 1.2.2. One is upgraded from 1.0.1 - 1.2 - 1.2.2. The other is a fresh install. They were bootet 12 days ago. Just after a reboot they use app. 8% of memory, and that has now increased to app. 50%. USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND root 554 3.3 27.7 288320 286384 d0- S 4Sep09 240:11.90 /usr/sbin/tcpdum any ideas? Kind regards Anders -Oprindelig meddelelse- Fra: a_subscribti...@fiberby.dk [mailto:a_subscribti...@fiberby.dk] Sendt: 4. september 2009 09:48 Til: support@pfsense.com Emne: SV: [pfSense Support] Running out of memory Thanks for the quick response. I surely do not have any packages installed. I didn't know of 'ps aux', but now I'll keep an eye on it. Kind regards Anders -Oprindelig meddelelse- Fra: cbuech...@gmail.com [mailto:cbuech...@gmail.com] På vegne af Chris Buechler Sendt: 4. september 2009 09:25 Til: support@pfsense.com Emne: Re: [pfSense Support] Running out of memory On Fri, Sep 4, 2009 at 3:20 AM, a_subscribti...@fiberby.dk wrote: Hi I have two pfsense running for approximately 1300 costumers each. They hold between 10-15 vlans each, and is acting as dhcp servers. All traffic is being natted. It's a full install with no packages installed. The platform they are running on is supermicro servers, with 1GB ram. These are the services that is running: DNS Forwarder (all users are given real dns-server addresses) NTP clock sync DHCP Service SNMP Service UPnP Service My problem is related to the use of memory. Over time they use more and more of the memory, and after approximately two month the memory is full (90+%), and they start to use the swap file. After that I restart them, because I can lower the amount of cpu used. You must have some package installed that has a memory leak. Nothing I'm aware of in the base system will do that. Check 'ps aux' in Diagnostics - Command for what process is leaking memory. - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense Support] Running out of memory
On Wed, Sep 16, 2009 at 09:36, a_subscribti...@fiberby.dk wrote: It seems like its tcpdump that is causing the problem. Both machines are running 1.2.2. One is upgraded from 1.0.1 - 1.2 - 1.2.2. The other is a fresh install. They were bootet 12 days ago. Just after a reboot they use app. 8% of memory, and that has now increased to app. 50%. USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND root 554 3.3 27.7 288320 286384 d0- S 4Sep09 240:11.90 /usr/sbin/tcpdum It very much looks like the tcpdump process generating the firewall log messages. Here the section from ps from a 1.2.1-embedded (although it's the same on 1.2.3-rc1): router:~# ps aux | grep tcpd root 450 0.0 21.7 56896 54936 d0- S 2Jan09 81:46.02 /usr/sbin/tcpdump -s 256 -v -l -n -e -ttt -i pflog0 Command line option -l activates line buffering -i t might be a memory leak in tcpdump (although it hasn't been a problem on any machine of ours). Do you have an unusually high load of connections and/or logging enabled on firewall-rules ? Tried to disable the logging of the default deny-all rule ? -Aarno -- Aarno Aukia Atrila GmbH Switzerland - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense Support] Help with static routing
Chris Buechler Fri, 11 Sep 2009 22:36:27 -0700 The configuration is 100% completely identical to m0n0wall for what you're doing. Finally, the problem was the Layer 3 switch! We had it replaced and now everything works fine. I went onsite and i found the problem by capturing the traffic with Wireshark. The returning packets were dying inside the switch. Don't ask me why, i don't know! Long live pfSense and thanks everybody for your help! Guy Boisvert, ing. IngTegration inc. - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: [pfSense Support] Help with static routing
On Wed, Sep 16, 2009 at 09:39:38AM -0400, Guy Boisvert wrote: Finally, the problem was the Layer 3 switch! We had it replaced and now everything works fine. Which switch? Not ProCurve, I hope? -- Eugen* Leitl a href=http://leitl.org;leitl/a http://leitl.org __ ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org
SV: [pfSense Support] Running out of memory
That immediately reduced the memory use from 50% -22% But as you state, it doesn't solve the underlying problem. Cheers Anders -Oprindelig meddelelse- Fra: Curtis LaMasters [mailto:curtislamast...@gmail.com] Sendt: 16. september 2009 14:13 Til: support@pfsense.com Emne: Re: [pfSense Support] Running out of memory I'd say just run kill 554 but that wouldn't fix the underlying issue. Curtis LaMasters http://www.curtis-lamasters.com http://www.builtnetworks.com On Wed, Sep 16, 2009 at 2:36 AM, a_subscribti...@fiberby.dk wrote: Hi again It seems like its tcpdump that is causing the problem. Both machines are running 1.2.2. One is upgraded from 1.0.1 - 1.2 - 1.2.2. The other is a fresh install. They were bootet 12 days ago. Just after a reboot they use app. 8% of memory, and that has now increased to app. 50%. USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND root 554 3.3 27.7 288320 286384 d0- S 4Sep09 240:11.90 /usr/sbin/tcpdum any ideas? Kind regards Anders -Oprindelig meddelelse- Fra: a_subscribti...@fiberby.dk [mailto:a_subscribti...@fiberby.dk] Sendt: 4. september 2009 09:48 Til: support@pfsense.com Emne: SV: [pfSense Support] Running out of memory Thanks for the quick response. I surely do not have any packages installed. I didn't know of 'ps aux', but now I'll keep an eye on it. Kind regards Anders -Oprindelig meddelelse- Fra: cbuech...@gmail.com [mailto:cbuech...@gmail.com] På vegne af Chris Buechler Sendt: 4. september 2009 09:25 Til: support@pfsense.com Emne: Re: [pfSense Support] Running out of memory On Fri, Sep 4, 2009 at 3:20 AM, a_subscribti...@fiberby.dk wrote: Hi I have two pfsense running for approximately 1300 costumers each. They hold between 10-15 vlans each, and is acting as dhcp servers. All traffic is being natted. It's a full install with no packages installed. The platform they are running on is supermicro servers, with 1GB ram. These are the services that is running: DNS Forwarder (all users are given real dns-server addresses) NTP clock sync DHCP Service SNMP Service UPnP Service My problem is related to the use of memory. Over time they use more and more of the memory, and after approximately two month the memory is full (90+%), and they start to use the swap file. After that I restart them, because I can lower the amount of cpu used. You must have some package installed that has a memory leak. Nothing I'm aware of in the base system will do that. Check 'ps aux' in Diagnostics - Command for what process is leaking memory. - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org
SV: [pfSense Support] Running out of memory
On Wed, Sep 16, 2009 at 09:36, a_subscribti...@fiberby.dk wrote: It seems like its tcpdump that is causing the problem. Both machines are running 1.2.2. One is upgraded from 1.0.1 - 1.2 - 1.2.2. The other is a fresh install. They were bootet 12 days ago. Just after a reboot they use app. 8% of memory, and that has now increased to app. 50%. USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND root 554 3.3 27.7 288320 286384 d0- S 4Sep09 240:11.90 /usr/sbin/tcpdum It very much looks like the tcpdump process generating the firewall log messages. Here the section from ps from a 1.2.1-embedded (although it's the same on 1.2.3-rc1): router:~# ps aux | grep tcpd root 450 0.0 21.7 56896 54936 d0- S 2Jan09 81:46.02 /usr/sbin/tcpdump -s 256 -v -l -n -e -ttt -i pflog0 Command line option -l activates line buffering -i t might be a memory leak in tcpdump (although it hasn't been a problem on any machine of ours). Do you have an unusually high load of connections and/or logging enabled on firewall-rules ? Tried to disable the logging of the default deny-all rule ? What is a high load of connections and how do I determine it? I haven't any logging enabled, and now I've disabled the logging of default deny rule. I'll be back with new information... Thanks Anders - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org
SV: [pfSense Support] Running out of memory
Although it only worked on one of my machines!? -Oprindelig meddelelse- Fra: a_subscribti...@fiberby.dk [mailto:a_subscribti...@fiberby.dk] Sendt: 16. september 2009 15:55 Til: support@pfsense.com Emne: SV: [pfSense Support] Running out of memory That immediately reduced the memory use from 50% -22% But as you state, it doesn't solve the underlying problem. Cheers Anders -Oprindelig meddelelse- Fra: Curtis LaMasters [mailto:curtislamast...@gmail.com] Sendt: 16. september 2009 14:13 Til: support@pfsense.com Emne: Re: [pfSense Support] Running out of memory I'd say just run kill 554 but that wouldn't fix the underlying issue. Curtis LaMasters http://www.curtis-lamasters.com http://www.builtnetworks.com On Wed, Sep 16, 2009 at 2:36 AM, a_subscribti...@fiberby.dk wrote: Hi again It seems like its tcpdump that is causing the problem. Both machines are running 1.2.2. One is upgraded from 1.0.1 - 1.2 - 1.2.2. The other is a fresh install. They were bootet 12 days ago. Just after a reboot they use app. 8% of memory, and that has now increased to app. 50%. USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND root 554 3.3 27.7 288320 286384 d0- S 4Sep09 240:11.90 /usr/sbin/tcpdum any ideas? Kind regards Anders -Oprindelig meddelelse- Fra: a_subscribti...@fiberby.dk [mailto:a_subscribti...@fiberby.dk] Sendt: 4. september 2009 09:48 Til: support@pfsense.com Emne: SV: [pfSense Support] Running out of memory Thanks for the quick response. I surely do not have any packages installed. I didn't know of 'ps aux', but now I'll keep an eye on it. Kind regards Anders -Oprindelig meddelelse- Fra: cbuech...@gmail.com [mailto:cbuech...@gmail.com] På vegne af Chris Buechler Sendt: 4. september 2009 09:25 Til: support@pfsense.com Emne: Re: [pfSense Support] Running out of memory On Fri, Sep 4, 2009 at 3:20 AM, a_subscribti...@fiberby.dk wrote: Hi I have two pfsense running for approximately 1300 costumers each. They hold between 10-15 vlans each, and is acting as dhcp servers. All traffic is being natted. It's a full install with no packages installed. The platform they are running on is supermicro servers, with 1GB ram. These are the services that is running: DNS Forwarder (all users are given real dns-server addresses) NTP clock sync DHCP Service SNMP Service UPnP Service My problem is related to the use of memory. Over time they use more and more of the memory, and after approximately two month the memory is full (90+%), and they start to use the swap file. After that I restart them, because I can lower the amount of cpu used. You must have some package installed that has a memory leak. Nothing I'm aware of in the base system will do that. Check 'ps aux' in Diagnostics - Command for what process is leaking memory. - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: SV: [pfSense Support] Running out of memory
a_subscribti...@fiberby.dk wrote: That immediately reduced the memory use from 50% -22% But as you state, it doesn't solve the underlying problem. Cheers Anders -Oprindelig meddelelse- Fra: Curtis LaMasters [mailto:curtislamast...@gmail.com] Sendt: 16. september 2009 14:13 Til: support@pfsense.com Emne: Re: [pfSense Support] Running out of memory I'd say just run kill 554 but that wouldn't fix the underlying issue. Curtis LaMasters http://www.curtis-lamasters.com http://www.builtnetworks.com --snip-- Are you using Packet Capture on this box? I have noticed that large packet captures raise the memory usage and don't seem to release it until I run another small capture to overwrite the file. -Oliver - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org
Re: SV: [pfSense Support] Running out of memory
On Wed, Sep 16, 2009 at 11:42 AM, Oliver Hansen oliver.han...@gmail.com wrote: a_subscribti...@fiberby.dk wrote: That immediately reduced the memory use from 50% -22% But as you state, it doesn't solve the underlying problem. Thanks, I just committed a change to prevent this from being a problem. Scott - To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org
