Re: Hardware for a PF box
On Mon, 10 May 2010, Chris Smith wrote: What about logging in this case? Can PF logs be sent to another system running a syslog daemon? You answered your own question. ;) Look at the 'action' field explanation in the manual page for syslog.conf(5) About the diskless machine, many of the so-called diskless machines actually use flash or ssd instead of a spinning magnetic platter. The base installation of openbsd is still quite small. If you are only running PF, you will have a lot of space left over on a 1GB CF to make a logging partition. Flash can be very slow, so volitile caches can be stored in an mfs partition. /Lars
Re: Hardware for a PF box
On Tue, May 11, 2010 at 4:56 PM, Lars Nooden lars.cura...@gmail.com wrote: On Mon, 10 May 2010, Chris Smith wrote: What about logging in this case? Can PF logs be sent to another system running a syslog daemon? You answered your own question. ;) Look at the 'action' field explanation in the manual page for syslog.conf(5) About the diskless machine, many of the so-called diskless machines actually use flash or ssd instead of a spinning magnetic platter. The base installation of openbsd is still quite small. If you are only running PF, you will have a lot of space left over on a 1GB CF to make a logging partition. Flash can be very slow, so volitile caches can be stored in an mfs partition. /Lars OpenBSD will happily fit into about 160mb by installing only base and etc which provide plenty for a firewall. My 1.4GHz Toshiba laptop acting as a wireless-wired gateway runs OpenBSD 4.6 on a 512mb USB drive (which I'd like to replace with a CF disk on a 2.5 compatible adapter) with space to spare. Sure it doesn't do anywhere near as many packets as you propose, but it handles a constantly-running seedbox and my gaming together without skipping a beat, which is more than I can ask for. -- Aaron Mason - Programmer, open source addict I've taken my software vows - for beta or for worse
Re: Hardware for a PF box
Hello, I'll try to answer every suggestion... I'm going to buy brand new HP servers, DL360 G5 or DL165 G7. So the choice for CPU is between AMD Opteron 24xx or Intel Xeon 55xx. I've read that a PIII would be sufficient : I have performance issues actually, running on a Xeon 2.8GHz (monocore, FSB 800, socket 604). I don't think they come from PF BTW, it should be logging/relayd/OpenVPN which makes the box lag. I'm actually on a test with dual xeon E5420 on GEMERIC.MP, it runs like a charm. But it's borrowed hardware, I have to give it back :) I'm very interested in separated log machine, I think I'll do that. Could you give me an estimation on how many Mbps I need on the log server ? I think I'll put this on a VM, we have an ESX cluster connected to a CX3-40 SAN which should give enough disk I/O... Installing SSD on the machines is way more expensive with HP hardware : 72 GB SAS 15Ktpm costs 260b,, 60 GB SSD costs 950b,. HP offers no way to install a compact flash as disk drive. Networks cards are Intel Gb, using em(4) driver. So, with all your considerations, here's my actual setup : * Xeon E5504 quad core @2Ghz (don't need AMD's 6 cores, and costs nearly the same prize than the only dual core remaining, E5502 @1.86GHz) * 3*1GB memory (Xeon are triple channel, so I need three DIMM for maximal memory bandwidth) * 2x72 Gb SAS drives on raid0 Does it sound correct to you ? Do you have any suggestion/modification ? Thank you very much for the help. -- Cordialement, Pierre BARDOU -Message d'origine- DeB : Aaron Mason [mailto:simplersolut...@gmail.com] EnvoyC)B : mardi 11 mai 2010 14:01 CB : Lars Nooden CcB : misc@openbsd.org ObjetB : Re: Hardware for a PF box On Tue, May 11, 2010 at 4:56 PM, Lars Nooden lars.cura...@gmail.com wrote: On Mon, 10 May 2010, Chris Smith wrote: What about logging in this case? Can PF logs be sent to another system running a syslog daemon? You answered your own question. ;) Look at the 'action' field explanation in the manual page for syslog.conf(5) About the diskless machine, many of the so-called diskless machines actually use flash or ssd instead of a spinning magnetic platter. The base installation of openbsd is still quite small. If you are only running PF, you will have a lot of space left over on a 1GB CF to make a logging partition. Flash can be very slow, so volitile caches can be stored in an mfs partition. /Lars OpenBSD will happily fit into about 160mb by installing only base and etc which provide plenty for a firewall. My 1.4GHz Toshiba laptop acting as a wireless-wired gateway runs OpenBSD 4.6 on a 512mb USB drive (which I'd like to replace with a CF disk on a 2.5 compatible adapter) with space to spare. Sure it doesn't do anywhere near as many packets as you propose, but it handles a constantly-running seedbox and my gaming together without skipping a beat, which is more than I can ask for. -- Aaron Mason - Programmer, open source addict I've taken my software vows - for beta or for worse
Re: Hardware for a PF box
Sorry, typo : SAS drives would be on RAID1. So the config would be : * Xeon E5504 quad core @2Ghz (don't need AMD's 6 cores, and costs nearly the same prize than the only dual core remaining, E5502 @1.86GHz) * 3*1GB memory (Xeon are triple channel, so I need three DIMM for maximal memory bandwidth) * 2x72 Gb SAS drives on raid1 * GENERIC.MP kernel -- Cordialement, Pierre BARDOU -Message d'origine- DeB : BARDOU Pierre EnvoyC)B : mardi 11 mai 2010 15:40 CB : 'misc@openbsd.org' ObjetB : RE: Hardware for a PF box Hello, I'll try to answer every suggestion... I'm going to buy brand new HP servers, DL360 G5 or DL165 G7. So the choice for CPU is between AMD Opteron 24xx or Intel Xeon 55xx. I've read that a PIII would be sufficient : I have performance issues actually, running on a Xeon 2.8GHz (monocore, FSB 800, socket 604). I don't think they come from PF BTW, it should be logging/relayd/OpenVPN which makes the box lag. I'm actually on a test with dual xeon E5420 on GEMERIC.MP, it runs like a charm. But it's borrowed hardware, I have to give it back :) I'm very interested in separated log machine, I think I'll do that. Could you give me an estimation on how many Mbps I need on the log server ? I think I'll put this on a VM, we have an ESX cluster connected to a CX3-40 SAN which should give enough disk I/O... Installing SSD on the machines is way more expensive with HP hardware : 72 GB SAS 15Ktpm costs 260b,, 60 GB SSD costs 950b,. HP offers no way to install a compact flash as disk drive. Networks cards are Intel Gb, using em(4) driver. So, with all your considerations, here's my actual setup : * Xeon E5504 quad core @2Ghz (don't need AMD's 6 cores, and costs nearly the same prize than the only dual core remaining, E5502 @1.86GHz) * 3*1GB memory (Xeon are triple channel, so I need three DIMM for maximal memory bandwidth) * 2x72 Gb SAS drives on raid0 Does it sound correct to you ? Do you have any suggestion/modification ? Thank you very much for the help. -- Cordialement, Pierre BARDOU -Message d'origine- DeB : Aaron Mason [mailto:simplersolut...@gmail.com] EnvoyC)B : mardi 11 mai 2010 14:01 CB : Lars Nooden CcB : misc@openbsd.org ObjetB : Re: Hardware for a PF box On Tue, May 11, 2010 at 4:56 PM, Lars Nooden lars.cura...@gmail.com wrote: On Mon, 10 May 2010, Chris Smith wrote: What about logging in this case? Can PF logs be sent to another system running a syslog daemon? You answered your own question. ;) Look at the 'action' field explanation in the manual page for syslog.conf(5) About the diskless machine, many of the so-called diskless machines actually use flash or ssd instead of a spinning magnetic platter. The base installation of openbsd is still quite small. If you are only running PF, you will have a lot of space left over on a 1GB CF to make a logging partition. Flash can be very slow, so volitile caches can be stored in an mfs partition. /Lars OpenBSD will happily fit into about 160mb by installing only base and etc which provide plenty for a firewall. My 1.4GHz Toshiba laptop acting as a wireless-wired gateway runs OpenBSD 4.6 on a 512mb USB drive (which I'd like to replace with a CF disk on a 2.5 compatible adapter) with space to spare. Sure it doesn't do anywhere near as many packets as you propose, but it handles a constantly-running seedbox and my gaming together without skipping a beat, which is more than I can ask for. -- Aaron Mason - Programmer, open source addict I've taken my software vows - for beta or for worse
Re: Hardware for a PF box
On Tue, 11 May 2010, BARDOU Pierre wrote: ... I don't think they come from PF BTW, it should be logging/relayd/OpenVPN which makes the box lag. Verify before you flush money. Tools like iostat, vmstat and pftop might help show where the load is. Does the load you have from OpenVPN suggest the need for hardware random number generator? I'm very interested in separated log machine, I think I'll do that. Could you give me an estimation on how many Mbps I need on the log server ? It depends on what you have chosen to log, the level of detail you have chosen to log at and how much that service is actually used. Try set up the logging rules and use tcpdump or pftop to track the connection to the log server to see. Does it sound correct to you ? It could be overkill on the hardware. Do you have any suggestion/modification ? Several have already mentioned that a diskless set up would work. For PF,relayd,OpenVPN you do not need much of a hard drive. You boot from a 1GB CF and fit base in way less than 250MB of it. The rest could be used for short-term logging with copies sent to a log server. If you are running squid or another cache, then the RAID set up might be useful. Or it might not be. If you have a lot of RAM, then you can put the cache onto a ramdisk using mfs, if the size is right. /Lars
Re: Hardware for a PF box
On Tue, May 11, 2010 at 2:56 AM, Lars Nooden lars.cura...@gmail.com wrote: You answered your own question. ;) B Look at the 'action' field explanation in the manual page for syslog.conf(5) Maybe I'm missing something: I can send normal syslog data to a remote logging server without writing log files but not PF log entries - there is no entry in syslog.conf for pflog. There's a neat trick listed here: http://www.openbsd.org/faq/pf/logging.html but the PF logs first have to be written locally to a the pflog file. The concern is repeated writing to the SSD or CF which apparently tends to shorten their life. If PF could write directly to syslog this problem would be ameliorated. Chris
Re: Hardware for a PF box
2010/5/11, Chris Smith obsd_m...@chrissmith.org: Maybe I'm missing something: You might want something like this: # mkdir /var/log/rd ; chmod 700 /var/log/rd ; chown _pflogd:_pflogd /var/log/rd # echo 'pflogd_flags=-f /var/log/rd/pflog ' /etc/rc.conf.local # echo 'swap /var/log/rd/ mfs rw,nodev,nosuid,-s=67108864 0 0' /etc/fstab # mount /var/log/rd/ # pkill pflogd ; sleep 1 ; pflogd -f /var/log/rd/pflog Filesystems in RAM are extremely handy, but make sure the remote logging works, because umount makes the data disappear - see mfs(8). Does anyone know neater solution? -- Martin PelikC!n, Steadynet Jabber: sztor...@jabber.cz web: http://cap.potazmo.cz/
Re: Hardware for a PF box
On Tue, 11 May 2010, Chris Smith wrote: ...http://www.openbsd.org/faq/pf/logging.html but the PF logs first have to be written locally to a the pflog file. Or you can pipe to logger(1) directly or go via a FIFO /Lars
Re: Hardware for a PF box
On Tue, 11 May 2010 12:43:17 -0400, Chris Smith wrote: On Tue, May 11, 2010 at 2:56 AM, Lars Nooden lars.cura...@gmail.com wrote: You answered your own question. ;) B Look at the 'action' field explanation in the manual page for syslog.conf(5) Maybe I'm missing something: I can send normal syslog data to a remote logging server without writing log files but not PF log entries - there is no entry in syslog.conf for pflog. There's a neat trick listed here: http://www.openbsd.org/faq/pf/logging.html but the PF logs first have to be written locally to a the pflog file. The concern is repeated writing to the SSD or CF which apparently tends to shorten their life. I have tried to kill a CF for years. For more than a year it was running spamd with the most verbose logging possible and lots of other read/writes the system could live without. It is still going. I suggest that you use CF and when upgrade time comes around you program a new one and then have a halt-swap-reboot event and send me the one you don't think has much life left. I'll try wearing it out for you. My clients have lost more hard drives last year (3) than CFs in my lifetime (0) and I've been using them since they were exorbitantly priced. Some of that is good luck but they sure are not easily worn out. *** NOTE *** Please DO NOT CC me. I am subscribed to the list. Mail to the sender address that does not originate at the list server is tarpitted. The reply-to: address is provided for those who feel compelled to reply off list. Thankyou. Rod/ --- This life is not the real thing. It is not even in Beta. If it was, then OpenBSD would already have a man page for it.
Re: Hardware for a PF box
On May 11, 2010, at 17:18, Rod Whitworth glis...@witworx.com wrote: On Tue, 11 May 2010 12:43:17 -0400, Chris Smith I have tried to kill a CF for years. For more than a year it was running spamd with the most verbose logging possible and lots of other read/writes the system could live without. It is still going. I suggest that you use CF and when upgrade time comes around you program a new one and then have a halt-swap-reboot event and send me the one you don't think has much life left. I'll try wearing it out for you. My clients have lost more hard drives last year (3) than CFs in my lifetime (0) and I've been using them since they were exorbitantly priced. Some of that is good luck but they sure are not easily worn out. I'd have to agree there. I had one CF fail after three years of heavy DNS logging and I had a brand new card fail immediately as well. I've had many more times the hard drives fail. I would also suggest looking at the flashrd project. http://www.nmedia.net/flashrd/ I just recently started using it on some individual firewalls as well as several clusters. The whole point of the setup is to mount everything possible as read only and the rest to mfs. Bryan
Hardware for a PF box
Hello, I'm going to buy hardware to create 4 PF/relayd/openVPN boxes (2 active, 2 passive). I have an average of 500 new connections/s, 40k states and 40kpps in PF, 20 remote concurrent accesses on OpenVPN. What CPU would you recommend between Intel and AMD ? Since PF is mono threaded, I think more than 2 CPU cores are useless. Am I right ? For the same reason, I think that the CPU with the highest frequency will be the best ? Would it be useful to replace 15ktpm SAS HDDs by SSDs ? Thank you. -- Cordialement, Pierre BARDOU CSIM - Bureau 002 [cid:image001.jpg@01CAF064.EC6665D0] 12 rue Michel Labrousse BP93668 F-31036 Toulouse CEDEX 1 Til : 05 67 69 71 84 Fax : 05 34 61 51 00 Mail : bardo...@mipih.frmailto:bardo...@mipih.fr [demime 1.01d removed an attachment of type image/jpeg which had a name of image001.jpg]
Re: Hardware for a PF box
* BARDOU Pierre bardo...@mipih.fr [2010-05-10 17:27]: Hello, I'm going to buy hardware to create 4 PF/relayd/openVPN boxes (2 active, 2 passive). I have an average of 500 new connections/s, 40k states and 40kpps in PF, 20 remote concurrent accesses on OpenVPN. that's not much. a PIII @ 1GHz probably easily suffices. What CPU would you recommend between Intel and AMD ? doesn't matter all that much. Since PF is mono threaded, I think more than 2 CPU cores are useless. Am I right ? For the same reason, I think that the CPU with the highest frequency will be the best ? you want to run GENERIC, not GENERIC.MP, unless you also do lots of stuff in userland on the pf box, then MP might pay out. and since you'll be using one core only anyway you want as few and as fast cores you can. Would it be useful to replace 15ktpm SAS HDDs by SSDs ? yes. harddisks don't matter on pure firewalls. what is written to disks? logs. not all that much. read? after boot, not much. so using your expensive SAS-disks elsewhere is a good idea. a cheap 40..64G SSD will do fine. -- Henning Brauer, h...@bsws.de, henn...@openbsd.org BS Web Services, http://bsws.de Full-Service ISP - Secure Hosting, Mail and DNS Services Dedicated Servers, Rootservers, Application Hosting
Re: Hardware for a PF box
On 2010-05-10, BARDOU Pierre bardo...@mipih.fr wrote: I'm going to buy hardware to create 4 PF/relayd/openVPN boxes (2 active, 2 passive). I have an average of 500 new connections/s, 40k states and 40kpps in PF, 20 remote concurrent accesses on OpenVPN. What CPU would you recommend between Intel and AMD ? This question is silly, the CPU manufacturer doesn't matter. There is a lot more difference between the various CPUs made by a manufacturer (486, atom, p4, p3, core2, nehalem, ...) than the difference between AMD's fastest CPU and Intel's fastest CPU. Since PF is mono threaded, I think more than 2 CPU cores are useless. Am I right ? More than 1 core might be useful for OpenVPN (especially if you can run multiple openvpn processes, for example maybe listening to different ports and distributing between them using a rdr pool). Since any new CPU you get is likely to be multi-core I suggest you benchmark both GENERIC and GENERIC.MP... For the same reason, I think that the CPU with the highest frequency will be the best ? Generally yes (but some arch like P4, Atom are much slower for a given clock speed than P3-based arch, for example). Also consider memory bandwidth and cache size. Would it be useful to replace 15ktpm SAS HDDs by SSDs ? Depends what you're writing to disk. Presumably you won't be doing much in the way of random disk access, but might be doing some sequential writes for logging, so in that case SSDs are more likely to hurt than help. You will probably do better to propose some specific system and ask if anyone knows of problems with that machine.
Re: Hardware for a PF box
BARDOU Pierre bardo...@mipih.fr wrote on Mon, 10 May 2010 17:24:21 Subject: Hardware for a PF box I'm going to buy hardware to create 4 PF/relayd/openVPN boxes (2 active, 2 passive). I have an average of 500 new connections/s, 40k states and 40kpps in PF, 20 remote concurrent accesses on OpenVPN. What CPU would you recommend between Intel and AMD ? As other people have said, models/versions vary much more over each vendor than overall between vendors. For the same reason, I think that the CPU with the highest frequency will be the best ? As other people have said, memory access time, cache size, and integer arithmetic performance matter. For any specific CPU version/architecture, faster clocks are better up to the point where CPU utilization is under (for instance) 50%. Choice of memory speed is also important. There are non-intuitive interactions between CPU clocks and RAM clocks - sometimes lower clock speeds can mean fewer clock cycles. If you lower the clock speed 10% and reduce access time from 6 cycles to 5, you get 6% improvement. Choice of network interfaces can make as much impact as CPU choice. Many of the gigabit chips have better performance and better driver interaction than older 10/100 chips. I use the gigabit RE (Realtek) because they're very cheap and quite fast. I can't say which other gigabit ones are as good or better but as a rule the 10/100 interfaces are expensive in CPU time. Would it be useful to replace 15ktpm SAS HDDs by SSDs ? If there are local servers available, what about running the firewalls as diskless machines? Cheaper, cooler, and if you are running a backed up RAID on your servers, more reliable. I currently run a lightly loaded firewall on a 1.5 GHz VIA CPU with 3 interfaces - most packets traverse 2 bridged interfaces. Running 20 Mbit/sec the CPU loading is 25%. There are usually 500 states or so with a moderately complex (200+ lines) pf rule set and 20-50 connections/sec. The VIA is very slow but also runs quite cool low power. Total power with a local SATA laptop disk is 24W. I have run that system with a USB flash stick as the only local disk for more than a year with no problems. I hope this helps. geoff steckel omnivore technology
Re: Hardware for a PF box
On Mon, May 10, 2010 at 1:57 PM, Geoff g...@oat.com wrote: If there are local servers available, what about running the firewalls as diskless machines? What about logging in this case? Can PF logs be sent to another system running a syslog daemon? Chris
