Re: FreeBSD router - large scale
On 28 May 2010 07:38, Bruce Cran br...@cran.org.uk wrote: This is possibly the wrong place to be saying this, but isn't OpenBSD usually recommended for routers? I believe the version of pf, for example, is normally kept more up-to-date than than in FreeBSD. The major downside I know of is that it's not nearly as user-friendly; for example my recollection of its installer is that you have to input sector offsets manually in the partition editor! Bruce - sorry for taking so long to reply, this project has been slow-moving. Yes, you are correct, OpenBSD is typically used in this situation and, if the project were strictly for a routing component, it may indeed be a better choice. My concern was that if we decided to add any proxy capability then we would need much more RAM than OpenBSD could address (this will front at least 8k users). I have found the OpenBSD installer to be quite friendly but that's probably because it is pretty minimal and just sort of clicks with me. As long as you're dedicating the system to *BSD, I generally prefer the OpenBSD installer for its flow but have found no particular allegiance with either their installer or sysinstall. As long as I can have a running system within four or five minutes of powering on with the install CD, I don't really care. kmw -- A: Maybe because some people are too annoyed by top-posting. Q: Why do I not get an answer to my question(s)? A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: FreeBSD router - large scale
On 27 May 2010 12:12, Matthew Seaman m.sea...@infracaninophile.co.uk wrote: The hardest job I've had an OpenBSD firewall do is actually as a mid-level firewall between a DMZ full of web servers and a back-end database layer. The thing to watch out for is running out of states in PF. It's trivial to change that in the config, and given a machine with 1GB or so RAM dedicated to running PF, you can up the number of states by a factor of a hundred or more without problem. Also if you know all your connections are from directly attached networks and very low latency, you can be a lot more aggressive about dropping old states. Matthew - thanks for the information! For other reasons I'm limited to about 500k states...since our typical hardware build has at least 4GB of RAM, I'm not overly concerned about RAM exhaustion when routing. As I stated in another post the potential for something like a squid cache does exist, in which case I'll take all the RAM I can get my hands on (a 16GB+ build is not out of the question at that point). Preliminary testing has been favorable. My big concerns have mostly been related to state and packets per second. The first test environment was as follows: | one NIC, 4 routable addresses | | -- | FreeBSD 8 Router | -- | | one NIC with aliases for | 10.10.10.254 | 10.10.20.254 | 10.10.30.254 | 10.10.40.254 | |switch| Attached to the switch are four workstations/laptops: 10.10.10.1/255.255.255.0 10.10.20.1/255.255.255.0 10.10.30.1/255.255.255.0 10.10.40.1/255.255.255.0 All connections are gigabit. The idea is that in a production environment, we'll have multiple /22 networks coming in so I wanted to test having multiple network aliases. There will be a pool of public addresses for the outside interface(s), possibly as large as a class C but probably 20 - 30 addresses. By using sticky-address on a NAT rule, we can watch each RFC-1918 address get mapped to a different outside address via round-robin while enforcing that all connections from one inside host are consistently mapped to the same external address. Generating 10k active pings on each of the workstations/laptops, we were able to get an idea of how the machine would respond with 80k active states (two per connection, one in each direction). Adding in a couple of BitTorrent and HTTP .iso downloads only supported the conclusions we were beginning to form. Currently I'm testing it with multiple BitTorrent downloads and a very lively World of Warcraft installer. While nowhere near an indication of what we could expect in production it is showing us RAM usage, processor usage and state maintenance behaviour that gives us pretty good indications that we can go ahead and test in a larger environment. Like I said, we are otherwise limited to approximately 500k states (actually 250k connections) and only about half of that will be allotted for the population this project is targeting so testing with 100k states is actually pretty realistic at this point. We will wait, of course, to attempt a production deployment until after we have tested with a larger sample of the target population. Thanks to everyone for their comments and suggestions, both on and off list! kmw -- A: Maybe because some people are too annoyed by top-posting. Q: Why do I not get an answer to my question(s)? A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 'Serious' crypto? (was: FreeBSD router - large scale)
Hi Chuck, Thanks for the response. Or is it still worthwhile to consider hardware accelerators such as the ones guys like soekris [1] and others offer? Does anyone have an idea how much such an accelerator may help on older vs. on newer hardware? Something like a 1GHz P3 or equivalent can generally do the symmetric crypto about as fast as a decent PCI crypto card like the HiFN 795x could; bus limitations made faster CPUs better, although a newer PCIe crypto device ought to be more competitive. What matters more for some common use cases is that crypto H/W tends to do asymmetric crypto like RSA/DSA signing to negotiate a shared session key-- aka SSL session creation for SSL websites, secure email, SSH keys, etc much faster than normal CPUs could. I guess I try first without and see where I hit the ceiling. Then go to plan b. I was more thinking of many IPSEC connections but then there's also only so many slots and so many NICs in them. I'll try without and monitor that for a while and then see what happens. Would multiple engines work (and help) at all? From crypto(4), I would not guess so. One consequence would be that there may be certain limitations in using a separate accelerator once the platform comes with its own accelerator device? Sure, you can setup multiple engines, although this does better if you have separate services using each, since you do want to use an SSL session cache, but you don't want to pollute one for HTTPS with sessions from IMAPS and vice versa. Also, the config interface for Apache/IIS/whatever, or Dovecot/Cyrus/Exchange, etc might not let you specify more than one SSLEngine. On the other hand, it's not very much coding to adjust things to use multiple engines even within Apache or whatever-- I can recall some custom webserver modules from CryptoSwift for NSAPI / ISAPI / ASAPI which let you use multiple CryptoSwift boxes via ethernet network or local PCI slots, for example. Hmm... I was thinking more like round-robin the devices but I probably now too little about 'serious' crypto to see the side-effects. Anyways, I think the question is a bit academic at this time since I probably divide the servers anyways. Thanks again, All the best regards, Peter. -- GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT! Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: FreeBSD router - large scale
On 27.05.2010 17:00, Kevin Wilcox wrote: Hello everyone. We're in the very early stages of considering [Free|Open]BSD on commodity hardware to handle NAT *and* firewall duties for (what I consider to be) a sizable deployment. Overall bandwidth is low, only a gigabit connection, but we handle approximately fifteen thousand devices. DHCP and DNS would be passed through to other servers, this hardware would only be responsible for address translation and pf. I've done this on a very, very small scale (small/home office, small business) but I'm curious how many other folks are doing it on this scale, the hardware they are running on and any gotchas they may have faced. Does pf on FreeBSD take advantage of multiple cores/SMP? Is it preferable, as with OpenBSD, to go for a very stout processor without much consideration to cores? Would freebsd-net@ be a better place to ask this? I'm getting ready to start digging in to memory and other resources needed based on available documentation but real-world usage is much preferred to my academic assessment. Actually, I'd find an answer from the FreeBSD Networking gurus useful as well. My trusted Cisco 3640 is getting old (had it's ten-years-of-service birthday a little while ago), so I guess I must be prepared to replace it with something new. Preferrably something that can do proper NAT port mapping to the inside servers in an RFC1918-adressed DMZ, proper NAT mapping for the client net, incoming VPDN (virtual private dialin network, such as PPTP+MPE and L2TP+IPSEC tunelling), sane IDS in the border-gateway, GRE or IPinIP tunelling with crypto for remote-sites, etc If somebody has a good starting-point for documentation on these features, I'm more than willing to do a procject on it to create a mini-howto/handbook-section on setting up FreeBSD as your border gateway, provided I have someone to ask when the documentation is ... flaky. ;) It would be interesting to see what kind of performance modern hardware could get, compared to dedicated hardware a decade old. :) //Svein -- +---+--- /\ |Svein Skogen | sv...@d80.iso100.no \ / |Solberg Østli 9| PGP Key: 0xE5E76831 X|2020 Skedsmokorset | sv...@jernhuset.no / \ |Norway | PGP Key: 0xCE96CE13 | | sv...@stillbilde.net ascii | | PGP Key: 0x58CD33B6 ribbon |System Admin | svein-listm...@stillbilde.net Campaign|stillbilde.net | PGP Key: 0x22D494A4 +---+--- |msn messenger: | Mobile Phone: +47 907 03 575 |sv...@jernhuset.no | RIPE handle:SS16503-RIPE +---+--- If you really are in a hurry, mail me at svein-mob...@stillbilde.net This mailbox goes directly to my cellphone and is checked even when I'm not in front of my computer. Picture Gallery: https://gallery.stillbilde.net/v/svein/ signature.asc Description: OpenPGP digital signature
Re: FreeBSD router - large scale
On 28/05/2010 12:31, Svein Skogen (Listmail Account) wrote: On 27.05.2010 17:00, Kevin Wilcox wrote: Hello everyone. We're in the very early stages of considering [Free|Open]BSD on commodity hardware to handle NAT *and* firewall duties for (what I consider to be) a sizable deployment. Overall bandwidth is low, only a gigabit connection, but we handle approximately fifteen thousand devices. DHCP and DNS would be passed through to other servers, this hardware would only be responsible for address translation and pf. I've done this on a very, very small scale (small/home office, small business) but I'm curious how many other folks are doing it on this scale, the hardware they are running on and any gotchas they may have faced. Does pf on FreeBSD take advantage of multiple cores/SMP? Is it preferable, as with OpenBSD, to go for a very stout processor without much consideration to cores? Would freebsd-net@ be a better place to ask this? I'm getting ready to start digging in to memory and other resources needed based on available documentation but real-world usage is much preferred to my academic assessment. Actually, I'd find an answer from the FreeBSD Networking gurus useful as well. My trusted Cisco 3640 is getting old (had it's ten-years-of-service birthday a little while ago), so I guess I must be prepared to replace it with something new. Preferrably something that can do proper NAT port mapping to the inside servers in an RFC1918-adressed DMZ, proper NAT mapping for the client net, incoming VPDN (virtual private dialin network, such as PPTP+MPE and L2TP+IPSEC tunelling), sane IDS in the border-gateway, GRE or IPinIP tunelling with crypto for remote-sites, etc If somebody has a good starting-point for documentation on these features, I'm more than willing to do a procject on it to create a mini-howto/handbook-section on setting up FreeBSD as your border gateway, provided I have someone to ask when the documentation is ... flaky. ;) This is possibly the wrong place to be saying this, but isn't OpenBSD usually recommended for routers? I believe the version of pf, for example, is normally kept more up-to-date than than in FreeBSD. The major downside I know of is that it's not nearly as user-friendly; for example my recollection of its installer is that you have to input sector offsets manually in the partition editor! -- Bruce Cran ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: FreeBSD router - large scale
On 28.05.2010 13:38, Bruce Cran wrote: *snip!* This is possibly the wrong place to be saying this, but isn't OpenBSD usually recommended for routers? I believe the version of pf, for example, is normally kept more up-to-date than than in FreeBSD. The major downside I know of is that it's not nearly as user-friendly; for example my recollection of its installer is that you have to input sector offsets manually in the partition editor! My main reasoning for wanting this done on FreeBSD i don't introduce yet another OS into the equation, there is sufficient confusion as there is ;) //Svein -- +---+--- /\ |Svein Skogen | sv...@d80.iso100.no \ / |Solberg Østli 9| PGP Key: 0xE5E76831 X|2020 Skedsmokorset | sv...@jernhuset.no / \ |Norway | PGP Key: 0xCE96CE13 | | sv...@stillbilde.net ascii | | PGP Key: 0x58CD33B6 ribbon |System Admin | svein-listm...@stillbilde.net Campaign|stillbilde.net | PGP Key: 0x22D494A4 +---+--- |msn messenger: | Mobile Phone: +47 907 03 575 |sv...@jernhuset.no | RIPE handle:SS16503-RIPE +---+--- If you really are in a hurry, mail me at svein-mob...@stillbilde.net This mailbox goes directly to my cellphone and is checked even when I'm not in front of my computer. Picture Gallery: https://gallery.stillbilde.net/v/svein/ signature.asc Description: OpenPGP digital signature
Re: FreeBSD router - large scale
Svein Skogen (Listmail Account) wrote: Actually, I'd find an answer from the FreeBSD Networking gurus useful as well. My trusted Cisco 3640 is getting old (had it's ten-years-of-service birthday a little while ago), so I guess I must be prepared to replace it with something new. Preferrably something that can do proper NAT port mapping to the inside servers in an RFC1918-adressed DMZ, proper NAT mapping for the client net, incoming VPDN (virtual private dialin network, such as PPTP+MPE and L2TP+IPSEC tunelling), sane IDS in the border-gateway, GRE or IPinIP tunelling with crypto for remote-sites, etc If somebody has a good starting-point for documentation on these features, I'm more than willing to do a procject on it to create a mini-howto/handbook-section on setting up FreeBSD as your border gateway, provided I have someone to ask when the documentation is ... flaky. ;) Although I feel that you'll have to write book to cover all the things mentioned above, I'll try to reply to your question... These is just pointers... Several forms of NAT are supported with the following tools: ipfw pf ipf ng_nat I doubt there is some form of NAT you will miss. the net/mpd5 port can do PPTP, the MPPE part is blurry to me. L2TP is supported for LNS/LAC scenarios. I don't know if you can/how difficult is to combine IPSEC with L2TP. The most famous open source IDS is snort, you'll find it in the ports. For GRE and IPIP read gre and gif manual pages. Again, IPSEC is not integrated to these, yet there is IKE support via ipsec-tools port. You'll have to check for yourself the documentation. Though I can say that all the FreeBSD stuff mentioned above are well documented as usual and there is always this list if you have questions. Good luck replacing the aging Cisco... Nikos ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
FreeBSD router - large scale
Hello everyone. We're in the very early stages of considering [Free|Open]BSD on commodity hardware to handle NAT *and* firewall duties for (what I consider to be) a sizable deployment. Overall bandwidth is low, only a gigabit connection, but we handle approximately fifteen thousand devices. DHCP and DNS would be passed through to other servers, this hardware would only be responsible for address translation and pf. I've done this on a very, very small scale (small/home office, small business) but I'm curious how many other folks are doing it on this scale, the hardware they are running on and any gotchas they may have faced. Does pf on FreeBSD take advantage of multiple cores/SMP? Is it preferable, as with OpenBSD, to go for a very stout processor without much consideration to cores? Would freebsd-net@ be a better place to ask this? I'm getting ready to start digging in to memory and other resources needed based on available documentation but real-world usage is much preferred to my academic assessment. Thanks! kmw -- A: Maybe because some people are too annoyed by top-posting. Q: Why do I not get an answer to my question(s)? A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: FreeBSD router - large scale
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 27/05/2010 16:00:12, Kevin Wilcox wrote: Hello everyone. We're in the very early stages of considering [Free|Open]BSD on commodity hardware to handle NAT *and* firewall duties for (what I consider to be) a sizable deployment. Overall bandwidth is low, only a gigabit connection, but we handle approximately fifteen thousand devices. DHCP and DNS would be passed through to other servers, this hardware would only be responsible for address translation and pf. I've done this on a very, very small scale (small/home office, small business) but I'm curious how many other folks are doing it on this scale, the hardware they are running on and any gotchas they may have faced. Does pf on FreeBSD take advantage of multiple cores/SMP? Is it preferable, as with OpenBSD, to go for a very stout processor without much consideration to cores? Would freebsd-net@ be a better place to ask this? I'm getting ready to start digging in to memory and other resources needed based on available documentation but real-world usage is much preferred to my academic assessment. I've used OpenBSD/pf + carp for several sites; also + relayd for a reasonably high traffic website, plus various setups using IPSec tunnels. All very successfully. On a reasonably fast modern processor, PF can run pretty much at GB wirespeed for straight packet forwarding or NAT. Doing serious crypto slows things up somewhat. The hardest job I've had an OpenBSD firewall do is actually as a mid-level firewall between a DMZ full of web servers and a back-end database layer. The thing to watch out for is running out of states in PF. It's trivial to change that in the config, and given a machine with 1GB or so RAM dedicated to running PF, you can up the number of states by a factor of a hundred or more without problem. Also if you know all your connections are from directly attached networks and very low latency, you can be a lot more aggressive about dropping old states. PF is basically single-threaded -- even on FreeBSD, multiple cores won't help you a great deal. (Unless you've got anything else running on the firewall, when several cores is really useful, of course.) On the other hand, PF is not hugely CPU intensive. Better to spend your money on the best NICs you can afford. There are some useful enhancements in OpenBSD-4.7/pf which haven't made it into FreeBSD yet -- FreeBSD pf is basically equivalent to about OpenBSD-4.1 I think. FreeBSD is compatible with more varieties of amd64/i386 based hardware, and it does threading and multi-cpu very much better. Cheers, Matthew - -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate JID: matt...@infracaninophile.co.uk Kent, CT11 9PW -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.14 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkv+mesACgkQ8Mjk52CukIyB4gCff56iOhw7jRwmH4jzhaRmZPiK COwAoINJQZ8YRk3s4plAuoru4CIdQr/h =xyZm -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
'Serious' crypto? (was: FreeBSD router - large scale)
Hi, NAT. Doing serious crypto slows things up somewhat. I've been pondering this since a while but thought that crypto engines on modern hardware would make 'extra' hardware accelerators obsolete? Or is it still worthwhile to consider hardware accelerators such as the ones guys like soekris [1] and others offer? Does anyone have an idea how much such an accelerator may help on older vs. on newer hardware? Would multiple engines work (and help) at all? From crypto(4), I would not guess so. One consequence would be that there may be certain limitations in using a separate accelerator once the platform comes with its own accelerator device? Thanks, Peter. --- [1] http://www.soekris.com/vpn1401.htm -- GRATIS für alle GMX-Mitglieder: Die maxdome Movie-FLAT! Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01 ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 'Serious' crypto? (was: FreeBSD router - large scale)
On May 27, 2010, at 1:49 PM, Peter Cornelius wrote: Hi, NAT. Doing serious crypto slows things up somewhat. I've been pondering this since a while but thought that crypto engines on modern hardware would make 'extra' hardware accelerators obsolete? It depends upon usage. Or is it still worthwhile to consider hardware accelerators such as the ones guys like soekris [1] and others offer? Does anyone have an idea how much such an accelerator may help on older vs. on newer hardware? Something like a 1GHz P3 or equivalent can generally do the symmetric crypto about as fast as a decent PCI crypto card like the HiFN 795x could; bus limitations made faster CPUs better, although a newer PCIe crypto device ought to be more competitive. What matters more for some common use cases is that crypto H/W tends to do asymmetric crypto like RSA/DSA signing to negotiate a shared session key-- aka SSL session creation for SSL websites, secure email, SSH keys, etc much faster than normal CPUs could. Would multiple engines work (and help) at all? From crypto(4), I would not guess so. One consequence would be that there may be certain limitations in using a separate accelerator once the platform comes with its own accelerator device? Sure, you can setup multiple engines, although this does better if you have separate services using each, since you do want to use an SSL session cache, but you don't want to pollute one for HTTPS with sessions from IMAPS and vice versa. Also, the config interface for Apache/IIS/whatever, or Dovecot/Cyrus/Exchange, etc might not let you specify more than one SSLEngine. On the other hand, it's not very much coding to adjust things to use multiple engines even within Apache or whatever-- I can recall some custom webserver modules from CryptoSwift for NSAPI / ISAPI / ASAPI which let you use multiple CryptoSwift boxes via ethernet network or local PCI slots, for example. Regards, -- -Chuck ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
