Re: CPU criteria for OpenBSD firewall
* ML mail [2015-02-19 09:07]: > I might also experiment if I should use bsd.mp or the standard non > SMP bsd. you'll want amd64, not i386. MP vs SP should make little difference, I use the MP kernels these days. -- Henning Brauer, h...@bsws.de, henn...@openbsd.org BS Web Services GmbH, http://bsws.de, Full-Service ISP Secure Hosting, Mail and DNS. Virtual & Dedicated Servers, Root to Fully Managed Henning Brauer Consulting, http://henningbrauer.com/
Re: CPU criteria for OpenBSD firewall
* ML mail [2015-02-18 23:32]: > Stupid question but if you would have to choose between two different > Intel CPUs for an OpenBSD firewall using 4 to 6 Intel NICs with all /24 > networks behind and around 50-60 Mbit/s average traffic would you > rather choose the CPU with higher Frequency and less cores or for a CPU > with lower frequency but more cores? The #1 criteria is memory bandwidth and even more so latency. Thus, more cache helps. Then it's the speed of a single core. Our kernel is mostly biglocked still, so almost everything is going to run on CPU (core) 0. There is ongoing work to unlock at least parts of the network stack to profit from multiple cores, but that doesn't help you right now, and even then I'd be super surprised if the faster cores wouldn't win against more cores, pushing packets isn't one of the workloads that is well suited for MP, due to quite a lot of shared data structures (think routing table, pf state table, ...). > For example: > - E5-2630Lv3, 20M Cache, 1.80 GHz, 8 cores: > - E5-2637v3, 15M Cache, 3.50 GHz, 4 cores: the latter. -- Henning Brauer, h...@bsws.de, henn...@openbsd.org BS Web Services GmbH, http://bsws.de, Full-Service ISP Secure Hosting, Mail and DNS. Virtual & Dedicated Servers, Root to Fully Managed Henning Brauer Consulting, http://henningbrauer.com/
Re: CPU criteria for OpenBSD firewall
On 2015-02-18 07:07 PM, System Administrator wrote: Actually, at this time and the near future, "passing traffic" (i.e. the kernel network stack) happens entirely on CPU0. The network gurus *are* working on making the network layer multiprocessor capable, but my impression from watching the tech@ list is that this goal is still some ways off. At the present time, only userland applications can and do make use of the additional CPU cores. According to the very first reports to come out of the s2k15 hackathon, mpi@ & dlg@ *just* did a bunch of the critical work to get that started. http://undeadly.org/cgi?action=article&sid=20150218085759&mode=expanded&count=0 If I've got my timeline right, we're already post-5.7-freeze, so I assume 5.8 is probably when us mere users will see a partially-SMP network stack. -- -Adam Thompson athom...@athompso.net +1 (204) 291-7950 - cell +1 (204) 489-6515 - fax
Re: CPU criteria for OpenBSD firewall
On 2015-02-19 06:26 AM, Stuart Henderson wrote: Modern Atoms (avoton, rangeley) will do just fine for this amount of "normal" traffic. If the ruleset is super-complicated or if there are very high PPS counts (which is by *far* more important than absolute bandwidth) _or_ if there is a reasonable chance of that happening during the expected life of the machine, then you might need faster. Somewhat OT, but I've heard the pfSense developers over at Netgate have done amazing things with Rangeley CPUs (i.e. actually useful w/10Gbps NICs). In general, what I'm hearing in the embedded world is that QuickAssist is a really, really nice feature *if* you can figure out how to use it properly. Reminds me of the buzz surrounding the Cell processor when it came out. -- -Adam Thompson athom...@athompso.net
Re: CPU criteria for OpenBSD firewall
On Thu, Feb 19, 2015 at 10:33 AM, Dmitrij D. Czarkoff wrote: > Stuart Henderson said: >> Half of that page is obsolete. > [...] >> Various things are recommended without explaining that they are a >> trade-off or can cause problems. There are >> It includes "tweaks" which may improve performance of an end host (but >> have trade-offs) in a page mostly talking about routers, other >> "tweaks" which are nothing to do with networking and in some cases >> dangerous. > > It would be nice if someone with expertise could write a detailed > explanation of the issues with that article... > It shouldn't be any developer's responsibility to refute or confirm unofficial guidance found anywhere else. Unnecessary configuration tweaks frequently confuse and discourage those who would otherwise help you troubleshoot legitimate issues, either because they have caused the problem in the first place or have introduced red herrings. The defaults are suitable as defaults. If and when a default no longer makes sense, it will likely be changed. If a default is not appropriate for YOUR personal needs, it is YOUR responsibility to figure out what needs fixing or changing. Refer to the official FAQs, manuals, source, mailing list archives, etc to figure out what to do -- prior to asking questions. --david
Re: CPU criteria for OpenBSD firewall
2015-02-19 16:33 GMT+01:00 Dmitrij D. Czarkoff : > It would be nice if someone with expertise could write a detailed > explanation of the issues with that article... Thou art not supposed to twiddle with your config.
Re: CPU criteria for OpenBSD firewall
Stuart Henderson said: > Half of that page is obsolete. [...] > Various things are recommended without explaining that they are a > trade-off or can cause problems. There are > It includes "tweaks" which may improve performance of an end host (but > have trade-offs) in a page mostly talking about routers, other > "tweaks" which are nothing to do with networking and in some cases > dangerous. It would be nice if someone with expertise could write a detailed explanation of the issues with that article... -- Dmitrij D. Czarkoff
Re: CPU criteria for OpenBSD firewall
Markus Kolb wrote: > Am 2015-02-19 10:51, schrieb Peter Hessler: > > > > :choose the CPU with higher Frequency and less cores or for a CPU with > > :lower frequency but more cores? > > > > Higher frequency. Period. > > > > Right now, network and PF processing is limited to CPU0. You want that > > as fast as possible. > > > > Additionally, you want as fast memory transfers (from CPU to RAM) as > > possible. > > > > That will give you the most performance. > > Is it as simple as "Higher frequency."? > Shouldn't there be a view on the instruction sets mostly used in network > traffic handling and cycle usage of these instructions? > Or is this equivalent at the up-to-date processors? > If not, it is possible that lower frequency is faster. Every iX-3xxx is about the same in terms of IPC. Same for iX-4xxx, etc. But some models do have AES-NI and some don't. This will clearly matter if you're also doing IPsec or so. Also, last I checked for desktop chips, i3 was 2 core hyperthreaded (4 threads) while i5 was quad core not-hyper (still 4 threads). OpenBSD still isn't really aware of hyperthreaded, so you'll get better, more consistent performance from the i5. The i7 then goes to four cores, 8 threads, which is less than ideal for openbsd. There's also some variation in the availability of various VT extensions. Last two points probably don't matter much for a pure firewall passing packets, but the few hundred mhz gained may not be a worthwhile trade for flexibilty if you ever intend to repurpose the box.
Re: CPU criteria for OpenBSD firewall
On 2015 Feb 19 (Thu) at 10:58:21 +0100 (+0100), Alexander Salmin wrote: :Good luck, when you have time I also recommend that you read this. :https:// calomel.org [snip dangerous url] : don't follow *any* recommendation from that site -- All I want is a warm bed and a kind word and unlimited power. -- Ashleigh Brilliant
Re: CPU criteria for OpenBSD firewall
On 2015-02-19, Alexander Salmin wrote: > Good luck, when you have time I also recommend that you read this. [snip link to calomel] Half of that page is obsolete. It keeps jumping between megabytes/sec and megabits/sec which is confusing. Various things are recommended without explaining that they are a trade-off or can cause problems. There are completely pointless things like talking about TTLs. It includes "tweaks" which may improve performance of an end host (but have trade-offs) in a page mostly talking about routers, other "tweaks" which are nothing to do with networking and in some cases dangerous. Still, at least re-reading this makes me see where a user in another recent mailing list thread got the idea to mess around with mssdflt.
Re: CPU criteria for OpenBSD firewall
On 2015-02-19, Nick Holland wrote: > On 02/18/15 17:30, ML mail wrote: >> Hi, >> >> Stupid question but if you would have to choose between two different >> Intel CPUs for an OpenBSD firewall using 4 to 6 Intel NICs with all >> /24 networks behind and around 50-60 Mbit/s average traffic would you >> rather choose the CPU with higher Frequency and less cores or for a >> CPU with lower frequency but more cores? > ... > actually, I'd ask more useful questions. > Realistically, most modern "fast" CPUs (let's leave out "special cases" > like the Intel Atom, though even that might do it for you) will do the > job just fine. Modern Atoms (avoton, rangeley) will do just fine for this amount of "normal" traffic. If the ruleset is super-complicated or if there are very high PPS counts (which is by *far* more important than absolute bandwidth) _or_ if there is a reasonable chance of that happening during the expected life of the machine, then you might need faster. Just choosing between the two suggested CPUs and taking only the basic description of workload into account I'd probably choose the lower-power one. >> Or asked differently, which are the importants criteria to look at >> first for a CPU intended to be used in an OpenBSD firewall? > > Discussing the merits of a CPU that's 95% idle vs. one that's 90% idle > really misses a few points. If I were looking for a box, I'd look at > more important issues: > (in no particular order. And your criteria WILL differ from mine) > * How fast a machine boots. > * Availability of repair and upgrade parts > * Low cost, so I can get a second machine and CARP 'em together. > * General usability of the system and support by OpenBSD > * Good bus structure for application > * Well-supported NICs > * Power consumption. > * Quiet > * Simple Availability of out-of-band console access. Working IPMI SOL with a dedicated NIC as a minimum, but preferably a standard serial port. Non-idiotic rack mounting hardware.
Re: CPU criteria for OpenBSD firewall
Am 2015-02-19 10:51, schrieb Peter Hessler: :choose the CPU with higher Frequency and less cores or for a CPU with :lower frequency but more cores? Higher frequency. Period. Right now, network and PF processing is limited to CPU0. You want that as fast as possible. Additionally, you want as fast memory transfers (from CPU to RAM) as possible. That will give you the most performance. Is it as simple as "Higher frequency."? Shouldn't there be a view on the instruction sets mostly used in network traffic handling and cycle usage of these instructions? Or is this equivalent at the up-to-date processors? If not, it is possible that lower frequency is faster.
Re: CPU criteria for OpenBSD firewall
2015-02-19 10:58 GMT+01:00 Alexander Salmin : > Good luck, when you have time I also recommend that you read this. > https://calomel.org/network_performance.html The consensus here seems to be to warn against any tweaks etc. by calomel.
Re: CPU criteria for OpenBSD firewall
On 19.02.2015 10:51, Peter Hessler wrote: :choose the CPU with higher Frequency and less cores or for a CPU with :lower frequency but more cores? Higher frequency. Period. That is why I chose an i3-4000 as they go up to 3,8Ghz. i5-4000 only go up to 3,5Ghz. If you want to go even faster you will need to buy i7-4000 with 4Ghz. But this one is 3 times the price of the i3. If you put the i3 on a board with C22x chipset, you can also use ECC RAM with the i3. my 2ct Lars
Re: CPU criteria for OpenBSD firewall
Good luck, when you have time I also recommend that you read this. https://calomel.org/network_performance.html On 2015-02-19 08:05:54, ML mail wrote: > Thanks to all of you for this interesting discussion. My OpenBSD firewall > will only be doing PF as I totally agree that a firewall should have the > least userland application running as possible of course if your budget > permits it. So far I have around 340 rules (as the number of lines in the > output of a "pf -sr") and a state table of around 12-20k entries depending > the time of the day. As per your recommendations I will go with a higher CPU > frequency and less cores as packet filtering still only takes place on one > single core. I might also experiment if I should use bsd.mp or the standard > non SMP bsd. > > I also agree with Nick that CPU is of course not the only criteria but the > rest I have luckily already sorted out :) For example by using nice and > modern Intel 10 Gbit/s NICs, CompactFlash industrial grade flash storage, > redundant setup with 2 firewalls and CARP, etc. OpenBSD does a great job > here, I don't even want to imagine the price of such a setup with C***o > hardware. > > Cheers
Re: CPU criteria for OpenBSD firewall
On 2015 Feb 18 (Wed) at 22:30:31 + (+), ML mail wrote: :Hi, : :Stupid question but if you would have to choose between two different :Intel CPUs for an OpenBSD firewall using 4 to 6 Intel NICs with all /24 :networks behind and around 50-60 Mbit/s average traffic would you rather Either one is perfectly fine for your traffic load. You only have to care once you get multi-gbit/s traffic (or over half a million pps) on your system. However. :choose the CPU with higher Frequency and less cores or for a CPU with :lower frequency but more cores? Higher frequency. Period. Right now, network and PF processing is limited to CPU0. You want that as fast as possible. Additionally, you want as fast memory transfers (from CPU to RAM) as possible. That will give you the most performance. -- I call them as I see them. If I can't see them, I make them up. -- Biff Barf
Re: CPU criteria for OpenBSD firewall
Thanks to all of you for this interesting discussion. My OpenBSD firewall will only be doing PF as I totally agree that a firewall should have the least userland application running as possible of course if your budget permits it. So far I have around 340 rules (as the number of lines in the output of a "pf -sr") and a state table of around 12-20k entries depending the time of the day. As per your recommendations I will go with a higher CPU frequency and less cores as packet filtering still only takes place on one single core. I might also experiment if I should use bsd.mp or the standard non SMP bsd. I also agree with Nick that CPU is of course not the only criteria but the rest I have luckily already sorted out :) For example by using nice and modern Intel 10 Gbit/s NICs, CompactFlash industrial grade flash storage, redundant setup with 2 firewalls and CARP, etc. OpenBSD does a great job here, I don't even want to imagine the price of such a setup with C***o hardware. Cheers
Re: CPU criteria for OpenBSD firewall
On 02/18/15 17:30, ML mail wrote: Hi, Stupid question but if you would have to choose between two different Intel CPUs for an OpenBSD firewall using 4 to 6 Intel NICs with all /24 networks behind and around 50-60 Mbit/s average traffic would you rather choose the CPU with higher Frequency and less cores or for a CPU with lower frequency but more cores? ... actually, I'd ask more useful questions. Realistically, most modern "fast" CPUs (let's leave out "special cases" like the Intel Atom, though even that might do it for you) will do the job just fine. Or asked differently, which are the importants criteria to look at first for a CPU intended to be used in an OpenBSD firewall? Discussing the merits of a CPU that's 95% idle vs. one that's 90% idle really misses a few points. If I were looking for a box, I'd look at more important issues: (in no particular order. And your criteria WILL differ from mine) * How fast a machine boots. * Availability of repair and upgrade parts * Low cost, so I can get a second machine and CARP 'em together. * General usability of the system and support by OpenBSD * Good bus structure for application * Well-supported NICs * Power consumption. * Quiet * Simple The last one probably deserves comment (and should probably be ranked at the top of my list): Simple wins out in reliability over complex. For a firewall, I'd rather have two non-RAIDed systems in a CARP setup over one machine with multiple power supplies, RAID controllers and other fluff that really does nothing for you IN THIS APPLICATION. If something takes your firewall down, you will lose more packets waiting for a "super server" to do its Power-on Self-test than you will because your processor is not the latest and greatest or theoretical "best". I'd rather a couple few-year-old desktops that can reboot in 60 seconds over a super-server that spends two minutes showing you the wonderful RAID controller you don't care about. Yes, OpenBSD's filtering and packet moving system uses only one processor, so if you are pushing the limits, you will want more power-per-core over more cores, but you probably won't be pushing the limits. You will have N-1 cores all but completely idle, and one that is not very busy, On board cache could matter too, but again, all it will do in your case is reduce the load on the CPU even more, but it won't pump any more packets. Nick.
Re: CPU criteria for OpenBSD firewall
On 19-02-2015 01:12, Eric Furman wrote: > A firewall should be a firewall. Period. > It's your first line of defense against attack. > Each and every additional thing you run on it just > makes it that much more vulnerable to attack. Of course it does. But since not all of us have the budget for this kind of setup, I believe this trade-off is an acceptable one, if you understand the risks. Also, there are some things you can't do if you run the services on a separate machine such as divert(4). Cheers, Giancarlo Razzolini
Re: CPU criteria for OpenBSD firewall
On Wed, Feb 18, 2015, at 07:54 PM, Giancarlo Razzolini wrote: > On 18-02-2015 20:30, ML mail wrote: > > Stupid question but if you would have to choose between two different Intel > > CPUs for an OpenBSD firewall using 4 to 6 Intel NICs with all /24 networks > > behind and around 50-60 Mbit/s average traffic would you rather choose the > > CPU with higher Frequency and less cores or for a CPU with lower frequency > > but more cores? > This question isn't stupid at all. And the answer is probably entirely > based on your setup. I do have a similar system, but with less average > traffic, 10MB/s, and one 6-port intel card. In my setup, having the > lower frequency, more cores is better, because my firewall isn't used > just for PF. If you're gonna use you OpenBSD firewall for other > processes such as, proxy, dns server, web server, dhcp server, it won't > hurt to have more cores. A firewall should be a firewall. Period. It's your first line of defense against attack. Each and every additional thing you run on it just makes it that much more vulnerable to attack.
Re: CPU criteria for OpenBSD firewall
On 18 Feb 2015 at 15:18, Gene wrote: > To expand on Alexander's point, look at the FAQ: > > http://www.openbsd.org/faq/pf/perf.html > > If you aren't doing a lot of filtering, just passing traffic over > multiple interfaces, more cores might be beneficial. > > -Eugene Actually, at this time and the near future, "passing traffic" (i.e. the kernel network stack) happens entirely on CPU0. The network gurus *are* working on making the network layer multiprocessor capable, but my impression from watching the tech@ list is that this goal is still some ways off. At the present time, only userland applications can and do make use of the additional CPU cores. So to quote the old-timers on this list -- only the OP can determine the characterstics of the specific workload and firewall configuration. But unless that firewall includes many CPU-intensive proxies, it will most likely perform best with fewer yet faster cores. -Jacob. > > On Wed, Feb 18, 2015 at 2:50 PM, Alexander Salmin > wrote: > > > I might start a flame now but the higher freq and less core model is > > the "better choice" unless your firewall will do other things than > > packetfiltering and routing. > > > > On 2015-02-18 22:30:31, ML mail wrote: > > > Hi, > > > > > > Stupid question but if you would have to choose between two > > > different > > Intel CPUs for an OpenBSD firewall using 4 to 6 Intel NICs with all > > /24 networks behind and around 50-60 Mbit/s average traffic would you > > rather choose the CPU with higher Frequency and less cores or for a > > CPU with lower frequency but more cores? > > > > > > For example: > > > > > > - E5-2630Lv3, 20M Cache, 1.80 GHz, 8 cores: > > http://ark.intel.com/products/83357/Intel-Xeon-Processor-E5-2630L-v3-2 > > 0M-Cache-1_80-GHz > > > - E5-2637v3, 15M Cache, 3.50 GHz, 4 cores: > > > > > http://ark.intel.com/products/83358/Intel-Xeon-Processor-E5-2637-v3-15 > > M-Cache-3_50-GHz > > > > > > Or asked differently, which are the importants criteria to look at > > > first > > for a CPU intended to be used in an OpenBSD firewall? > > > > > > Regards > > > ML
Re: CPU criteria for OpenBSD firewall
On 18-02-2015 20:30, ML mail wrote: > Stupid question but if you would have to choose between two different Intel > CPUs for an OpenBSD firewall using 4 to 6 Intel NICs with all /24 networks > behind and around 50-60 Mbit/s average traffic would you rather choose the > CPU with higher Frequency and less cores or for a CPU with lower frequency > but more cores? This question isn't stupid at all. And the answer is probably entirely based on your setup. I do have a similar system, but with less average traffic, 10MB/s, and one 6-port intel card. In my setup, having the lower frequency, more cores is better, because my firewall isn't used just for PF. If you're gonna use you OpenBSD firewall for other processes such as, proxy, dns server, web server, dhcp server, it won't hurt to have more cores. Cheers, Giancarlo Razzolini
Re: CPU criteria for OpenBSD firewall
To expand on Alexander's point, look at the FAQ: http://www.openbsd.org/faq/pf/perf.html If you aren't doing a lot of filtering, just passing traffic over multiple interfaces, more cores might be beneficial. -Eugene On Wed, Feb 18, 2015 at 2:50 PM, Alexander Salmin wrote: > I might start a flame now but the higher freq and less core model is the > "better choice" unless your firewall will do other things than > packetfiltering and routing. > > On 2015-02-18 22:30:31, ML mail wrote: > > Hi, > > > > Stupid question but if you would have to choose between two different > Intel CPUs for an OpenBSD firewall using 4 to 6 Intel NICs with all /24 > networks behind and around 50-60 Mbit/s average traffic would you rather > choose the CPU with higher Frequency and less cores or for a CPU with lower > frequency but more cores? > > > > For example: > > > > - E5-2630Lv3, 20M Cache, 1.80 GHz, 8 cores: > http://ark.intel.com/products/83357/Intel-Xeon-Processor-E5-2630L-v3-20M-Cache-1_80-GHz > > - E5-2637v3, 15M Cache, 3.50 GHz, 4 cores: > > > http://ark.intel.com/products/83358/Intel-Xeon-Processor-E5-2637-v3-15M-Cache-3_50-GHz > > > > Or asked differently, which are the importants criteria to look at first > for a CPU intended to be used in an OpenBSD firewall? > > > > Regards > > ML
Re: CPU criteria for OpenBSD firewall
I might start a flame now but the higher freq and less core model is the "better choice" unless your firewall will do other things than packetfiltering and routing. On 2015-02-18 22:30:31, ML mail wrote: > Hi, > > Stupid question but if you would have to choose between two different Intel > CPUs for an OpenBSD firewall using 4 to 6 Intel NICs with all /24 networks > behind and around 50-60 Mbit/s average traffic would you rather choose the > CPU with higher Frequency and less cores or for a CPU with lower frequency > but more cores? > > For example: > > - E5-2630Lv3, 20M Cache, 1.80 GHz, 8 cores: > http://ark.intel.com/products/83357/Intel-Xeon-Processor-E5-2630L-v3-20M-Cache-1_80-GHz > - E5-2637v3, 15M Cache, 3.50 GHz, 4 cores: > http://ark.intel.com/products/83358/Intel-Xeon-Processor-E5-2637-v3-15M-Cache-3_50-GHz > > Or asked differently, which are the importants criteria to look at first for > a CPU intended to be used in an OpenBSD firewall? > > Regards > ML
CPU criteria for OpenBSD firewall
Hi, Stupid question but if you would have to choose between two different Intel CPUs for an OpenBSD firewall using 4 to 6 Intel NICs with all /24 networks behind and around 50-60 Mbit/s average traffic would you rather choose the CPU with higher Frequency and less cores or for a CPU with lower frequency but more cores? For example: - E5-2630Lv3, 20M Cache, 1.80 GHz, 8 cores: http://ark.intel.com/products/83357/Intel-Xeon-Processor-E5-2630L-v3-20M-Cache-1_80-GHz - E5-2637v3, 15M Cache, 3.50 GHz, 4 cores: http://ark.intel.com/products/83358/Intel-Xeon-Processor-E5-2637-v3-15M-Cache-3_50-GHz Or asked differently, which are the importants criteria to look at first for a CPU intended to be used in an OpenBSD firewall? Regards ML
