> On Feb 23, 2016, at 7:47 PM, Walter Parker <[email protected]> wrote: > > On Tue, Feb 23, 2016 at 3:19 PM, Giles Davis <[email protected]> wrote: > >> On 19/02/2016 17:12, David Burgess wrote: >>> I'm a little surprised at your experience. A few years ago I built a >>> PFSense unit with an Intel motherboard, 1st gen Core i3 CPU, and a >>> single onboard Intel (em) GBE NIC. All routing was done through vlans >>> and it had no trouble reaching wire speed with around 50% CPU usage.
perhaps you have a different definition of ‘wire speed’. You have to fill the link with min-sized packets for “wire speed”. (It’s trivial with large packets.) This is, of course, what is probably happing with 2-3K ‘hardcore gamers’. Lots of short packets. >>> I do recommend using the net.inet.ip.fastforwarding=1 tweak if you >>> can. Note that it breaks IPSEC and captive portal. You’ll find that there is no such setting in pfSense software version 2.3, because we now use tryforward() which gives you all the speed of ‘fast forwarding’ without breaking IPsec or captive portal. (and therefore, there is nothing to ‘set’) We tried to put this into FreeBSD 10.3, but there is a rare combination of factors that result in it breaking NAT (but not the NAT used in pfSense). >>> As far as 10G NICs, I was sure I read recently that the FreeNAS people >>> were recommending Chelsio, but I can't find the reference now. >> I imagine it's probably going to be our ridiculous PPS figures that >> start to bottleneck things. There's 2-3 thousand hardcore gamers behind >> these boxes when we run our events all generating shedloads of tiny UDP >> packets, as well as a big demand for normal web browsing, downloading, >> streaming on top of all that. What we used to see was the ix (and before >> the 10G NICs the bge) driver heavily pushing single CPU cores - but at >> about ~1.2Gbit we just start seeing small amounts of packet loss - even >> when there's no obvious single cause. I'm guessing its a combination of >> a few factors, but to be honest we just move traffic off to another box >> - PL for gamers is the end of the world. :( >> >> I don't think we had set fastforwarding yet - so i'll definitely look >> into that. Don't care about IPSec or captive portal at all! >> >> We're also getting pricing for Chelsio NICs now too - so perhaps that'll >> help as well. >> >> Thanks again (and thanks Ed for those stats too). >> >> Cheers, >> Giles. > > Fun fact, Netflix is using FreeBSD and is pushing >30 Gbps from systems > using Chelsio NICs. See > http://www.slideshare.net/facepalmtarbz2/slides-41343025 for details. Fun fact, this ’Netflix’ success is using the AES-GCM code that Netgate co-developed with the FreeBSD Foundation for use with IPsec. https://lists.freebsd.org/pipermail/freebsd-security/2014-November/008029.html Fun fact #2, a future variant of that work will leverage QuickAssist. http://store.netgate.com/QuickAssist-and-Other-Cards-C210.aspx Fun fact #3, we can achieve much higher PPS with the router we’re writing (leverages DPDK) and netmap-fwd than you can with fastforward. (Where Chelsio NICs make life a bit more complex.) https://github.com/Netgate/netmap-fwd/blob/master/netmap-fwd.pdf _______________________________________________ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
