Re: [pfSense] NIC Offloading Setting Questions
On Fri, Mar 6, 2015 at 4:02 PM, Jim Thompson wrote: > Second, none of these were offload-related. > > Third, the config file doesn't overwrite loader.conf.local. > I didn't say they were related; I just said it would be a nice thing if the hardware specific settings were publicly stated on the product pages. ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] NIC Offloading Setting Questions
> On Mar 6, 2015, at 4:00 AM, Vick Khera wrote: > > >> On Wed, Mar 4, 2015 at 5:08 PM, Jim Thompson wrote: >> > Ah, so I should have asked _before_ ordering the NICs? $;-) >> >> There are many of you, and few of us. > > As a Netgate and pfSense customer, I think it would help *everyone* if you > just posted the "special" settings for the devices you sell. For example, the > NIC settings in loader.con.local, and the options for things like the thermal > sensors and these NIC offloading settings. I now they come pre-configured > with such, but the first thing I do is upload my old config to replace the > old device, and now those settings are unknown to me. Having to look thru > every page to find them before is just a time suck. First, there were some special /boot/loader.conf.local settings in 2.1.x, but I believe the 2.2 factory load uses the same version of this file that the "community" build does. Second, none of these were offload-related. Third, the config file doesn't overwrite loader.conf.local. Jim___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] NIC Offloading Setting Questions
On 2015-Mar-05, at 11:46 AM, Chris Buechler wrote: > The description of what's enabled/disabled got confused from Jim's > earlier post I think. LRO and TSO are both disabled by default, > hardware checksum offloading is enabled by default. Just for the record, Jim's message ended with: --- It’s possible, if everything else is right, then IP checksum offload can provide a modest performance improvement, but this is unlikely to be more than “noticeable” at the speeds where most individuals run pfSense. However, at 10Gbps (or above), these engines become quite useful. Support for these is an important component of our “3.0” effort. In case it’s not clear by now, these settings are all *disabled* by default in pfSense. --- It sounds like the "all disabled" setting would be the "safest" and, for all but the high-volume installs, offer essentially the same performance. I'll update/resend the request to the email address for whomever updates the WiKi. Regardless, Jim's explanation helped (some of) us better understand these settings. ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] NIC Offloading Setting Questions
On Thu, Mar 5, 2015 at 1:26 PM, DV wrote: > > On Wed, Mar 4, 2015 at 10:17 AM, Jim Thompson wrote: >> >> In case it’s not clear by now, these settings are all *disabled* by >> default in pfSense. > > I am pretty sure that I haven't changed these settings in my various > production pfsense devices, but they all have "Disable hardware checksum > offload" unchecked/disabled which is actually *enabled* isn't it? The description of what's enabled/disabled got confused from Jim's earlier post I think. LRO and TSO are both disabled by default, hardware checksum offloading is enabled by default. Leave things at defaults if it's working. Occasionally you might hit a driver or NIC issue with hardware checksum offloading and have to disable it, but you'll know if that's the case. Things are very slow if they function at all in those problem scenarios. ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] NIC Offloading Setting Questions
On Wed, Mar 4, 2015 at 10:17 AM, Jim Thompson wrote: > In case it's not clear by now, these settings are all *disabled* by > default in pfSense. I am pretty sure that I haven't changed these settings in my various production pfsense devices, but they all have "Disable hardware checksum offload" unchecked/disabled which is actually *enabled* isn't it? These are both 2.1 and 2.2 versions. Should i check that box? ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] NIC Offloading Setting Questions
On Wed, Mar 4, 2015 at 5:08 PM, Jim Thompson wrote: > > Ah, so I should have asked _before_ ordering the NICs? $;-) > > There are many of you, and few of us. > As a Netgate and pfSense customer, I think it would help *everyone* if you just posted the "special" settings for the devices you sell. For example, the NIC settings in loader.con.local, and the options for things like the thermal sensors and these NIC offloading settings. I now they come pre-configured with such, but the first thing I do is upload my old config to replace the old device, and now those settings are unknown to me. Having to look thru every page to find them before is just a time suck. If the special settings are shared on the product tech specs page, it would make life just a lot easier and lower your support load. ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] NIC Offloading Setting Questions
> On Mar 4, 2015, at 5:48 PM, Bryan D. wrote: > > On 2015-Mar-04, at 2:08 PM, Jim Thompson wrote: >> >> You’re aware that I work for Netgate, right? > > Well ... yes, but that item was in response to the posting by > comp...@hotrodpc.com. > > More importantly, when I see "Jim Thompson" I immediately think "ah, > expert-level response follows" -- and you always seem to come from the > understanding that many of us don't "breath 'n eat networking." I sincerely > appreciate (and learn from) such list/forum/blog/etc. postings. Interestingly, when people inside the company see email from “Jim Thompson” the reaction is often, “oh no...” > OTOH, I admit that I've sort o' lumped Netgate with pfSense, assuming little > separation ... which, I'm guessing is not the right way to think of things. > As a low-priority item, it'd be nice to see a statement about this > relationship (which may already exist, but I was unable to coax it out of Mr. > Google -- maybe I just don't know the "magic phrase”). It’s been covered to death on the forum. Jim ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] NIC Offloading Setting Questions
On 2015-Mar-04, at 2:08 PM, Jim Thompson wrote: > >> On Mar 4, 2015, at 2:02 PM, Bryan D. wrote: >> >> On 2015-Mar-04, at 6:20 AM, compdoc wrote: >> >>> For me, what happens after enabling or disabling those settings are >>> immediately apparent. >> >> I guess my approach w.r.t. a mailing list has always been that I'd like to >> help others avoid spending time learning something I can help with. As such >> (paraphrasing) "try it and you'll see" isn't a response I'd give. Of >> course, in absence of finding the answer in the documentation or via Mr. >> Google, we can always set up a test system and investigate (given the >> ominous warnings, I wouldn't have done so on a production system) ... but >> then why have the list? > > You’re aware that I work for Netgate, right? Well ... yes, but that item was in response to the posting by comp...@hotrodpc.com. More importantly, when I see "Jim Thompson" I immediately think "ah, expert-level response follows" -- and you always seem to come from the understanding that many of us don't "breath 'n eat networking." I sincerely appreciate (and learn from) such list/forum/blog/etc. postings. OTOH, I admit that I've sort o' lumped Netgate with pfSense, assuming little separation ... which, I'm guessing is not the right way to think of things. As a low-priority item, it'd be nice to see a statement about this relationship (which may already exist, but I was unable to coax it out of Mr. Google -- maybe I just don't know the "magic phrase"). > >> On 2015-Mar-04, at 7:17 AM, Jim Thompson wrote: >> >> So your effort can be of maximum benefit, I've submitted a slightly >> edited/formatted version of this to be included in the WiKi's applicable >> pfSense documentation page. > > I’m sure the pfSense guys will enjoy that. ... and, hopefully, others. ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] NIC Offloading Setting Questions
> On Mar 4, 2015, at 2:02 PM, Bryan D. wrote: > > On 2015-Mar-04, at 6:20 AM, compdoc wrote: > >> For me, what happens after enabling or disabling those settings are >> immediately apparent. > > I guess my approach w.r.t. a mailing list has always been that I'd like to > help others avoid spending time learning something I can help with. As such > (paraphrasing) "try it and you'll see" isn't a response I'd give. Of course, > in absence of finding the answer in the documentation or via Mr. Google, we > can always set up a test system and investigate (given the ominous warnings, > I wouldn't have done so on a production system) ... but then why have the > list? You’re aware that I work for Netgate, right? > On 2015-Mar-04, at 7:17 AM, Jim Thompson wrote: > >> Answering any question post-sale is “support”. > > Ah, so I should have asked _before_ ordering the NICs? $;-) There are many of you, and few of us. >>> Does anyone know the answer to my questions about the various offloading >>> settings that should be used with these cards? >> >> LRO works by aggregating [...] >> >> In case it’s not clear by now, these settings are all *disabled* by default >> in pfSense. > > Thank you for an answer that nicely goes "above and beyond" my expected "(we) > use these settings" response. > > So your effort can be of maximum benefit, I've submitted a slightly > edited/formatted version of this to be included in the WiKi's applicable > pfSense documentation page. I’m sure the pfSense guys will enjoy that. Jim ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] NIC Offloading Setting Questions
Am 04.03.2015 um 16:17 schrieb Jim Thompson: > LRO works by aggregating multiple incoming packets from a single > stream into a larger buffer before they are passed higher up the > networking stack, thus reducing the number of packets that have to be > processed. > > LRO should not be used on machines acting as routers, (and it is > quite likely that you’re using pfSense as a router or, equivalently, > a router), as it breaks the end-to-end principle and can > significantly impact performance. > > TSO is similar, but for sending. It works by queuing up large > buffers and letting the network interface card (NIC) split them into > separate packets just before transmit. > > Both LRO and TSO can help if you are an endpoint, *not a router*. > If you were using pfSense an an appliance (say, for DNS), they would > possibly help performance. > > Now onto “hardware checksum offload”: > > First, let’s briefly discuss where checksumming is used. > > The Ethernet hardware calculates the Ethernet CRC32 checksum and the > receive engine validates this checksum. If the received checksum is > wrong pfSense won’t even see the packet, as the Ethernet hardware > internally throws away the packet. (There are exceptions, such as if > the interface is in promiscuous mode.) > > Higher level checksums are “traditionally” calculated by the protocol > implementation and the completed packet is then handed over to the > hardware. Recent network hardware can perform the IP checksum > calculation, also known as checksum offloading. The network driver > won’t calculate the checksum itself but will simply hand over an > empty (zero or garbage filled) checksum field to the hardware. > > Some cards will additionally process TCP and UDP checksums, as above, > this isn’t going to be of any value on a router. > > It’s possible, if everything else is right, then IP checksum offload > can provide a modest performance improvement, but this is unlikely to > be more than “noticeable” at the speeds where most individuals run > pfSense. However, at 10Gbps (or above), these engines become quite > useful. Support for these is an important component of our “3.0” > effort. > > In case it’s not clear by now, these settings are all *disabled* by > default in pfSense. This good explanation should find a way into the wiki! Jens ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] NIC Offloading Setting Questions
On 2015-Mar-04, at 6:20 AM, compdoc wrote: > For me, what happens after enabling or disabling those settings are > immediately apparent. I guess my approach w.r.t. a mailing list has always been that I'd like to help others avoid spending time learning something I can help with. As such (paraphrasing) "try it and you'll see" isn't a response I'd give. Of course, in absence of finding the answer in the documentation or via Mr. Google, we can always set up a test system and investigate (given the ominous warnings, I wouldn't have done so on a production system) ... but then why have the list? On 2015-Mar-04, at 7:17 AM, Jim Thompson wrote: > Answering any question post-sale is “support”. Ah, so I should have asked _before_ ordering the NICs? $;-) >> Does anyone know the answer to my questions about the various offloading >> settings that should be used with these cards? > > LRO works by aggregating [...] > > In case it’s not clear by now, these settings are all *disabled* by default > in pfSense. Thank you for an answer that nicely goes "above and beyond" my expected "(we) use these settings" response. So your effort can be of maximum benefit, I've submitted a slightly edited/formatted version of this to be included in the WiKi's applicable pfSense documentation page. Bryan D. http://www.derman.com/ ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] NIC Offloading Setting Questions
> On Mar 4, 2015, at 12:54 AM, Bryan D. wrote: > > Today, having received a pair of SuperMicro AOC-SG-i2 NICs from the pfSense > store, I asked about the applicable pfSense "offloading" settings (via the > pfSense contact form). > > > Receiving an oblique (non-)response, I re-sent a query that included the > following text: > --- > [...] specifically, what should the pfSense settings be for: > - Hardware Checksum Offloading > - Hardware TCP Segmentation Offloading > - Hardware Large Receive Offloading > > I.E., can each of these be enabled when using AOC-SG-i2 NICs? > > With my current systems, segmentation and large receive offloads are > disabled. I don't remember what the default was (and it's not stated on the > configurator page) [...] > > Understand that the configurator page has warnings about these capabilities > being "... broken in some hardware drivers, ..." so, even though the NICs are > spec'd to support these capabilities, there's still the question whether the > drivers work properly [...] That's the reason for my query. > --- > > > To which I received the following response (an attitude that left me feeling > considerably less enthusiastic about trying to support the project): > --- > We do not provide pfSense support for these cards unless they are installed > in a system we sell. My suggestion is to search the forums for the tuning you > desire. > > I know this is not the answer you desire but that is our official response. > --- > > For the record, I don't really consider these questions to be "support" ... > just a clarification of the specs, which should be straightforward given that > it's a 1-product organization (and would be best stated on the store's > web-page). Answering any question post-sale is “support”. You are using a free open source product. The only cost to you is to figure out how to make it work. If you are unable or unwilling, then we (and others) offer paid support options. There is also, as whomever from Netgate explained, support options including the forum and this list. > Does anyone know the answer to my questions about the various offloading > settings that should be used with these cards? LRO works by aggregating multiple incoming packets from a single stream into a larger buffer before they are passed higher up the networking stack, thus reducing the number of packets that have to be processed. LRO should not be used on machines acting as routers, (and it is quite likely that you’re using pfSense as a router or, equivalently, a router), as it breaks the end-to-end principle and can significantly impact performance. TSO is similar, but for sending. It works by queuing up large buffers and letting the network interface card (NIC) split them into separate packets just before transmit. Both LRO and TSO can help if you are an endpoint, *not a router*. If you were using pfSense an an appliance (say, for DNS), they would possibly help performance. Now onto “hardware checksum offload”: First, let’s briefly discuss where checksumming is used. The Ethernet hardware calculates the Ethernet CRC32 checksum and the receive engine validates this checksum. If the received checksum is wrong pfSense won’t even see the packet, as the Ethernet hardware internally throws away the packet. (There are exceptions, such as if the interface is in promiscuous mode.) Higher level checksums are “traditionally” calculated by the protocol implementation and the completed packet is then handed over to the hardware. Recent network hardware can perform the IP checksum calculation, also known as checksum offloading. The network driver won’t calculate the checksum itself but will simply hand over an empty (zero or garbage filled) checksum field to the hardware. Some cards will additionally process TCP and UDP checksums, as above, this isn’t going to be of any value on a router. It’s possible, if everything else is right, then IP checksum offload can provide a modest performance improvement, but this is unlikely to be more than “noticeable” at the speeds where most individuals run pfSense. However, at 10Gbps (or above), these engines become quite useful. Support for these is an important component of our “3.0” effort. In case it’s not clear by now, these settings are all *disabled* by default in pfSense. Jim ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] NIC Offloading Setting Questions
> With my current systems, segmentation and large receive offloads are >disabled. I don't remember what the default was (and it's not stated >on the configurator page) [...] For me, what happens after enabling or disabling those settings are immediately apparent. ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
[pfSense] NIC Offloading Setting Questions
Today, having received a pair of SuperMicro AOC-SG-i2 NICs from the pfSense store, I asked about the applicable pfSense "offloading" settings (via the pfSense contact form). Receiving an oblique (non-)response, I re-sent a query that included the following text: --- [...] specifically, what should the pfSense settings be for: - Hardware Checksum Offloading - Hardware TCP Segmentation Offloading - Hardware Large Receive Offloading I.E., can each of these be enabled when using AOC-SG-i2 NICs? With my current systems, segmentation and large receive offloads are disabled. I don't remember what the default was (and it's not stated on the configurator page) [...] Understand that the configurator page has warnings about these capabilities being "... broken in some hardware drivers, ..." so, even though the NICs are spec'd to support these capabilities, there's still the question whether the drivers work properly [...] That's the reason for my query. --- To which I received the following response (an attitude that left me feeling considerably less enthusiastic about trying to support the project): --- We do not provide pfSense support for these cards unless they are installed in a system we sell. My suggestion is to search the forums for the tuning you desire. I know this is not the answer you desire but that is our official response. --- For the record, I don't really consider these questions to be "support" ... just a clarification of the specs, which should be straightforward given that it's a 1-product organization (and would be best stated on the store's web-page). Does anyone know the answer to my questions about the various offloading settings that should be used with these cards? ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
