Re: [pfSense] IPsec - how to assess encryption is active?
Thanks Jim for this explanation. It clarifies a lot of things. Including, if I followed you correctly, that I did the right thing for now to switch to IPsec using AES-GCM, mostly blindly following the recommendations of the latest pfSense book (January 2016). -- Meilleures salutations, Met vriendelijke groeten, Best Regards, Olivier Mascia (from mobile device), integral.be/om > Le 29 avr. 2016 à 20:45, Jim Thompson a écrit : > > Because OpenVPN uses tun/tap, and there is a HUGE amount of overhead in that. > >“HUGGGEEE!” — Donald J. Trump > > The statement "On a modern intel system, the intel chip itself (or AMD) has > AES128 or better implemented in hardware. “ is incorrect. Modern Intel / > AMD parts have instructions that can accelerate the AES algorithm. > >• AESENC. This instruction performs a single round of encryption. The > instruction combines the four steps of the AES algorithm - ShiftRows, > SubBytes, MixColumns & AddRoundKey into a single instruction. >• AESENCLAST. Instruction for the last round of encryption. Combines the > ShiftRows, SubBytes, & AddRoundKey steps into one instruction. >• AESDEC. Instruction for a single round of decryption. This combines the > four steps of AES - InvShiftRows, InvSubBytes, InvMixColumns, AddRoundKey > into a single instruction >• AESDECLAST. Performs last round of decryption. It combines InvShiftRows, > InvSubBytes, AddRoundKey into one instruction. >• AESKEYGENASSIST is used for generating the round keys used for > encryption. >• AESIMC is used for converting the encryption round keys to a form usable > for decryption using the Equivalent Inverse Cipher. >• PCLMULQDQ is used for carryless multiply (CLMUL), which is used in > AES-GCM. > > The other issue is that encryption without a HMAC is all but worthless. (It > increases privacy, but not security.) Typically the HMAC involves an entire > second pass over the packet, and this isn’t accelerated. Very new Intel CPUs > have some acceleration support for SHA (SHA1, SHA256, etc), but it’s not > anything like hardware offload. > > This is why AEAD modes (such as AES-GCM) exist, and why we added support for > AES-GCM to IPsec for FreeBSD.OpenVPN is supposed to get support for AEAD > (GCM) in OpenVPN 2.4. > But that’s not going to solve the issue with the overhead of tun/tap. That’s > going to take actual work, putting OpenVPN over netmap, or DPDK, or something > like that. > > Versus AES-NI, actual hardware offload, using something like Intel > QuickAssist, is much (much) faster. We’ve run nearly 20Gbps using a CPIC > card. This tweet says “10Gbps”, but using two tunnels, we got it to 17Gbps > https://twitter.com/gonzopancho/status/703677820694720512 with an otherwise > unmodified system. That was AES-CBC-128 + HMAC-SHA1, IIRC. Yes, QAT will > accelerate SHA. That’s not going to happen on FreeBSD without a lot of > work, because the IPsec stack on FreeBSD needs….. a lot of work. (It’s a bit > of a hot mess, see upcoming BSDcan talk. > http://www.bsdcan.org/2016/schedule/events/727.en.html) > > net-net: we accelerated IPsec using AES-GCM (leveraging AES-NI) first, > because that was going to be the most benefit. > > Jim > (Yes, we tried OpenVPN with QAT, tun/tap is the blocker here. See above, or > my repeated statements on this list, the forum, and elsewhere.) > > >> On Apr 29, 2016, at 1:10 PM, Olivier Mascia wrote: >> >> Indeed. >> Why didn't the OpenVPN tunnel show me that level of perf, despite settings >> for using hardware acceleration, is another story, but I'm happy with the >> IPsec results and will stick to that on this link. >> >> Thanks for having confirmed me I hadn't fallen in a rabbit hole. >> :) >> >> -- >> Meilleures salutations, Met vriendelijke groeten, Best Regards, >> Olivier Mascia, integral.be/om >> >>> Le 29 avr. 2016 à 19:58, ED Fochler a écrit : >>> >>> On a modern intel system, the intel chip itself (or AMD) has AES128 or >>> better implemented in hardware. I get ~700Mb on sftp on my macbook air >>> 2012 like that, so those numbers are exactly where I’d expect the CPU to be >>> maxed out doing AES128 or AES256 encryption. That’s what hardware >>> acceleration feels like. You should see the CPU (or one core at least) on >>> the IPSec tunnel ends being fully occupied at that throughput. >>> >>>ED. >>> >>> On 2016, Apr 29, at 1:52 PM, Olivier Mascia wrote: Seeing throughput I did not expected with an IPsec tunnel compared to what I was seeing using OpenVPN (which I always used up to the perf discrepancy I discovered today on a new link), I wonder if it really encrypts anything. Phase 1 is set for AES256, SHA256 DH group 2. Phase 2 is set for ESP AES256-GCM 128 bits and SHA256. No other encryption / hash is checked as alternatives on Phase 2. I'd say I'm good to go that way, but I'm driving be
Re: [pfSense] IPsec - how to assess encryption is active?
Because OpenVPN uses tun/tap, and there is a HUGE amount of overhead in that. “HUGGGEEE!” — Donald J. Trump The statement "On a modern intel system, the intel chip itself (or AMD) has AES128 or better implemented in hardware. “ is incorrect. Modern Intel / AMD parts have instructions that can accelerate the AES algorithm. • AESENC. This instruction performs a single round of encryption. The instruction combines the four steps of the AES algorithm - ShiftRows, SubBytes, MixColumns & AddRoundKey into a single instruction. • AESENCLAST. Instruction for the last round of encryption. Combines the ShiftRows, SubBytes, & AddRoundKey steps into one instruction. • AESDEC. Instruction for a single round of decryption. This combines the four steps of AES - InvShiftRows, InvSubBytes, InvMixColumns, AddRoundKey into a single instruction • AESDECLAST. Performs last round of decryption. It combines InvShiftRows, InvSubBytes, AddRoundKey into one instruction. • AESKEYGENASSIST is used for generating the round keys used for encryption. • AESIMC is used for converting the encryption round keys to a form usable for decryption using the Equivalent Inverse Cipher. • PCLMULQDQ is used for carryless multiply (CLMUL), which is used in AES-GCM. The other issue is that encryption without a HMAC is all but worthless. (It increases privacy, but not security.) Typically the HMAC involves an entire second pass over the packet, and this isn’t accelerated. Very new Intel CPUs have some acceleration support for SHA (SHA1, SHA256, etc), but it’s not anything like hardware offload. This is why AEAD modes (such as AES-GCM) exist, and why we added support for AES-GCM to IPsec for FreeBSD.OpenVPN is supposed to get support for AEAD (GCM) in OpenVPN 2.4. But that’s not going to solve the issue with the overhead of tun/tap. That’s going to take actual work, putting OpenVPN over netmap, or DPDK, or something like that. Versus AES-NI, actual hardware offload, using something like Intel QuickAssist, is much (much) faster. We’ve run nearly 20Gbps using a CPIC card. This tweet says “10Gbps”, but using two tunnels, we got it to 17Gbps https://twitter.com/gonzopancho/status/703677820694720512 with an otherwise unmodified system. That was AES-CBC-128 + HMAC-SHA1, IIRC. Yes, QAT will accelerate SHA. That’s not going to happen on FreeBSD without a lot of work, because the IPsec stack on FreeBSD needs….. a lot of work. (It’s a bit of a hot mess, see upcoming BSDcan talk. http://www.bsdcan.org/2016/schedule/events/727.en.html) net-net: we accelerated IPsec using AES-GCM (leveraging AES-NI) first, because that was going to be the most benefit. Jim (Yes, we tried OpenVPN with QAT, tun/tap is the blocker here. See above, or my repeated statements on this list, the forum, and elsewhere.) > On Apr 29, 2016, at 1:10 PM, Olivier Mascia wrote: > > Indeed. > Why didn't the OpenVPN tunnel show me that level of perf, despite settings > for using hardware acceleration, is another story, but I'm happy with the > IPsec results and will stick to that on this link. > > Thanks for having confirmed me I hadn't fallen in a rabbit hole. > :) > > -- > Meilleures salutations, Met vriendelijke groeten, Best Regards, > Olivier Mascia, integral.be/om > >> Le 29 avr. 2016 à 19:58, ED Fochler a écrit : >> >> On a modern intel system, the intel chip itself (or AMD) has AES128 or >> better implemented in hardware. I get ~700Mb on sftp on my macbook air 2012 >> like that, so those numbers are exactly where I’d expect the CPU to be maxed >> out doing AES128 or AES256 encryption. That’s what hardware acceleration >> feels like. You should see the CPU (or one core at least) on the IPSec >> tunnel ends being fully occupied at that throughput. >> >> ED. >> >> >>> On 2016, Apr 29, at 1:52 PM, Olivier Mascia wrote: >>> >>> Seeing throughput I did not expected with an IPsec tunnel compared to what >>> I was seeing using OpenVPN (which I always used up to the perf discrepancy >>> I discovered today on a new link), I wonder if it really encrypts anything. >>> >>> Phase 1 is set for AES256, SHA256 DH group 2. >>> Phase 2 is set for ESP AES256-GCM 128 bits and SHA256. >>> >>> No other encryption / hash is checked as alternatives on Phase 2. >>> >>> I'd say I'm good to go that way, but I'm driving between 500 and 750 Mbps >>> through the tunnel (transfer rate of ~45 to ~80 MB/sec in Windows File >>> explorer between filesystems on each side of the tunnel), and I quite >>> couldn't believe it. >>> >>> Could something be wrong? >>> >>> -- >>> Meilleures salutations, Met vriendelijke groeten, Best Regards, >>> Olivier Mascia, integral.be/om >>> >>> >>> ___ >>> pfSense mailing list >>> https://lists.pfsense.org/mailman/listinfo/list >>> Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] IPsec - how to assess encryption is active?
Indeed. Why didn't the OpenVPN tunnel show me that level of perf, despite settings for using hardware acceleration, is another story, but I'm happy with the IPsec results and will stick to that on this link. Thanks for having confirmed me I hadn't fallen in a rabbit hole. :) -- Meilleures salutations, Met vriendelijke groeten, Best Regards, Olivier Mascia, integral.be/om > Le 29 avr. 2016 à 19:58, ED Fochler a écrit : > > On a modern intel system, the intel chip itself (or AMD) has AES128 or better > implemented in hardware. I get ~700Mb on sftp on my macbook air 2012 like > that, so those numbers are exactly where I’d expect the CPU to be maxed out > doing AES128 or AES256 encryption. That’s what hardware acceleration feels > like. You should see the CPU (or one core at least) on the IPSec tunnel ends > being fully occupied at that throughput. > > ED. > > >> On 2016, Apr 29, at 1:52 PM, Olivier Mascia wrote: >> >> Seeing throughput I did not expected with an IPsec tunnel compared to what I >> was seeing using OpenVPN (which I always used up to the perf discrepancy I >> discovered today on a new link), I wonder if it really encrypts anything. >> >> Phase 1 is set for AES256, SHA256 DH group 2. >> Phase 2 is set for ESP AES256-GCM 128 bits and SHA256. >> >> No other encryption / hash is checked as alternatives on Phase 2. >> >> I'd say I'm good to go that way, but I'm driving between 500 and 750 Mbps >> through the tunnel (transfer rate of ~45 to ~80 MB/sec in Windows File >> explorer between filesystems on each side of the tunnel), and I quite >> couldn't believe it. >> >> Could something be wrong? >> >> -- >> Meilleures salutations, Met vriendelijke groeten, Best Regards, >> Olivier Mascia, integral.be/om >> >> >> ___ >> pfSense mailing list >> https://lists.pfsense.org/mailman/listinfo/list >> Support the project with Gold! https://pfsense.org/gold > > ___ > pfSense mailing list > https://lists.pfsense.org/mailman/listinfo/list > Support the project with Gold! https://pfsense.org/gold ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] IPsec - how to assess encryption is active?
On a modern intel system, the intel chip itself (or AMD) has AES128 or better implemented in hardware. I get ~700Mb on sftp on my macbook air 2012 like that, so those numbers are exactly where I’d expect the CPU to be maxed out doing AES128 or AES256 encryption. That’s what hardware acceleration feels like. You should see the CPU (or one core at least) on the IPSec tunnel ends being fully occupied at that throughput. ED. > On 2016, Apr 29, at 1:52 PM, Olivier Mascia wrote: > > Seeing throughput I did not expected with an IPsec tunnel compared to what I > was seeing using OpenVPN (which I always used up to the perf discrepancy I > discovered today on a new link), I wonder if it really encrypts anything. > > Phase 1 is set for AES256, SHA256 DH group 2. > Phase 2 is set for ESP AES256-GCM 128 bits and SHA256. > > No other encryption / hash is checked as alternatives on Phase 2. > > I'd say I'm good to go that way, but I'm driving between 500 and 750 Mbps > through the tunnel (transfer rate of ~45 to ~80 MB/sec in Windows File > explorer between filesystems on each side of the tunnel), and I quite > couldn't believe it. > > Could something be wrong? > > -- > Meilleures salutations, Met vriendelijke groeten, Best Regards, > Olivier Mascia, integral.be/om > > > ___ > pfSense mailing list > https://lists.pfsense.org/mailman/listinfo/list > Support the project with Gold! https://pfsense.org/gold ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
[pfSense] IPsec - how to assess encryption is active?
Seeing throughput I did not expected with an IPsec tunnel compared to what I was seeing using OpenVPN (which I always used up to the perf discrepancy I discovered today on a new link), I wonder if it really encrypts anything. Phase 1 is set for AES256, SHA256 DH group 2. Phase 2 is set for ESP AES256-GCM 128 bits and SHA256. No other encryption / hash is checked as alternatives on Phase 2. I'd say I'm good to go that way, but I'm driving between 500 and 750 Mbps through the tunnel (transfer rate of ~45 to ~80 MB/sec in Windows File explorer between filesystems on each side of the tunnel), and I quite couldn't believe it. Could something be wrong? -- Meilleures salutations, Met vriendelijke groeten, Best Regards, Olivier Mascia, integral.be/om ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
Re: [pfSense] Debugging a bi-directional speed discrepancy through an OpenVPN tunnel --> IPsec
Having tried a large number of things, it looks like an UDP receive issue on (B). Connecting the tunnel through TCP transport instead of UDP brings the throughput around 75 Mbps in both directions. Over UDP, it is a good ~300 Mbps in one direction and a mere ~15 Mbps in the opposite (A to B). That means in the direction being problematic, throughput is actually higher over TCP than UDP, which of course is counterintuitive. So I decided to test an IPsec tunnel instead, in order to rule out OpenVPN specific issues... I never used IPsec tunnel before but could establish one rather simply following the book recommended settings and... I'm quite surprised at the throughput I get (very high). To the point that I'm wondering if it's really encrypting anything! I'm leaving this (self) thread here and starting another one about IPsec. -- Meilleures salutations, Met vriendelijke groeten, Best Regards, Olivier Mascia, integral.be/om > Le 29 avr. 2016 à 11:45, Olivier Mascia a écrit : > > Dear all, > > In case some of you would have an idea what to look for and adjust, here is a > strange issue I have between two end-points of an OpenVPN tunnel. Both sites > each have >= 1 Gbps connectivity to Internet. > > One site (A) is still using pfSense 2.2.2-REL on 'Intel(R) Xeon(R) CPU E31270 > @ 3.40GHz, 4 CPUs: 1 package(s) x 4 core(s)' (hyper-threading turned off). > This is a nanobsd configuration. > > New site (B) is using pfSense 2.3-REL on 'Intel(R) Xeon(R) CPU E5-2690 v2 @ > 3.00GHz, 2 CPUs: 1 package(s) x 2 core(s)' (this is actually a VM). This is > full setup. > > I have an OpenVPN tunnel between both (peer to peer, shared key, AES-128-CBC, > SHA1). > > Using the tunnel for file transfers between both sites, I peak over 350 Mbps > inside the tunnel from (B) to (A). But from (A) to (B) I peak at ~14 Mbps. > Which looks really strange. I'm wondering where is the culprit: sending from > (A), or receiving on (B). > > Using iperf3 with 3 to 5 threads, outside of the VPN, but through both > pfSense anyway, I consistently get 800 to 900 Mbps, either (A) to (B) or (B) > to (A). It is only within the OpenVPN tunnel that I can see the asymmetric > speed. And it puzzles me. > > If you have any kind of idea about what to look for, I'll take whatever you > give me. > Thanks for reading me, > -- > Meilleures salutations, Met vriendelijke groeten, Best Regards, > Olivier Mascia, integral.be/om ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
[pfSense] Debugging a bi-directional speed discrepancy through an OpenVPN tunnel
Dear all, In case some of you would have an idea what to look for and adjust, here is a strange issue I have between two end-points of an OpenVPN tunnel. Both sites each have >= 1 Gbps connectivity to Internet. One site (A) is still using pfSense 2.2.2-REL on 'Intel(R) Xeon(R) CPU E31270 @ 3.40GHz, 4 CPUs: 1 package(s) x 4 core(s)' (hyper-threading turned off). This is a nanobsd configuration. New site (B) is using pfSense 2.3-REL on 'Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz, 2 CPUs: 1 package(s) x 2 core(s)' (this is actually a VM). This is full setup. I have an OpenVPN tunnel between both (peer to peer, shared key, AES-128-CBC, SHA1). Using the tunnel for file transfers between both sites, I peak over 350 Mbps inside the tunnel from (B) to (A). But from (A) to (B) I peak at ~14 Mbps. Which looks really strange. I'm wondering where is the culprit: sending from (A), or receiving on (B). Using iperf3 with 3 to 5 threads, outside of the VPN, but through both pfSense anyway, I consistently get 800 to 900 Mbps, either (A) to (B) or (B) to (A). It is only within the OpenVPN tunnel that I can see the asymmetric speed. And it puzzles me. If you have any kind of idea about what to look for, I'll take whatever you give me. Thanks for reading me, -- Meilleures salutations, Met vriendelijke groeten, Best Regards, Olivier Mascia, integral.be/om ___ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
