On Sat, Dec 15, 2018 at 06:18:39PM -0600, Theodore Wynnychenko wrote:
<snip>
> On the local gateway:
>
> 17:37:00.199269 (authentic,confidential): SPI 0x7b90f84c: 172.30.1.20.20692 >
> 172.30.6.201.443: S 3823001077:3823001077(0) win 16384 <mss
> 1460,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 48604571 0> (encap)
>
> 17:37:00.235313 (authentic,confidential): SPI 0x29a95d18: 172.30.6.201.443 >
> 172.30.1.20.20692: S 833135398:833135398(0) ack 3823001078 win 16384 <mss
> 65496,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 730029501 48604571> (DF)
> (encap)
>
> 17:37:00.235335 (authentic,confidential): SPI 0x7b90f84c: 172.30.1.20.20692 >
> 172.30.6.201.443: . ack 1 win 256 <nop,nop,timestamp 48604571 730029501>
> (encap)
>
> 17:37:00.236086 (unprotected): SPI 0x000054ab: 172.30.1.20.20692 >
> 172.30.6.201.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 48604571
> 730029501> (encap)
>
> 17:37:01.726509 (unprotected): SPI 0x000054ab: 172.30.1.20.20692 >
> 172.30.6.201.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 48604574
> 730029501> (encap)
>
> 17:37:04.726571 (unprotected): SPI 0x000054ab: 172.30.1.20.20692 >
> 172.30.6.201.443: P 1:197(196) ack 1 win 256 <nop,nop,timestamp 48604580
> 730029501> (encap)
This looks odd to me. Why unprotected?
> 17:37:10.726697 (unprotected): SPI 0x000054ab: 172.30.1.20.20692 >
> 172.30.6.201.443: FP 1:197(196) ack 1 win 256 <nop,nop,timestamp 48604592
> 730029516> (encap)
>
> 17:37:11.724643 (authentic,confidential): SPI 0x29a95d18: 172.30.6.201.443 >
> 172.30.1.20.20692: F 1:1(0) ack 1 win 271 <nop,nop,timestamp 730029524
> 48604571> (DF) (encap)
>
> 17:37:11.724680 (authentic,confidential): SPI 0x7b90f84c: 172.30.1.20.20692 >
> 172.30.6.201.443: F 197:197(0) ack 2 win 256 <nop,nop,timestamp 48604594
> 730029524> (encap)
>
> 17:37:11.759035 (authentic,confidential): SPI 0x29a95d18: 172.30.6.201.443 >
> 172.30.1.20.20692: . ack 1 win 271 <nop,nop,timestamp 730029524 48604571>
> (DF) (encap)
Packet fragmentation is something to investigate.
>
> But, tcpdump of enc0 on the remote gateway only shows:
>
> 17:37:00.207517 (authentic,confidential): SPI 0x7b90f84c: 172.30.1.20.20692 >
> 172.30.6.201.443: S 3823001077:3823001077(0) win 16384 <mss
> 1460,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 48604571 0> (encap)
>
> 17:37:00.207551 (authentic,confidential): SPI 0x29a95d18: 172.30.6.201.443 >
> 172.30.1.20.20692: S 833135398:833135398(0) ack 3823001078 win 16384 <mss
> 65496,nop,nop,sackOK,nop,wscale 6,nop,nop,timestamp 730029501 48604571> (DF)
> (encap)
>
> 17:37:00.244461 (authentic,confidential): SPI 0x7b90f84c: 172.30.1.20.20692 >
> 172.30.6.201.443: . ack 1 win 256 <nop,nop,timestamp 48604571 730029501>
> (encap)
>
> 17:37:07.792702 (authentic,confidential): SPI 0x7b90f84c: 172.30.1.20.20692 >
> 172.30.6.201.443: F 197:197(0) ack 1 win 256 <nop,nop,timestamp 48604586
> 730029501> (encap)
>
> 17:37:07.792724 (authentic,confidential): SPI 0x29a95d18: 172.30.6.201.443 >
> 172.30.1.20.20692: . ack 1 win 271 <nop,nop,timestamp 730029516
> 48604571,nop,nop,sack 1 {197:197} > (DF) (encap)
> Needless to say, if I try these connections WITHOUT traversing the iked
> vpn, the openssl s_client to s_server connection works as expected.
>
> Not being in any way an expert on these things, I cannot understand why
> "regular" tcp packets are able to traverse the VPN and emerge on the other
> side, while tcp pakets to the same port attempting to establish a secure
> connection are lost. Why are the tcp PUSH data packets that leave the
> local gateway on enc0 never seen on the remote gateway's ecn0?
>
> I am really hoping that somebody has some sort idea of what I can do to
> fix whatever it is that I have broken. I will be happy to send more
> specific information, I just don't know what that might be.
I'm not well versed in these issues but if I were in your shoes I would
1) Figure out why those packets were unprotected. (Could be normal for all I
know, but in a quick test on my enc0 I didn't see any packets like that.)
2) Make sure the tunnel handles fragmentation correctly. Are fragments being
dropped? Are reassembled fragments being dropped?
2.a) Relatedly, make sure the tunnel handles pMTU discovery. Do ICMP
Fragmentation Needed packets make it back through the tunnel? pMTU and ICMP
issues are very common with IPSec tunnels. IME most people "fix" these
issues by forcing a lower MSS or setting a lower MTU at the ingress point
rather than properly configuring routing so ICMP errors are properly routed.
I've experienced this issue myself and had to learn the hard way.
3) From an earlier post it looks like you're using ipcomp. You should remove
this complication while debugging. It's possible ipcomp is hiding MTU
issues.
