Ah, it looks like a pMTU problem. We negotiate a 1440 byte mss, but that's
apparently too large for something along the way; if you look at the
tcpdump below, you'll see that the sequence number from download.ceph.com
jumps from 1 up to 4285 without the packets inbetween ever arriving.
08:39:23.459684 IP6 2001:470:e959:201:f652:14ff:fe09:530.50714 >
2607:f298:6050:51f3:f816:3eff:fe50:5ec.80: Flags [S], seq 1826359551, win
26820, options [mss 8940,sackOK,TS val 60514795 ecr 0,nop,wscale 7], length
0
08:39:23.577649 IP6 2607:f298:6050:51f3:f816:3eff:fe50:5ec.80 >
2001:470:e959:201:f652:14ff:fe09:530.50714: Flags [S.], seq 3196528483, ack
1826359552, win 28560, options [mss 1440,sackOK,TS val 917313247 ecr
60514795,nop,wscale 7], length 0
08:39:23.577679 IP6 2001:470:e959:201:f652:14ff:fe09:530.50714 >
2607:f298:6050:51f3:f816:3eff:fe50:5ec.80: Flags [.], ack 1, win 210,
options [nop,nop,TS val 60514825 ecr 917313247], length 0
08:39:23.577749 IP6 2001:470:e959:201:f652:14ff:fe09:530.50714 >
2607:f298:6050:51f3:f816:3eff:fe50:5ec.80: Flags [P.], seq 1:82, ack 1, win
210, options [nop,nop,TS val 60514825 ecr 917313247], length 81
08:39:23.655644 IP6 2607:f298:6050:51f3:f816:3eff:fe50:5ec.80 >
2001:470:e959:201:f652:14ff:fe09:530.50714: Flags [.], ack 82, win 224,
options [nop,nop,TS val 917313267 ecr 60514825], length 0
08:39:23.818794 IP6 2607:f298:6050:51f3:f816:3eff:fe50:5ec.80 >
2001:470:e959:201:f652:14ff:fe09:530.50714: Flags [P.], seq 4285:4764, ack
82, win 224, options [nop,nop,TS val 917313308 ecr 60514825], length 479
08:39:23.818806 IP6 2001:470:e959:201:f652:14ff:fe09:530.50714 >
2607:f298:6050:51f3:f816:3eff:fe50:5ec.80: Flags [.], ack 1, win 218,
options [nop,nop,TS val 60514885 ecr 917313267,nop,nop,sack 1 {4285:4764}],
length 0
08:39:28.706360 IP6 2607:f298:6050:51f3:f816:3eff:fe50:5ec.80 >
2001:470:e959:201:f652:14ff:fe09:530.50714: Flags [F.], seq 4764, ack 82,
win 224, options [nop,nop,TS val 917314529 ecr 60514885], length 0
08:39:28.706384 IP6 2001:470:e959:201:f652:14ff:fe09:530.50714 >
2607:f298:6050:51f3:f816:3eff:fe50:5ec.80: Flags [.], ack 1, win 218,
options [nop,nop,TS val 60516107 ecr 917313267,nop,nop,sack 1 {4285:4765}],
length 0
08:40:28.709596 IP6 2001:470:e959:201:f652:14ff:fe09:530.50714 >
2607:f298:6050:51f3:f816:3eff:fe50:5ec.80: Flags [.], ack 1, win 218,
options [nop,nop,TS val 60531108 ecr 917313267,nop,nop,sack 1 {4285:4765}],
length 0
08:40:28.875876 IP6 2607:f298:6050:51f3:f816:3eff:fe50:5ec.80 >
2001:470:e959:201:f652:14ff:fe09:530.50714: Flags [.], ack 82, win 224,
options [nop,nop,TS val 917329571 ecr 60516107], length 0
FWIW, it looks like eu.ceph.com uses an MSS of 1420, not 1440.
Scott
On Sat, Nov 7, 2015 at 8:30 AM Scott Laird <[email protected]> wrote:
> FWIW, I'm also having problems connecting to download.ceph.com over IPv6,
> from a HE tunnel. I can talk to eu.ceph.com just fine.
>
> I'm seeing 100% failures over HTTP. Here's a traceroute, including my
> address:
>
> # traceroute6 download.ceph.com
> traceroute to download.ceph.com (2607:f298:6050:51f3:f816:3eff:fe50:5ec)
> from 2001:470:e959:201:f652:14ff:fe09:530, 30 hops max, 24 byte packets
> 1 2001:470:e959:201:52c5:8dff:febe:a981
> (2001:470:e959:201:52c5:8dff:febe:a981) 1.013 ms 0.923 ms 0.8 ms
> 2 2001:470:e959:300:210:db01:2cff:1000
> (2001:470:e959:300:210:db01:2cff:1000) 0.313 ms 0.343 ms 0.177 ms
> 3 2001:470:b:685::1 (2001:470:b:685::1) 0.728 ms 0.408 ms 0.452 ms
> 4 scottlaird-2.tunnel.tserv14.sea1.ipv6.he.net (2001:470:a:685::1)
> 7.75 ms 7.831 ms 7.487 ms
> 5 v225.core1.sea1.he.net (2001:470:0:9b::1) 4.928 ms 15.432 ms 5.014
> ms
> 6 10ge13-4.core1.sjc2.he.net (2001:470:0:1c7::1) 34.031 ms 24.544 ms
> 25.762 ms
> 7 2001:470:0:34f::2 (2001:470:0:34f::2) 23.92 ms 23.942 ms 24.185 ms
> 8 2001:428::205:171:203:158 (2001:428::205:171:203:158) 74.452 ms
> 74.568 ms 74.53 ms
> 9 2001:428:2402:10:0:d:0:2 (2001:428:2402:10:0:d:0:2) 75.029 ms 74.792
> ms 75.145 ms
> 10 border11-bbnet1.wdc002.pnap.net (2600:c08:0:101:0:2:1:11) 160.907 ms
> 245.192 ms 204.006 ms
> 11 2600:c08:2002:d::2 (2600:c08:2002:d::2) 78.141 ms 78.102 ms 77.949
> ms
> 12 ip-2607-f298-5-cc01--1.dreamhost.com (2607:f298:5:cc01::1) 79.807 ms
> 80.133 ms 77.979 ms
> 13 ip-2607-f298-5-cc08--2.dreamhost.com (2607:f298:5:cc08::2) 76.345 ms
> 76.229 ms 77.439 ms
> 14 2607:f298:5:110d:f816:3eff:fe79:ad5d
> (2607:f298:5:110d:f816:3eff:fe79:ad5d) 84.761 ms 79.603 ms 78.729 ms
> 15 * * *
> 16 * * *
>
>
> On Thu, Oct 15, 2015 at 11:32 PM Corin Langosch <
> [email protected]> wrote:
>
>> download.ceph.com resolves to 2607:f298:6050:51f3:f816:3eff:fe50:5ec
>> here. Ping seems to be blocked. Connect to port 80
>> works every few requests, probably 50%. So I assume there's some
>> load-balancer there with a dead backend, which the
>> load-balancer didn't detect/ kick...just guessing. Best Corin
>>
>> Am 16.10.2015 um 08:27 schrieb Björn Lässig:
>> > Getting the same error here.
>> > With sixxs 4 out of 5 wgets are failing. ping6 is dropped.
>> >
>> > We tried from different sites in .at .uk and .de. Tries from uk and de
>> are failing mostly. at is working.
>> >
>> > does ''ping6 download.ceph.com'' works for you?
>> >
>> > regards
>> > Björn Lässig
>> >
>> > _______________________________________________
>> > ceph-users mailing list
>> > [email protected]
>> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>> _______________________________________________
>> ceph-users mailing list
>> [email protected]
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
>
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
