Thanks Wido, it's fixed. I'm going to put the explanation if somebody runs into the same error.
The MTU was defined on the client side and it was 9000. The 'ifconfig' shows the value established but if I ask directly the /proc filesystem it shows the following: root@ceph-node03:~# cat /proc/sys/net/ipv6/conf/eno1/mtu 1500 root@ceph-node03:~# If I restart the interface it shows 9000 for a while and then it changes back to 1500. After some research it turns out that the router offers a MTU 1500 in the SLAAC parameters so when the session is 'refreshed', the client applies the wrong value (1500). The network guys changed the MTU parameter offered via SLAAC and now it's working: root@ceph-node03:~# cat /proc/sys/net/ipv6/conf/eno1/mtu 9000 root@ceph-node03:~# ping6 -c 3 -M do -s 8952 ceph-node01 PING ceph-node01(2a02:x:x:x:x:x:x:x) 8952 data bytes 8960 bytes from 2a02:x:x:x:x:x:x:x: icmp_seq=1 ttl=64 time=0.271 ms 8960 bytes from 2a02:x:x:x:x:x:x:x: icmp_seq=2 ttl=64 time=0.216 ms 8960 bytes from 2a02:x:x:x:x:x:x:x: icmp_seq=3 ttl=64 time=0.280 ms --- ceph-node01 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 0.216/0.255/0.280/0.033 ms root@ceph-node03:~# 2017-10-27 16:02 GMT+02:00 Wido den Hollander <[email protected]>: > > > Op 27 oktober 2017 om 14:22 schreef Félix Barbeira <[email protected] > >: > > > > > > Hi, > > > > I'm trying to configure a ceph cluster using IPv6 only but I can't enable > > jumbo frames. I made the definition on the > > 'interfaces' file and it seems like the value is applied but when I test > it > > looks like only works on IPv4, not IPv6. > > > > It works on IPv4: > > > > root@ceph-node01:~# ping -c 3 -M do -s 8972 ceph-node02 > > > > PING ceph-node02 (x.x.x.x) 8972(9000) bytes of data. > > 8980 bytes from ceph-node02 (x.x.x.x): icmp_seq=1 ttl=64 time=0.474 ms > > 8980 bytes from ceph-node02 (x.x.x.x): icmp_seq=2 ttl=64 time=0.254 ms > > 8980 bytes from ceph-node02 (x.x.x.x): icmp_seq=3 ttl=64 time=0.288 ms > > > > Verify with Wireshark/tcpdump if it really sends 9k packets. I doubt it. > > > --- ceph-node02 ping statistics --- > > 3 packets transmitted, 3 received, 0% packet loss, time 2000ms > > rtt min/avg/max/mdev = 0.254/0.338/0.474/0.099 ms > > > > root@ceph-node01:~# > > > > But *not* in IPv6: > > > > root@ceph-node01:~# ping6 -c 3 -M do -s 8972 ceph-node02 > > PING ceph-node02(x:x:x:x:x:x:x:x) 8972 data bytes > > ping: local error: Message too long, mtu=1500 > > ping: local error: Message too long, mtu=1500 > > ping: local error: Message too long, mtu=1500 > > > > Like Ronny already mentioned, check the switches and the receiver. There > is a 1500 MTU somewhere configured. > > Wido > > > --- ceph-node02 ping statistics --- > > 4 packets transmitted, 0 received, +4 errors, 100% packet loss, time > 3024ms > > > > root@ceph-node01:~# > > > > > > > > root@ceph-node01:~# ifconfig > > eno1 Link encap:Ethernet HWaddr 24:6e:96:05:55:f8 > > inet6 addr: 2a02:x:x:x:x:x:x:x/64 Scope:Global > > inet6 addr: fe80::266e:96ff:fe05:55f8/64 Scope:Link > > UP BROADCAST RUNNING MULTICAST *MTU:9000* Metric:1 > > RX packets:633318 errors:0 dropped:0 overruns:0 frame:0 > > TX packets:649607 errors:0 dropped:0 overruns:0 carrier:0 > > collisions:0 txqueuelen:1000 > > RX bytes:463355602 (463.3 MB) TX bytes:498891771 (498.8 MB) > > > > lo Link encap:Local Loopback > > inet addr:127.0.0.1 Mask:255.0.0.0 > > inet6 addr: ::1/128 Scope:Host > > UP LOOPBACK RUNNING MTU:65536 Metric:1 > > RX packets:127420 errors:0 dropped:0 overruns:0 frame:0 > > TX packets:127420 errors:0 dropped:0 overruns:0 carrier:0 > > collisions:0 txqueuelen:1 > > RX bytes:179470326 (179.4 MB) TX bytes:179470326 (179.4 MB) > > > > root@ceph-node01:~# > > > > root@ceph-node01:~# cat /etc/network/interfaces > > # This file describes network interfaces avaiulable on your system > > # and how to activate them. For more information, see interfaces(5). > > > > source /etc/network/interfaces.d/* > > > > # The loopback network interface > > auto lo > > iface lo inet loopback > > > > # The primary network interface > > auto eno1 > > iface eno1 inet6 auto > > post-up ifconfig eno1 mtu 9000 > > root@ceph-node01:# > > > > > > Please help! > > > > -- > > Félix Barbeira. > > _______________________________________________ > > ceph-users mailing list > > [email protected] > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Félix Barbeira.
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
