Dear CephFS experts... We are using Ceph and CephFS 9.2.0. CephFS clients are being mounted via ceph-fuse.
We recently noticed the firewall from certain CephFS clients dropping connections with OSDs as SRC. This is something which is not systematic but we noticed happening at least once. Here is an example of the firewall log. IPTABLES-IN IN=eth0 OUT= MAC=<SOME_MAC_ADDRESS> SRC=<SOME_OSD_IP> DST=<SOME_CLIENT_IP> LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=42315 DF PROTO=TCP SPT=6802 DPT=57950 WINDOW=243 RES=0x00 ACK URGP=0 and of the netstat output executed in the OSD at the same time tcp 0 0 <SOME_OSD_IP>:6800 <SOME_CLIENT_IP>:37893 ESTABLISHED 23987/ceph-osd tcp 0 0 <SOME_OSD_IP>:6810 <SOME_CLIENT_IP>:50875 ESTABLISHED 23971/ceph-osd tcp 1 1 <SOME_OSD_IP>:6802 <SOME_CLIENT_IP>:57950 CLOSING - <--------- (...) In normal situations, we always have seen that it is the ceph-fuse client that starts the connections to OSDs, and the connection back from OSDs is accepted by the client firewall because it is a related,established connection. The fact that we saw drops in our client firewalls tell us that OSDs are trying to connect to the clients as a new connection. Up to now I haven't seen the situation again but the infrastructure is not being used at the moment while when it happened it was under (some) load. My basic dd bare tests do not show anything abnormal and I have not been able to replicate the event. We do not understand our observation and we wonder if this is 1) something expected (and in this case we will open our clients firewalls) 2) an unexpected behaviour (such as ceph-fuse breaking the established connections and the OSD is not aware of it) Any comment is welcome. TIA G. _______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
