Hi, sorry to bump this old thread,
but I had this problem recently, with a linux firewall between cephfs client and cluster the problem was easy to reproduce with #firewall is enable with iptables -A FORWARD -m conntrack --ctstate INVALID -j DROP iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT (conntrack have cephfs connection) then flushing rules #iptables -F #iptables -X (Still working, conntrack still have the connection) then, reenable rules iptables -A FORWARD -m conntrack --ctstate INVALID -j DROP iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT And the cephfs mds connection is now hanging for 15minutes. (monitor connection is reestablished correctly in less than 1s) Then reason was that conntrack flag packets as invalid, because of out of window sequence. (as conntrack stop to track them when rules are flushed). A simple workaround: net.netfilter.nf_conntrack_tcp_be_liberal=1 Hope this help ! ----- Mail original ----- De: "Nick Fisk" <[email protected]> À: "ceph-users" <[email protected]> Envoyé: Vendredi 21 Octobre 2016 16:19:03 Objet: [ceph-users] Ceph and TCP States Hi, I'm just testing out using a Ceph client in a DMZ behind a FW from the main Ceph cluster. One thing I have noticed is that if the state table on the FW is emptied maybe by restarting it or just clearing the state table...etc. Then the Ceph client will hang for a long time as the TCP session can no longer pass through the FW and just gets blocked instead. I believe this behaviour can be adjusted by the "ms tcp read timeout" setting to limit its impact, but wondering if anybody has any other ideas. I'm also thinking of experimenting with either stateless FW rules for Ceph or getting the FW to send back RST packets instead of silently dropping packets. Thanks, Nick _______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
