Re: [ceph-users] CephFS client hanging and cache issues
Thanks a lot and sorry for the spam, I should have checked ! We are on 18.04, kernel is currently upgrading so if you don't hear back from me then it is fixed. Thanks for the amazing support ! On Wed, 30 Oct 2019, 09:54 Lars Täuber, wrote: > Hi. > > Sounds like you use kernel clients with kernels from canonical/ubuntu. > Two kernels have a bug: > 4.15.0-66 > and > 5.0.0-32 > > Updated kernels are said to have fixes. > Older kernels also work: > 4.15.0-65 > and > 5.0.0-31 > > > Lars > > Wed, 30 Oct 2019 09:42:16 + > Bob Farrell ==> ceph-users > : > > Hi. We are experiencing a CephFS client issue on one of our servers. > > > > ceph version 14.2.0 (3a54b2b6d167d4a2a19e003a705696d4fe619afc) nautilus > > (stable) > > > > Trying to access, `umount`, or `umount -f` a mounted CephFS volumes > causes > > my shell to hang indefinitely. > > > > After a reboot I can remount the volumes cleanly but they drop out after > < > > 1 hour of use. > > > > I see this log entry multiple times when I reboot the server: > > ``` > > cache_from_obj: Wrong slab cache. inode_cache but object is from > > ceph_inode_info > > ``` > > The machine then reboots after approx. 30 minutes. > > > > All other Ceph/CephFS clients and servers seem perfectly happy. CephFS > > cluster is HEALTH_OK. > > > > Any help appreciated. If I can provide any further details please let me > > know. > > > > Thanks in advance, > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] CephFS client hanging and cache issues
Hi. Sounds like you use kernel clients with kernels from canonical/ubuntu. Two kernels have a bug: 4.15.0-66 and 5.0.0-32 Updated kernels are said to have fixes. Older kernels also work: 4.15.0-65 and 5.0.0-31 Lars Wed, 30 Oct 2019 09:42:16 + Bob Farrell ==> ceph-users : > Hi. We are experiencing a CephFS client issue on one of our servers. > > ceph version 14.2.0 (3a54b2b6d167d4a2a19e003a705696d4fe619afc) nautilus > (stable) > > Trying to access, `umount`, or `umount -f` a mounted CephFS volumes causes > my shell to hang indefinitely. > > After a reboot I can remount the volumes cleanly but they drop out after < > 1 hour of use. > > I see this log entry multiple times when I reboot the server: > ``` > cache_from_obj: Wrong slab cache. inode_cache but object is from > ceph_inode_info > ``` > The machine then reboots after approx. 30 minutes. > > All other Ceph/CephFS clients and servers seem perfectly happy. CephFS > cluster is HEALTH_OK. > > Any help appreciated. If I can provide any further details please let me > know. > > Thanks in advance, ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] CephFS client hanging and cache issues
Kernel bug due to a bad backport, see recent posts here. Paul -- Paul Emmerich Looking for help with your Ceph cluster? Contact us at https://croit.io croit GmbH Freseniusstr. 31h 81247 München www.croit.io Tel: +49 89 1896585 90 On Wed, Oct 30, 2019 at 10:42 AM Bob Farrell wrote: > > Hi. We are experiencing a CephFS client issue on one of our servers. > > ceph version 14.2.0 (3a54b2b6d167d4a2a19e003a705696d4fe619afc) nautilus > (stable) > > Trying to access, `umount`, or `umount -f` a mounted CephFS volumes causes my > shell to hang indefinitely. > > After a reboot I can remount the volumes cleanly but they drop out after < 1 > hour of use. > > I see this log entry multiple times when I reboot the server: > ``` > cache_from_obj: Wrong slab cache. inode_cache but object is from > ceph_inode_info > ``` > The machine then reboots after approx. 30 minutes. > > All other Ceph/CephFS clients and servers seem perfectly happy. CephFS > cluster is HEALTH_OK. > > Any help appreciated. If I can provide any further details please let me know. > > Thanks in advance, > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] CephFS client hanging and cache issues
Hi. We are experiencing a CephFS client issue on one of our servers. ceph version 14.2.0 (3a54b2b6d167d4a2a19e003a705696d4fe619afc) nautilus (stable) Trying to access, `umount`, or `umount -f` a mounted CephFS volumes causes my shell to hang indefinitely. After a reboot I can remount the volumes cleanly but they drop out after < 1 hour of use. I see this log entry multiple times when I reboot the server: ``` cache_from_obj: Wrong slab cache. inode_cache but object is from ceph_inode_info ``` The machine then reboots after approx. 30 minutes. All other Ceph/CephFS clients and servers seem perfectly happy. CephFS cluster is HEALTH_OK. Any help appreciated. If I can provide any further details please let me know. Thanks in advance, ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
