On Fri, Sep 6, 2019 at 7:33 AM Vang Le-Quy <[email protected]> wrote: > Hi Paul > > Thank you for suggestion. > > Firstly, a correction “I am suspecting that there is a bug in > 12.2.8-0ubuntu0.18.04.1” > is incorrect. I meant that version 12.2.8-0ubuntu0.18.04.1 on node N01 > works, . 12.2.12-0ubuntu0.18.04.2 is having problem. > > > > After upgrading to version 12.2.12-0ubuntu0.18.04.2, it doesn’t work > anymore. No changes on client keys file or on the Ceph cluster. Therefore > I deduce that it is a problem with client (ceph-common) software. >
Looks like a CephX caps issue for your client. Please refer to the documentation and verify you are using the correct caps [1]. > > > ~ Vang > > > > *From: *Paul Emmerich <[email protected]> > *Date: *Friday, 6 September 2019 at 13.13 > *To: *Vang Le-Quy <[email protected]> > *Cc: *"[email protected]" <[email protected]> > *Subject: *Re: [ceph-users] Ceph client failed to mount RBD device after > reboot > > > > failure to break locks is usually due to incorrect permissions on the key, > check if the client key has "profile rbd" mon permissions > > > > -- > Paul Emmerich > > Looking for help with your Ceph cluster? Contact us at https://croit.io > > croit GmbH > Freseniusstr. 31h > 81247 München > www.croit.io > Tel: +49 89 1896585 90 > > > > > > On Fri, Sep 6, 2019 at 1:06 PM Vang Le-Quy <[email protected]> wrote: > > I am suspecting that there is a bug in 12.2.8-0ubuntu0.18.04.1. We have > too ceph client n01 and n03 with following software versions. > > > > # n01 ceph-common: > > Installed: 12.2.8-0ubuntu0.18.04.1 > > Candidate: 12.2.12-0ubuntu0.18.04.2 > > > > # n03 ceph-common: > > Installed: 12.2.11-0ubuntu0.18.04.2 > > Candidate: 12.2.12-0ubuntu0.18.04.2 > > > > # n01 uname -a > > Linux n01 4.15.0-43-generic #46-Ubuntu SMP Thu Dec 6 14:45:28 UTC 2018 > x86_64 x86_64 x86_64 GNU/Linux > > > > # n03 uname -a > > Linux n03 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 > x86_64 x86_64 x86_64 GNU/Linux > > > > > > Problem: > > N01 can mount its RBD after reboot whereas N03 can’t. To verify that it is > something on client side, I upgrade n01 to have the same package version to > n03 including ceph-common. And n01 does indeed fail to mount its CEPH RBD > partition. Syslog shows messages like the following: > > Sep 6 12:52:33 n01 kernel: [ 875.160275] rbd: rbd0: no lock owners > detected > > Sep 6 12:52:33 n01 kernel: [ 875.161069] rbd: rbd0: client2508785 seems > dead, breaking lock > > Sep 6 12:52:33 n01 kernel: [ 875.161723] rbd: rbd0: blacklist of > client2508785 failed: -13 > > Sep 6 12:52:33 n01 kernel: [ 875.161724] rbd: rbd0: failed to acquire > lock: -13 > > Sep 6 12:52:33 n01 kernel: [ 875.161956] rbd: rbd0: no lock owners > detected > > Sep 6 12:52:33 n01 kernel: [ 875.162755] rbd: rbd0: client2508785 seems > dead, breaking lock > > Sep 6 12:52:33 n01 kernel: [ 875.163415] rbd: rbd0: blacklist of > client2508785 failed: -13 > > Sep 6 12:52:33 n01 kernel: [ 875.163415] rbd: rbd0: failed to acquire > lock: -13 > > Sep 6 12:52:33 n01 kernel: [ 875.163645] rbd: rbd0: no lock owners > detected > > Sep 6 12:52:33 n01 kernel: [ 875.164431] rbd: rbd0: client2508785 seems > dead, breaking lock > > Sep 6 12:52:33 n01 kernel: [ 875.165082] rbd: rbd0: blacklist of > client2508785 failed: -13 > > Sep 6 12:52:33 n01 kernel: [ 875.165082] rbd: rbd0: failed to acquire > lock: -13 > > Sep 6 12:52:33 n01 kernel: [ 875.165355] rbd: rbd0: no lock owners > detected > > > > > > > > [image: signature_1482564062] > Vang Quy Le > Special Consultant in Data Science and Infrastructure > > T: (+45) 9940 7710 | Email: [email protected] <[email protected]> > Kontor 0-1-91 | Selma Lagerløfs Vej 300 | DK-9220 Aalborg Ø | > > > > _______________________________________________ > ceph-users mailing list -- [email protected] > To unsubscribe send an email to [email protected] > > _______________________________________________ > ceph-users mailing list -- [email protected] > To unsubscribe send an email to [email protected] > [1] https://docs.ceph.com/docs/luminous/rbd/rados-rbd-cmds/#create-a-block-device-user -- Jason
_______________________________________________ ceph-users mailing list -- [email protected] To unsubscribe send an email to [email protected]
