Update: I have identified the root cause: user caps are not correct. Errornous caps: root@ld4257:/etc/ceph# ceph auth get client.gbsadm exported keyring for client.gbsadm [client.gbsadm] key = AQBd0klcFknvMRAAwuu30bNG7L7PHk5d8cSVvg== caps mon = "allow r" caps osd = "allow pool backup object_prefix rbd_data.18102d6b8b4567; allow rwx pool backup object_prefix rbd_header.18102d6b8b4567; allow rx pool backup object_prefix rbd_id.gbs"
Correct caps: root@ld4257:/etc/ceph# ceph auth get client.gbsadm exported keyring for client.gbsadm [client.gbsadm] key = AQBd0klcFknvMRAAwuu30bNG7L7PHk5d8cSVvg== caps mon = "allow r" caps osd = "allow rwx pool backup object_prefix rbd_data.18102d6b8b4567; allow rwx pool backup object_prefix rbd_header.18102d6b8b4567; allow rx pool backup object_prefix rbd_id.gbs" The error was caused by failure when copy & paste from Eugen's instructions that are 100% correct! Thanks for your great support!!! Maybe another question related to this topic: If I write a backup into a RBD, will Ceph use single IO stream or multi IO stream on storage side? Regards Thomas ------------------------------------------------------------------------------------------------- Hi, unfortunately it's not working, yet. I have modified user gbsadm: root@ld4257:/etc/ceph# ceph auth get client.gbsadm exported keyring for client.gbsadm [client.gbsadm] key = AQBd0klcFknvMRAAwuu30bNG7L7PHk5d8cSVvg== caps mon = "allow r" caps osd = "allow pool backup object_prefix rbd_data.18102d6b8b4567; allow rwx pool backup object_prefix rbd_header.18102d6b8b4567; allow rx pool backup object_prefix rbd_id.gbs" But mapping fails with same error: ld7581:/etc/ceph # rbd map backup/gbs --user gbsadm -k /etc/ceph/ceph.client.gbsadm.keyring -c /etc/ceph/ceph.conf rbd: sysfs write failed 2019-01-25 13:19:29.158211 7fc629ffb700 -1 librbd::image::OpenRequest: failed to stat v2 image header: (1) Operation not permitted 2019-01-25 13:19:29.158476 7fc6297fa700 -1 librbd::ImageState: 0x55b623a91f70 failed to open image: (1) Operation not permitted rbd: error opening image gbs: (1) Operation not permitted In some cases useful info is found in syslog - try "dmesg | tail". rbd: map failed: (1) Operation not permitted Regards Thomas Am 25.01.2019 um 12:31 schrieb Eugen Block: > You can check all objects of that pool to see if your caps match: > > rados -p backup ls | grep rbd_id > > > Zitat von Eugen Block <[email protected]>: > >>> caps osd = "allow pool backup object_prefix >>> rbd_data.18102d6b8b4567; allow rwx pool backup object_prefix >>> rbd_header.18102d6b8b4567; allow rx pool backup object_prefix >>> rbd_id.rbd-image" >> >> I think your caps are not entirely correct, the part "[...] >> object_prefix rbd_id.rbd-image" should contain the >> actual image name, so in your case it should be "[...] rbd_id.gbs". >> >> Regards, >> Eugen >> >> Zitat von Thomas <[email protected]>: >> >>> Thanks. >>> >>> Unfortunately this is still not working. >>> >>> Here's the info of my image: >>> root@ld4257:/etc/ceph# rbd info backup/gbs >>> rbd image 'gbs': >>> size 500GiB in 128000 objects >>> order 22 (4MiB objects) >>> block_name_prefix: rbd_data.18102d6b8b4567 >>> format: 2 >>> features: layering >>> flags: >>> create_timestamp: Thu Jan 24 16:01:55 2019 >>> >>> And here's the user caps ouput: >>> root@ld4257:/etc/ceph# ceph auth get client.gbsadm >>> exported keyring for client.gbsadm >>> [client.gbsadm] >>> key = AQBd0klcFknvMRAAwuu30bNG7L7PHk5d8cSVvg== >>> caps mon = "allow r" >>> caps osd = "allow pool backup object_prefix >>> rbd_data.18102d6b8b4567; allow rwx pool backup object_prefix >>> rbd_header.18102d6b8b4567; allow rx pool backup object_prefix >>> rbd_id.rbd-image" >>> >>> >>> Trying to map rbd "backup/gbs" now fails with this error; this >>> operation >>> should be permitted: >>> ld7581:/etc/ceph # rbd map backup/gbs --user gbsadm -k >>> /etc/ceph/ceph.client.gbsadm.keyring -c /etc/ceph/ceph.conf >>> rbd: sysfs write failed >>> 2019-01-25 12:15:19.786724 7fe4357fa700 -1 librbd::image::OpenRequest: >>> failed to stat v2 image header: (1) Operation not permitted >>> 2019-01-25 12:15:19.786962 7fe434ff9700 -1 librbd::ImageState: >>> 0x55b6522177f0 failed to open image: (1) Operation not permitted >>> rbd: error opening image gbs: (1) Operation not permitted >>> In some cases useful info is found in syslog - try "dmesg | tail". >>> rbd: map failed: (1) Operation not permitted >>> >>> The same error is shown when I try to map rbd "backup/isa"; this >>> operation must be prohibited: >>> ld7581:/etc/ceph # rbd map backup/isa --user gbsadm -k >>> /etc/ceph/ceph.client.gbsadm.keyring -c /etc/ceph/ceph.conf >>> rbd: sysfs write failed >>> 2019-01-25 12:15:04.850151 7f8041ffb700 -1 librbd::image::OpenRequest: >>> failed to stat v2 image header: (1) Operation not permitted >>> 2019-01-25 12:15:04.850350 7f80417fa700 -1 librbd::ImageState: >>> 0x5643668a5700 failed to open image: (1) Operation not permitted >>> rbd: error opening image isa: (1) Operation not permitted >>> In some cases useful info is found in syslog - try "dmesg | tail". >>> rbd: map failed: (1) Operation not permitted >>> >>> >>> Regards >>> Thomas >>> >>> Am 25.01.2019 um 11:52 schrieb Eugen Block: >>>> osd 'allow rwx >>>> pool <pool> object_prefix rbd_data.2b36cf238e1f29; allow rwx pool >>>> <pool> >>>> object_prefix rbd_header.2b36cf238e1f29 >> >> >> >> _______________________________________________ >> ceph-users mailing list >> [email protected] >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > _______________________________________________ > ceph-users mailing list > [email protected] > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
