Hi,
we want upgrade our ceph clusters from jewel to luminous. And also want
to migrate the osds to ceph-volume described in
http://docs.ceph.com/docs/luminous/ceph-volume/simple/scan/#ceph-volume-simple-scan
The clusters are running since dumpling and are setup with dmcrypt.
Since dumpling there are until now three different types of dmcrypt
plain dmcrypt with keys local
luks with keys local
luks with keys on the ceph monitors
Now it seems only the last type can be migrated to ceph-volume.
ceph-volume simple scan trys to mount a lockbox which does not exists
on the older OSDs. Are those OSDs not supported with ceph-volume?
This are the errors:
# ceph-volume simple scan /var/lib/ceph/osd/ceph-183
stderr: lsblk: /var/lib/ceph/osd/ceph-183: not a block device
stderr: lsblk: /var/lib/ceph/osd/ceph-183: not a block device
Running command: /usr/sbin/cryptsetup status
/dev/mapper/21ad7722-002f-464c-b460-a8976a7b4872
Running command: /usr/sbin/cryptsetup status
21ad7722-002f-464c-b460-a8976a7b4872
Running command: mount -v /tmp/tmp3t1WRC
stderr: mount: is write-protected, mounting read-only
stderr: mount: unknown filesystem type '(null)'
--> RuntimeError: command returned non-zero exit status: 32
and this is in the ceph-volume.log
[2019-01-22 12:39:31,456][ceph_volume.process][INFO ] Running command:
/usr/sbin/blkid -p /dev/mapper/9b68b7e9-854e-498a-8381-4eef128a9d7a
[2019-01-22 12:39:31,533][ceph_volume.devices.simple.scan][INFO ] detecting if
argument is a device or a directory: /var/lib/ceph/osd/ceph-183
[2019-01-22 12:39:31,533][ceph_volume.devices.simple.scan][INFO ] will scan
directly, path is a directory
[2019-01-22 12:39:31,533][ceph_volume.devices.simple.scan][INFO ] will scan
encrypted OSD directory at path: /var/lib/ceph/osd/ceph-183
[2019-01-22 12:39:31,534][ceph_volume.process][INFO ] Running command:
/usr/sbin/blkid -s PARTUUID -o value /dev/sdv1
[2019-01-22 12:39:31,539][ceph_volume.process][INFO ] stdout
21ad7722-002f-464c-b460-a8976a7b4872
[2019-01-22 12:39:31,540][ceph_volume.process][INFO ] Running command:
/usr/sbin/cryptsetup status 21ad7722-002f-464c-b460-a8976a7b4872
[2019-01-22 12:39:31,546][ceph_volume.process][INFO ] stdout
/dev/mapper/21ad7722-002f-464c-b460-a8976a7b4872 is active and is in use.
[2019-01-22 12:39:31,547][ceph_volume.process][INFO ] stdout type: PLAIN
[2019-01-22 12:39:31,547][ceph_volume.process][INFO ] stdout cipher:
aes-cbc-essiv:sha256
[2019-01-22 12:39:31,547][ceph_volume.process][INFO ] stdout keysize: 256 bits
[2019-01-22 12:39:31,547][ceph_volume.process][INFO ] stdout key location:
dm-crypt
[2019-01-22 12:39:31,547][ceph_volume.process][INFO ] stdout device: /dev/sdv1
[2019-01-22 12:39:31,547][ceph_volume.process][INFO ] stdout sector size: 512
[2019-01-22 12:39:31,547][ceph_volume.process][INFO ] stdout offset: 0 sectors
[2019-01-22 12:39:31,547][ceph_volume.process][INFO ] stdout size:
7805646479 sectors
[2019-01-22 12:39:31,547][ceph_volume.process][INFO ] stdout mode:
read/write
[2019-01-22 12:39:31,548][ceph_volume.process][INFO ] Running command: mount
-v /tmp/tmp3t1WRC
[2019-01-22 12:39:31,597][ceph_volume.process][INFO ] stderr mount: is
write-protected, mounting read-only
[2019-01-22 12:39:31,622][ceph_volume.process][INFO ] stderr mount: unknown
filesystem type '(null)'
[2019-01-22 12:39:31,622][ceph_volume][ERROR ] exception caught by decorator
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ceph_volume/decorators.py", line 59,
in newfunc
return f(*a, **kw)
File "/usr/lib/python2.7/site-packages/ceph_volume/main.py", line 148, in main
terminal.dispatch(self.mapper, subcommand_args)
File "/usr/lib/python2.7/site-packages/ceph_volume/terminal.py", line 182, in
dispatch
instance.main()
File "/usr/lib/python2.7/site-packages/ceph_volume/devices/simple/main.py",
line 33, in main
terminal.dispatch(self.mapper, self.argv)
File "/usr/lib/python2.7/site-packages/ceph_volume/terminal.py", line 182, in
dispatch
instance.main()
File "/usr/lib/python2.7/site-packages/ceph_volume/devices/simple/scan.py",
line 353, in main
self.scan(args)
File "/usr/lib/python2.7/site-packages/ceph_volume/decorators.py", line 16,
in is_root
return func(*a, **kw)
File "/usr/lib/python2.7/site-packages/ceph_volume/devices/simple/scan.py",
line 244, in scan
osd_metadata = self.scan_encrypted(osd_path)
File "/usr/lib/python2.7/site-packages/ceph_volume/devices/simple/scan.py",
line 169, in scan_encrypted
with system.tmp_mount(lockbox) as lockbox_path:
File "/usr/lib/python2.7/site-packages/ceph_volume/util/system.py", line 145,
in __enter__
self.path
File "/usr/lib/python2.7/site-packages/ceph_volume/process.py", line 153, in
run
raise RuntimeError(msg)
RuntimeError: command returned non-zero exit status: 32
ceph version 12.2.10 (177915764b752804194937482a39e95e0ca3de94)
luminous (stable)
Regards
Manuel
--
Manuel Lausch
Systemadministrator
Storage Services
1&1 Mail & Media Development & Technology GmbH | Brauerstraße 48 | 76135
Karlsruhe | Germany
Phone: +49 721 91374-1847
E-Mail: [email protected] | Web: www.1und1.de
Hauptsitz Montabaur, Amtsgericht Montabaur, HRB 5452
Geschäftsführer: Thomas Ludwig, Jan Oetjen, Sascha Vollmer
Member of United Internet
Diese E-Mail kann vertrauliche und/oder gesetzlich geschützte Informationen
enthalten. Wenn Sie nicht der bestimmungsgemäße Adressat sind oder diese E-Mail
irrtümlich erhalten haben, unterrichten Sie bitte den Absender und vernichten
Sie diese E-Mail. Anderen als dem bestimmungsgemäßen Adressaten ist untersagt,
diese E-Mail zu speichern, weiterzuleiten oder ihren Inhalt auf welche Weise
auch immer zu verwenden.
This e-mail may contain confidential and/or privileged information. If you are
not the intended recipient of this e-mail, you are hereby notified that saving,
distribution or use of the content of this e-mail in any way is prohibited. If
you have received this e-mail in error, please notify the sender and delete the
e-mail.
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
