Dear Cephers,
I still can attach volume to my cloud machines, ceph version is 0.94.5
(9764da52395923e0b32908d83a9f7304401fee43) and Openstack Juno
Nova+cinder are able to create volumes on Ceph
cephvolume:~ # rados ls --pool volumes
rbd_header.1f7784a9e1c2e
rbd_id.volume-5e2ab5c2-4710-4c28-9755-b5bc4ff6a52a
rbd_directory
rbd_id.volume-7da08f12-fb0f-4269-931a-d528c1507fee
rbd_header.23d5e33b4c15c
rbd_id.volume-4d26bb31-91e8-4646-8010-82127b775c8e
rbd_header.20407190ce77f
cloud:~ # cinder list
+--------------------------------------+--------+--------------+------+-------------+----------+------------------------------------------------------------------------------------------+
| ID |
Status | Display Name | Size | Volume Type | Bootable |
Attached to |
+--------------------------------------+--------+--------------+------+-------------+----------+--------------------------------|---------------------------------------------------------+
| 4d26bb31-91e8-4646-8010-82127b775c8e | in-use | None | 2
| rbd | false |
59aa021e-bb4c-4154-9b18-9d09f5fd3aeb |
+--------------------------------------+--------+--------------+------+-------------+----------+------------------------------------------------------------------------------------------+
nova:~ # nova volume-attach 59aa021e-bb4c-4154-9b18-9d09f5fd3aeb
4d26bb31-91e8-4646-8010-82127b775c8e auto
+----------+------------------------------------------------------------+
| Property | Value
|
+----------+------------------------------------------------------------+
| device | /dev/xvdd
|
| id | 4d26bb31-91e8-4646-8010-82127b775c8e |
| serverId | 59aa021e-bb4c-4154-9b18-9d09f5fd3aeb |
| volumeId | 4d26bb31-91e8-4646-8010-82127b775c8e |
+----------+--------------------------------------+
>From nova-compute (Ubuntu 14.04 LTS \n \l) node I see the
attaching/detaching:
cloud01:~ # dpkg -l | grep ceph
ii ceph-common 0.94.5-1trusty
amd64 common utilities to mount and interact with a ceph storage
cluster
ii libcephfs1 0.94.5-1trusty
amd64 Ceph distributed file system client library
ii python-cephfs 0.94.5-1trusty
amd64 Python libraries for the Ceph libcephfs library
ii librbd1 0.94.5-1trusty
amd64 RADOS block device client library
ii python-rbd 0.94.5-1trusty
amd64 Python libraries for the Ceph librbd library
*at cinder.conf*
*rbd_user = cinder*
*rbd_secret_uuid = 67a6d4a1-e53a-42c7-9bc9-xxxxxxxxxxx*
*[rbd-cephvolume]*
*volume_backend_name=rbd*
*volume_driver = cinder.volume.drivers.rbd.RBDDriver*
*rbd_pool = volumes*
*rbd_ceph_conf = /etc/ceph/ceph.conf*
*rbd_flatten_volume_from_snapshot = false*
*rbd_max_clone_depth = 5*
*rbd_store_chunk_size = 4*
*rados_connect_timeout = -1*
*glance_api_version = 2*
*in nova.conf*
*rbd_user=cinder*
*# The libvirt UUID of the secret for the rbd_uservolumes*
*# (string value)*
*rbd_secret_uuid=67a6d4a1-e53a-42c7-9bc9-xxxxxxxxxxx*
*images_rbd_pool=volumes# Path to the ceph configuration file to use
(string value)images_rbd_ceph_conf=/etc/ceph/ceph.conf*
*ls -la /etc/libvirt/secrets*
*total 16*
*drwx------ 2 root root 4096 Nov 4 10:28 .*
*drwxr-xr-x 7 root root 4096 Oct 22 13:15 ..*
*-rw------- 1 root root 40 Nov 4 10:28
67a6d4a1-e53a-42c7-9bc9-xxxxxxxxx.base64*
*-rw------- 1 root root 170 Nov 4 10:25
67a6d4a1-e53a-42c7-9bc9-xxxxxxxxxx.xml*
2015-11-04 10:39:42.573 11653 INFO nova.compute.manager
[req-8b2a9793-4b39-4cb0-b291-e492c350387e b7aadbb4a85745feb498b74e437129cc
ce2dd2951bd24c1ea3b43c3b3716f604 - - -] [instance:
59aa021e-bb4c-4154-9b18-9d09f5fd3aeb] Detach volume
4d26bb31-91e8-4646-8010-82127b775c8e from mountpoint /dev/xvdd
2015-11-04 10:40:43.266 11653 INFO nova.compute.manager
[req-35218de0-3f26-496b-aad9-5c839143da17 b7aadbb4a85745feb498b74e437129cc
ce2dd2951bd24c1ea3b43c3b3716f604 - - -] [instance:
59aa021e-bb4c-4154-9b18-9d09f5fd3aeb] Attaching volume
4d26bb31-91e8-4646-8010-82127b775c8e to /dev/xvdd
but one on cloud machine (SL6) the volume y never showed (xvdd).
[root@cloud5 ~]# cat /proc/partitions
major minor #blocks name
202 0 20971520 xvda
202 16 209715200 xvdb
202 32 10485760 xvdc
Thanks in advance, I
2015-11-03 11:18 GMT+01:00 Iban Cabrillo <[email protected]>:
> Hi all,
> During last week I been trying to deploy the pre-existing ceph cluster
> with out openstack intance.
> The ceph-cinder integration was easy (or at least I think so!!)
> There is only one volume to attach block storage to out cloud machines.
>
> The client.cinder has permission on this volume (following the guides)
> ...............
> client.cinder
> key: AQAonXXXXXXXRAAPIAj9iErv001a0k+vyFdUg==
> caps: [mon] allow r
> *caps: [osd] allow class-read object_prefix rbd_children, allow rwx
> pool=volumes*
>
> ceph.conf file seems to be OK:
>
> *[global]*
> *fsid = 6f5a65a7-316c-4825-afcb-428608941dd1*
> *mon_initial_members = cephadm, cephmon02, cephmon03*
> *mon_host = 10.10.3.1,10.10.3.2,10.10.3.3*
> *auth_cluster_required = cephx*
> *auth_service_required = cephx*
> *auth_client_required = cephx*
> *filestore_xattr_use_omap = true*
> *osd_pool_default_size = 2*
> *public_network = 10.10.0.0/16 <http://10.10.0.0/16>*
> *cluster_network = 192.168.254.0/27 <http://192.168.254.0/27>*
>
> *[osd]*
> *osd_journal_size = 20000*
>
> *[client.cinder]*
> *keyring = /etc/ceph/ceph.client.cinder.keyring*
>
> *[client]*
> *rbd cache = true*
> *rbd cache writethrough until flush = true*
> *admin socket = /var/run/ceph/$cluster-$type.$id.$pid.$cctid.asok*
>
>
> The trouble seems that blocks are created using the client.admin instead
> of client.cinder
>
> From cinder machine:
>
> cinder:~ # rados ls --pool volumes
> rbd_id.volume-5e2ab5c2-4710-4c28-9755-b5bc4ff6a52a
> rbd_directory
> rbd_id.volume-7da08f12-fb0f-4269-931a-d528c1507fee
> rbd_header.23d5e33b4c15c
> rbd_header.20407190ce77f
>
> But if I try to look for using cinder client:
>
>
> cinder:~ #rados ls --pool volumes --secret client.cinder
> "empty answer"
>
> cinder:~ # ls -la /etc/ceph
> total 24
> drwxr-xr-x 2 root root 4096 nov 3 10:17 .
> drwxr-xr-x 108 root root 4096 oct 29 09:52 ..
> -rw------- 1 root root 63 nov 3 10:17 ceph.client.admin.keyring
> -rw-r--r-- 1 cinder cinder 67 oct 28 13:44 ceph.client.cinder.keyring
> -rw-r--r-- 1 root root 454 oct 1 13:56 ceph.conf
> -rw-r--r-- 1 root root 73 sep 27 09:36 ceph.mon.keyring
>
>
> from a client (I have supposed that this machine only need the cinder
> key...)
>
> cloud28:~ # ls -la /etc/ceph/
> total 28
> drwx------ 2 root root 4096 nov 3 11:01 .
> drwxr-xr-x 116 root root 12288 oct 30 14:37 ..
> -rw-r--r-- 1 nova nova 67 oct 28 11:43 ceph.client.cinder.keyring
> -rw-r--r-- 1 root root 588 nov 3 10:59 ceph.conf
> -rw-r--r-- 1 root root 92 oct 26 16:59 rbdmap
>
> cloud28:~ # rbd -p volumes ls
> 2015-11-03 11:01:58.782795 7fc6c714b840 -1 monclient(hunting): ERROR:
> missing keyring, cannot use cephx for authentication
> 2015-11-03 11:01:58.782800 7fc6c714b840 0 librados: client.admin
> initialization error (2) No such file or directory
> rbd: couldn't connect to the cluster!
>
> Any help will be welcome.
>
>
--
############################################################################
Iban Cabrillo Bartolome
Instituto de Fisica de Cantabria (IFCA)
Santander, Spain
Tel: +34942200969
PGP PUBLIC KEY:
http://pgp.mit.edu/pks/lookup?op=get&search=0xD9DF0B3D6C8C08AC
############################################################################
Bertrand Russell:
*"El problema con el mundo es que los estúpidos están seguros de todo y los
inteligentes están llenos de dudas*"
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
