Hi All,
This seems to be a weird issue. Firstly all deployment is done with
"ceph-deploy" and 3 host machines acting as MON and OSD using the Hammer
release on Ubuntu 14.04.3 and running KVM (libvirt).
When using vanilla CEPH, single rbd pool no log device or cache tiering, the
virtual machine will start without any problem. I can see CEPH doing data work
and the virtual machine runs the OS installer fine.
However when I enable cache tiering so that I have a separate RBD and SSD pool,
libvirt is unable to start and virtual machines at all with an error of a
feature I think CEPH disabled (not sure). This is entirely repeatable as I did
re-installs of software and even the operating system.
I'm not sure if it is CEPH in this case but it only seems to happen when doing
cache tiering which leads me to believe it has some of the blame.
My setup is:
3 Host Servers (doing both KVM and CEPH)
Distributor ID: Ubuntu
Description: Ubuntu 14.04.3 LTS
Release: 14.04
Codename: trusty
Each server has 2x OSD devices with a total of 6 across the cluster. 1x 500GB
HDD and 1x 120GB SSD in each host. All JBOD. (Operating system disks excluded
from this)
My error received from libvirt: (libvirt and ceph secrets omitted)
internal error: process exited while connecting to monitor:
libust[19895/19895]: Warning: HOME environment variable not set. Disabling
LTTng-UST per-user tracing. (in setup_local_apps() at lttng-ust-comm.c:305)
libust[19895/19896]: Error: Error opening shm /lttng-ust-wait-5 (in
get_wait_shm() at lttng-ust-comm.c:886) libust[19895/19896]: Error: Error
opening shm /lttng-ust-wait-5 (in get_wait_shm() at lttng-ust-comm.c:886)
qemu-system-x86_64: -drive
file=rbd:rbd/osc-maas-1:id=libvirt:key=XXXXXXXXXXXXXX:auth_supported=cephx\;none:mon_host=osc-mgmt-1\:6789\;osc-mgmt-2\:6789\;osc-mgmt-3\:6789,if=none,id=drive-virtio-disk0,format=raw:
error reading header from osc-maas-1 qemu-system-x86_64: -drive
file=rbd:rbd/osc-maas-1:id=libvirt:key=XXXXXXXXXXXXXX:auth_supported=cephx\;none:mon_host=osc-mgmt-1\:6789\;osc-mgmt-2\:6789\;osc-mgmt-3\:6789,if=none,id=drive-virtio-disk0,format=raw:
could not open disk image rbd:rbd/osc-maas-1:id=libvirt:key=XXXXXXXXXXXXXX
I have followed these guides to split pools onto SSD and do cache tiering.
http://www.sebastien-han.fr/blog/2014/08/25/ceph-mix-sata-and-ssd-within-the-same-box/
http://ceph.com/docs/master/rados/operations/cache-tiering/
Thanks
Pieter
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
