Did you compile libvirt and qemu with RBD support? Centos6.5 ( libvitt and qemu) does not support RBD/ceph as storage out of the box... On Sep 14, 2015 7:37 AM, "Shetty, Pradeep" <[email protected]> wrote:
> Hi, > > Is Ceph 0.94 Hammer compatible/stable with CentOS 6.5 build and cloudstack > version 4.4.2 > > Thanks, > Pradeep > > -----Original Message----- > From: Andrija Panic [mailto:[email protected]] > Sent: 11 September 2015 20:42 > To: Vadim Kimlaychuk > Cc: [email protected] > Subject: Re: Unable to add Ceph KVM node in cloudstack > > Dont really know that to be honest... even previously, you can patch qemu > source code yourslef with RBD support - nothing stops you to do so - so > this is just decision from RH or whatever - Im just guessing - anyway it is > the way it is - ubuntu oit of the box suport, CentOS not afaik... > > On 11 September 2015 at 13:56, Vadim Kimlaychuk <[email protected]> > wrote: > > > Andrija, > > > > If RBD libvirt driver now belongs to RH does that mean > > Ubuntu/Debian will loose it some day as it is proprietary software? > > Haven't follow Ceph achievements since "Emperor" > > > > Vadim. > > > > > > On 2015-09-11 14:39, Andrija Panic wrote: > > > > Np, Im actually very disapointed by RH decision to have this approach > > - > >> since now CEPH/Inktank is part of RH etc... > >> I didnt want to play with Ubuntu at all (dont let me start about all > >> the possible issues/bugs we found....) but I dont like idea of > >> compiling stuff on production... > >> > >> On 11 September 2015 at 13:34, Vadim Kimlaychuk <[email protected]> > >> wrote: > >> > >> Andrija, > >> > >> I don't have such broad experience with different CentOS/RHEL > >> versions, but I am a bit surprised that these distributions does not > >> have RBD support at all. That means they have to maintain their own > >> "restricted" > >> version of these libraries apart from mainstream. Now I am even more > >> happy with my Debian/Ubuntu installations :) > >> > >> Thank you for information! > >> > >> Vadim. > >> > >> On 2015-09-11 14:20, Andrija Panic wrote: > >> > >> Vadim, > >> afaik you are right for mgmt server. As for KVM nodes - libvirt and > >> qemu/qemu-img in CentOS6.x does NOT have RBD support at all :) I have > >> not played a lot with CentOS7, but did quick check long time ago and > >> I thing it also doesnt support RBD - Red Hat wants you to buy RHEV to > >> support RBD - of course, there are workarround, self-compile libvirt > >> etc...similar for qemu/qemu-img. > >> > >> I know for sure that Ubuntu 14.x suports RBD out of the box - since > >> we are running it with CEPH. > >> > >> Best > >> > >> On 11 September 2015 at 10:47, Vadim Kimlaychuk <[email protected]> > >> wrote: > >> > >> Andrija, > >> > >> Are you talking about CS Management server OS? I thought it does not > >> really matter if it includes libvirt or not. All calls to storage > >> should go through hypervisor and any KVM host goes with libvirt. The > >> version of libvirt for old OS may not be aligned with latest Ceph > >> release that is true. Cloud architect must check compatibility > >> version matrix for all products Ceph/KVM/CS anyway. > >> > >> Vadim. > >> > >> On 2015-09-11 11:24, Andrija Panic wrote: > >> > >> RBD for KVM comes out of the box for Ubuntu 14.x (perhaps also 12.x) > >> - not with CentOS6, and not sure really about CentOS7 (probably not > >> also...) > >> > >> On 11 September 2015 at 09:53, Vadim Kimlaychuk <[email protected]> > >> wrote: > >> > >> For KVM hosts RBD comes "out of the box" with libvirt. For XenServer > >> Ceph > >> (RBD) storage type is not yet implemented. I would say it is very > >> hypervisor-specific and does not depend much from Cloudstack. > >> > >> Regards, > >> > >> Vadim. > >> > >> On 2015-09-11 08:55, Shetty, Pradeep wrote: > >> > >> I have one question: > >> Is ceph rbd protocol supported by default on libvirt version > >> 0.10.2-54.el6.x86_64 OR do we need to compile libvirt to support rbd > >> within cloudstack. > >> > >> Regards, > >> Pradeep > >> > >> -----Original Message----- > >> From: Vadim Kimlaychuk [mailto:[email protected]] > >> Sent: 09 September 2015 12:07 > >> To: [email protected] > >> Cc: Shetty, Pradeep > >> Subject: RE: Unable to add Ceph KVM node in cloudstack > >> > >> Pradeep, > >> > >> I have installed CS 4.2 with Ceph storage successfully around year ago. > >> There were no problem with configuration except the fact that you > >> can't configure RBD storage as primary storage during Zone set-up. > >> It must be NFS. > >> After Zone is set-up and running you may add another primary storage > >> and there you have list of choices where RBD type is present. > >> After RBD is added you may assign this as default storage for > >> particular cluster assuming that each cluster has its own primary > storage. > >> > >> PS. I have also used Ceph object gateway to connect secondary > >> storage. It works as expected. It would be nice to know which way are > >> you going to integrate Ceph into CS? > >> > >> Vadim. > >> > >> On 2015-09-09 08:09, Shetty, Pradeep wrote: > >> > >> Please share steps for adding ceph RBD as primary storage using KVM > >> hypervisor in cloudstack. > >> > >> We are well aware of the best practices/recommendations for cloudstack. > >> > >> Regards, > >> Pradeep > >> > >> -----Original Message----- > >> From: Vadim Kimlaychuk [mailto:[email protected]] > >> Sent: 08 September 2015 12:14 > >> To: [email protected] > >> Subject: RE: Unable to add Ceph KVM node in cloudstack > >> > >> Pradeep, > >> > >> From documentation : > >> Warning -- It is NOT recommended to run services on this host not > >> controlled by CloudStack. > >> > >> Good practice -- KVM host must be virgin-clean. > >> > >> Vadim. > >> > >> On 2015-09-08 09:14, Shetty, Pradeep wrote: > >> > >> Hello Vadim, > >> > >> We have followed the same guide and using OVS version 2.3.2 . > >> > >> Ceph storage cluster is running on same KVM hypervisor and it fails > >> when adding to cloudstack. > >> > >> Regards, > >> Pradeep > >> > >> -----Original Message----- > >> From: Vadim Kimlaychuk [mailto:[email protected]] > >> Sent: 08 September 2015 11:34 > >> To: [email protected] > >> Cc: Shetty, Pradeep > >> Subject: Re: Unable to add Ceph KVM node in cloudstack > >> > >> Hello Shetty, > >> > >> It seems you agent or libvirt configuration is wrong. Please, specify > >> also do you use OVS or Linux bridge? > >> Did you follow this guide? > >> http://cloudstack-installation.readthedocs.org/en/latest/hypervisor/k > >> v > >> [1] > >> [1 [1]] > >> [1 [1]] > >> [1 [1]] > >> [1 [1]] > >> m.html > >> [1 [2]] > >> > >> Regards, > >> > >> Vadim > >> > >> On 2015-09-08 08:18, Shetty, Pradeep wrote: > >> > >> Hello Folks, > >> > >> We are working on ceph(hammer, 0.94.3) implementation on kvm > >> hypervisor(centos 6.5) which will be integrated on cloudstack(4.4.2), > >> > >> --- Single KVM node( OS- CentOS 6.5, libvirt-0.10.2-54.el6.x86_64, > >> openvswitch 2.3.2) + ceph hammer, cluster health is Ok. > >> > >> We are unable to add the KVM host(new cluster) in cloudstack in an > >> existing zone, below is the error log. > >> > >> ----> agent log-- libvirt: XML-RPC error : Cannot write data: Broken > >> pipe > >> 03/09/2015 13:13:15 4882 jsvc.exec error: Service exit with a return > >> value of 143 > >> > >> Please let us know the exact steps to add Ceph KVM host to cloudstack. > >> > >> Thanks, > >> Pradeep > >> > >> Links: > >> ------ > >> > > > > > > > -- > > Andrija Panić >
