cloud you give some guide about complied latest libvirt and qemu ? thanks
2013/8/20 Andrei Mikhailovsky <and...@arhont.com>: > > > Well, I do not think that it is down to 2.6.32 kernel that is causing your > issue as CS and RBD integration doesn't use kernel module. Please correct me > if I am wrong, but I think it uses userspace libraries. Try asking people on > OFTC irc #ceph, there should be people using centos/RH. > > When I tried rbd with cs using centos 6.4 I have compiled the latest libvirt > and qemu from sources and didn't have that many issues apart from heavy load > server reboots. I was running fio with 4 files each threading 16 random > reads/writes with block size of 4K. Larger block sizes didn't give me any > issues. At that time I had to use centos because of the infiniband support > issues which i had with ubuntu, but the latest ofed drivers sorted out the > issue and I am happily back to ubuntu. > > Do you need to use Centos for a reason? Taking into account that Ceph is > heavily developed and tested using Ubuntu servers, I would use it instead of > Centos, unless there are issues with hardware, etc. which do not work in > Ubuntu. > > Andrei > ----- Original Message ----- > > From: "不坏阿峰" <onlydeb...@gmail.com> > To: users@cloudstack.apache.org > Sent: Tuesday, 20 August, 2013 3:18:36 AM > Subject: Re: Fail to add RBD ceph as Primary storage on Cloudstack > 4.1+Centos6.4 KVM host > > thank you for your response. > > i have done Widoh's way too and running well on ubuntu. so i know it > is the centos some kernel and features too old. > Hope have some one make a success deploy on Centos 6.4 > > 2013/8/20 Andrei Mikhailovsky <and...@arhont.com>: >> I've been testing centos 6.4 extensively with rbd and had a lot of stability >> issues where the host server would freeze or reboot during high load >> benchmarks, especially with small block sizes. I have switched to ubuntu >> 12.04 with 3.5 kernel and didn't see this behaviour. Just my two cents. >> >> P.S. i've not had any issues with adding rbd cluster to cloudstack following >> the Widoh's howto on the net. >> >> Andrei >> ----- Original Message ----- >> >> From: "不坏阿峰" <onlydeb...@gmail.com> >> To: users@cloudstack.apache.org >> Sent: Monday, 19 August, 2013 1:40:46 PM >> Subject: Fail to add RBD ceph as Primary storage on Cloudstack 4.1+Centos6.4 >> KVM host >> >> Environment: >> ################## >> Cloudstack 4.1.1, Ceph 0.6.1.7, Qemu 0.12.1.2-2.355 with RBD enable >> >> I have added ceph RBD as primary successfully with ubuntu 12.04 kvm. >> but fail on centos 6.4 kvm host in cloudstack. >> >> i did some test on centos kvm host, i think can use rbd to access ceph >> (have the ceph.conf on KVM host) >> ####################### >> [root@centos-kvm01 ~]# qemu-img -v | grep rbd >> Supported formats: raw cow qcow vdi vmdk cloop dmg bochs vpc vvfat >> qcow2 qed parallels nbd blkdebug host_cdrom host_floppy host_device >> file rbd >> >> [root@centos-kvm01 ~]# rbd -m 192.168.250.15 -p volumes ls >> 08b376f0-5af4-4b46-b41e-d83dc0f93a70 >> 1e9f8d76-94d9-4d56-a8a6-fdff32ba9d1b >> b0f7fae3-716b-4210-a89d-af63c6e30859 >> >> [root@centos-kvm01 ~]# rados lspools >> data >> metadata >> rbd >> volumes >> cloudstack >> >> [root@centos-kvm01 ~]# rbd create test --size 4096 >> [root@centos-kvm01 ~]# rbd ls >> test >> >> cloudstack log: >> #################### >> 2013-08-19 17:28:25,683 DEBUG [agent.transport.Request] >> (AgentManager-Handler-9:null) Seq 26-368640010: Processing: { Ans: , >> MgmtId: 345050143793, via: 26, Ver: v1, Flags: 10, >> [{"Answer":{"result":false,"details":"java.lang.NullPointerException\n\tat >> com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createStoragePool(LibvirtStorageAdaptor.java:540)\n\tat >> com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:111)\n\tat >> com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:104)\n\tat >> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2304)\n\tat >> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1094)\n\tat >> com.cloud.agent.Agent.processRequest(Agent.java:525)\n\tat >> com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852)\n\tat >> com.cloud.utils.nio.Task.run(Task.java:83)\n\tat >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)\n\tat >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\n\tat >> java.lang.Thread.run(Thread.java:679)\n","wait":0}}] } >> 2013-08-19 17:28:25,683 DEBUG [agent.transport.Request] >> (catalina-exec-1:null) Seq 26-368640010: Received: { Ans: , MgmtId: >> 345050143793, via: 26, Ver: v1, Flags: 10, { Answer } } >> 2013-08-19 17:28:25,683 DEBUG [agent.manager.AgentManagerImpl] >> (catalina-exec-1:null) Details from executing class >> com.cloud.agent.api.ModifyStoragePoolCommand: >> java.lang.NullPointerException >> at >> com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createStoragePool(LibvirtStorageAdaptor.java:540) >> at >> com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:111) >> at >> com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:104) >> at >> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2304) >> at >> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1094) >> at com.cloud.agent.Agent.processRequest(Agent.java:525) >> at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852) >> at com.cloud.utils.nio.Task.run(Task.java:83) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:679) >> >> 2013-08-19 17:28:25,689 WARN [cloud.storage.StorageManagerImpl] >> (catalina-exec-1:null) Unable to establish a connection between >> Host[-26-Routing] and Pool[207|RBD] >> com.cloud.exception.StorageUnavailableException: Resource >> [StoragePool:207] is unreachable: Unable establish connection from >> storage head to storage pool 207 due to java.lang.NullPointerException >> at >> com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createStoragePool(LibvirtStorageAdaptor.java:540) >> at >> com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:111) >> at >> com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:104) >> at >> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.execute(LibvirtComputingResource.java:2304) >> at >> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1094) >> at com.cloud.agent.Agent.processRequest(Agent.java:525) >> at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:852) >> at com.cloud.utils.nio.Task.run(Task.java:83) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:679) >> at >> com.cloud.storage.StorageManagerImpl.connectHostToSharedPool(StorageManagerImpl.java:1637) >> at >> com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:1402) >> at >> com.cloud.storage.StorageManagerImpl.createPool(StorageManagerImpl.java:147) >> at >> org.apache.cloudstack.api.command.admin.storage.CreateStoragePoolCmd.execute(CreateStoragePoolCmd.java:123) >> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162) >> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:505) >> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:355) >> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:302) >> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:66) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) >> at >> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) >> at >> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) >> at >> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) >> at >> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) >> at >> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) >> at >> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) >> at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:615) >> at >> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) >> at >> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293) >> at >> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889) >> at >> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:744) >> at >> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2282) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:679) >> 2013-08-19 17:28:25,690 WARN [cloud.storage.StorageManagerImpl] >> (catalina-exec-1:null) No host can access storage pool Pool[207|RBD] >> on cluster 3 >> 2013-08-19 17:28:25,693 INFO [cloud.api.ApiServer] >> (catalina-exec-1:null) Failed to add storage pool >> >