I'm trying to add a Ceph RBD pool for primary storage use. I've 5 Ceph MONs in my POC lab. Ping and telnet to all the Ceph MONs with port 6789 works.
I'm following the steps from this: - https://docs.ceph.com/en/reef/rbd/rbd-cloudstack/ - https://rohityadav.cloud/blog/ceph/ Agent log when specifying 5 monitors: 2024-08-07 17:12:34,691 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:db5277f2) Attempting to create storage pool eb5ec036-c08a-3d3d-996d-40968077d391 (RBD) in libvirt 2024-08-07 17:12:34,706 WARN [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:db5277f2) Storage pool eb5ec036-c08a-3d3d-996d-40968077d391 was not found running in libvirt. Need to create it. 2024-08-07 17:12:34,706 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:db5277f2) Didn't find an existing storage pool eb5ec036-c08a-3d3d-996d-40968077d391 by UUID, checking for pools with duplicate paths 2024-08-07 17:17:34,738 ERROR [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:db5277f2) Failed to create RBD storage pool: org.libvirt.LibvirtException: failed to connect to the RADOS monitor on: 10.0.32.71,10.0.32.72,10.0.32.73,10.0.32.74,10.0.32.75,: No such file or directory 2024-08-07 17:17:34,739 ERROR [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-3:null) (logid:db5277f2) Failed to create the RBD storage pool, cleaning up the libvirt secret 2024-08-07 17:17:34,739 WARN [cloud.agent.Agent] (agentRequest-Handler-3:null) (logid:db5277f2) Caught: com.cloud.utils.exception.CloudRuntimeException: Failed to create storage pool: eb5ec036-c08a-3d3d-996d-40968077d391 at com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createStoragePool(LibvirtStorageAdaptor.java:743) at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:364) at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:358) at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtModifyStoragePoolCommandWrapper.execute(LibvirtModifyStoragePoolCommandWrapper.java:42) at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtModifyStoragePoolCommandWrapper.execute(LibvirtModifyStoragePoolCommandWrapper.java:35) at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1929) at com.cloud.agent.Agent.processRequest(Agent.java:683) at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1106) at com.cloud.utils.nio.Task.call(Task.java:83) at com.cloud.utils.nio.Task.call(Task.java:29) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Agent log when specifying 1 monitors: 2024-08-07 17:06:09,791 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) (logid:c790784b) Attempting to create storage pool 69b2f6e0-12c8-31a3-bdc6-71b3a1e265f2 (RBD) in libvirt 2024-08-07 17:06:09,806 WARN [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) (logid:c790784b) Storage pool 69b2f6e0-12c8-31a3-bdc6-71b3a1e265f2 was not found running in libvirt. Need to create it. 2024-08-07 17:06:09,806 INFO [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) (logid:c790784b) Didn't find an existing storage pool 69b2f6e0-12c8-31a3-bdc6-71b3a1e265f2 by UUID, checking for pools with duplicate paths 2024-08-07 17:11:09,840 ERROR [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) (logid:c790784b) Failed to create RBD storage pool: org.libvirt.LibvirtException: failed to connect to the RADOS monitor on: 10.0.32.71,: No such file or directory 2024-08-07 17:11:09,840 ERROR [kvm.storage.LibvirtStorageAdaptor] (agentRequest-Handler-2:null) (logid:c790784b) Failed to create the RBD storage pool, cleaning up the libvirt secret 2024-08-07 17:11:09,841 WARN [cloud.agent.Agent] (agentRequest-Handler-2:null) (logid:c790784b) Caught: com.cloud.utils.exception.CloudRuntimeException: Failed to create storage pool: 69b2f6e0-12c8-31a3-bdc6-71b3a1e265f2 at com.cloud.hypervisor.kvm.storage.LibvirtStorageAdaptor.createStoragePool(LibvirtStorageAdaptor.java:743) at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:364) at com.cloud.hypervisor.kvm.storage.KVMStoragePoolManager.createStoragePool(KVMStoragePoolManager.java:358) at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtModifyStoragePoolCommandWrapper.execute(LibvirtModifyStoragePoolCommandWrapper.java:42) at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtModifyStoragePoolCommandWrapper.execute(LibvirtModifyStoragePoolCommandWrapper.java:35) at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1929) at com.cloud.agent.Agent.processRequest(Agent.java:683) at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1106) at com.cloud.utils.nio.Task.call(Task.java:83) at com.cloud.utils.nio.Task.call(Task.java:29) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Ran these commands on my Ceph node but still no luck. I refer this: - https://github.com/apache/cloudstack/issues/5741 ceph config set mon auth_expose_insecure_global_id_reclaim false ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false ceph config set mon auth_allow_insecure_global_id_reclaim false ceph orch restart mon Thanks for the help :)
