GitHub user poli1025 added a comment to the discussion: Degraded cloudstack agent
We’ve identified that the issue is specifically related to the agent — there is no noticeable delay in the API calls. We haven’t found any errors in the logs, but we are observing that the agent is progressively taking longer to perform the required actions. The bulk operations are being executed via Terraform, such as the creation of 50 machines. The next day, when we destroy and recreate them, the process takes significantly more time. However, if we restart the agent, the performance returns to normal. Over time, the degradation happens again, and the creation time can eventually triple. `2025-07-28 22:21:59,724 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-51:[]) (logid:) Trying to fetch storage pool 0f79bc6b-d365-3005-a4b7-8a41d336c6c6 from libvirt 2025-07-28 22:21:59,726 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-51:[]) (logid:) Asking libvirt to refresh storage pool 0f79bc6b-d365-3005-a4b7-8a41d336c6c6 2025-07-28 22:23:00,459 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-62:[]) (logid:) Trying to fetch storage pool 0f79bc6b-d365-3005-a4b7-8a41d336c6c6 from libvirt 2025-07-28 22:23:00,460 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-62:[]) (logid:) Asking libvirt to refresh storage pool 0f79bc6b-d365-3005-a4b7-8a41d336c6c6 2025-07-28 22:23:00,863 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-60:[]) (logid:) Trying to fetch storage pool 00b05d76-e83c-377e-9005-32f9a1d18679 from libvirt 2025-07-28 22:23:00,864 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-60:[]) (logid:) Asking libvirt to refresh storage pool 00b05d76-e83c-377e-9005-32f9a1d18679 2025-07-28 22:23:00,918 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-65:[]) (logid:) Trying to fetch storage pool e65cc751-30c7-3404-8932-d9d0c1577411 from libvirt 2025-07-28 22:23:00,919 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-65:[]) (logid:) Asking libvirt to refresh storage pool e65cc751-30c7-3404-8932-d9d0c1577411 2025-07-28 22:23:01,420 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-64:[]) (logid:) Trying to fetch storage pool 6aa94ea7-c6a7-344c-a11f-9a00654e1c23 from libvirt 2025-07-28 22:23:01,421 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-64:[]) (logid:) Asking libvirt to refresh storage pool 6aa94ea7-c6a7-344c-a11f-9a00654e1c23 2025-07-28 22:23:01,824 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-67:[]) (logid:) Trying to fetch storage pool 149ab1fd-3757-3bde-b359-60f522b8a17c from libvirt 2025-07-28 22:23:01,825 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-67:[]) (logid:) Asking libvirt to refresh storage pool 149ab1fd-3757-3bde-b359-60f522b8a17c 2025-07-28 22:24:01,665 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-55:[]) (logid:) Trying to fetch storage pool 00b05d76-e83c-377e-9005-32f9a1d18679 from libvirt 2025-07-28 22:24:01,667 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-55:[]) (logid:) Asking libvirt to refresh storage pool 00b05d76-e83c-377e-9005-32f9a1d18679 2025-07-28 22:24:02,026 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-68:[]) (logid:) Trying to fetch storage pool 0f79bc6b-d365-3005-a4b7-8a41d336c6c6 from libvirt 2025-07-28 22:24:02,028 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-68:[]) (logid:) Asking libvirt to refresh storage pool 0f79bc6b-d365-3005-a4b7-8a41d336c6c6 2025-07-28 22:24:02,092 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-75:[]) (logid:) Trying to fetch storage pool 5d8d05bd-451b-33be-ab65-b9d8e3f02528 from libvirt 2025-07-28 22:24:02,094 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-75:[]) (logid:) Asking libvirt to refresh storage pool 5d8d05bd-451b-33be-ab65-b9d8e3f02528 2025-07-28 22:24:02,205 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-61:[]) (logid:) Trying to fetch storage pool 6aa94ea7-c6a7-344c-a11f-9a00654e1c23 from libvirt 2025-07-28 22:24:02,207 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-61:[]) (logid:) Asking libvirt to refresh storage pool 6aa94ea7-c6a7-344c-a11f-9a00654e1c23 2025-07-28 22:24:02,312 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-70:[]) (logid:) Trying to fetch storage pool cacc288a-c318-3b0e-858d-1293480acec6 from libvirt 2025-07-28 22:24:02,313 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-70:[]) (logid:) Asking libvirt to refresh storage pool cacc288a-c318-3b0e-858d-1293480acec6 2025-07-28 22:24:02,492 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-72:[]) (logid:) Trying to fetch storage pool 00b05d76-e83c-377e-9005-32f9a1d18679 from libvirt 2025-07-28 22:24:02,493 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-72:[]) (logid:) Asking libvirt to refresh storage pool 00b05d76-e83c-377e-9005-32f9a1d18679 2025-07-28 22:24:02,543 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-73:[]) (logid:) Trying to fetch storage pool e65cc751-30c7-3404-8932-d9d0c1577411 from libvirt 2025-07-28 22:24:02,544 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-73:[]) (logid:) Asking libvirt to refresh storage pool e65cc751-30c7-3404-8932-d9d0c1577411 2025-07-28 22:25:02,455 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-83:[]) (logid:) Trying to fetch storage pool dec8139e-7163-3ea4-b488-cfce1f138e58 from libvirt 2025-07-28 22:25:02,456 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-83:[]) (logid:) Asking libvirt to refresh storage pool dec8139e-7163-3ea4-b488-cfce1f138e58` GitHub link: https://github.com/apache/cloudstack/discussions/12450#discussioncomment-15515803 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
