Re: Cloudstack Agent 4.18.1 metrics issue
Thanks Wei, Unsuspending helped Cheers Jon :) On Wed, Dec 13, 2023 at 4:07 PM Wei ZHOU wrote: > yes. > > the domain state is not supported in libvirt-java > I just created a PR > https://gitlab.com/libvirt/libvirt-java/-/merge_requests/40 > > > -Wei > > On Wed, 13 Dec 2023 at 11:36, Joan g wrote: > > > yes. restarting libvirtd does not help. > > > > "virsh list" command output. Is the "pmsuspended" causing issue ? > > > > root@kvm-1:~# virsh list > > IdName State > > --- > > 11i-2-281-VMrunning > > 25i-2-228-VMrunning > > 27i-2-230-VMrunning > > 52i-2-483-VMrunning > > 121 i-2-585-VMpmsuspended > > 122 i-2-584-VMrunning > > 123 i-2-590-VMrunning > > 139 i-2-635-VMrunning > > 145 i-2-479-VMrunning > > 164 i-8-703-VM running > > > > > > Jon > > > > On Wed, Dec 13, 2023 at 3:48 PM Wei ZHOU wrote: > > > > > Can you restart libvirtd ? If it does not work, can you share the > result > > of > > > "virsh list" command ? > > > > > > > > > > > > -Wei > > > > > > On Wed, 13 Dec 2023 at 09:48, Joan g wrote: > > > > > > > Hi Wei. > > > > > > > > OS : Ubuntu 20.04.6 LTS > > > > > > > > root@kvm-1:~# virsh version > > > > Compiled against library: libvirt 6.0.0 > > > > Using library: libvirt 6.0.0 > > > > Using API: QEMU 6.0.0 > > > > Running hypervisor: QEMU 4.2.1 > > > > > > > > Regards, > > > > Jon > > > > > > > > On Wed, Dec 13, 2023 at 2:10 PM Wei ZHOU > > wrote: > > > > > > > > > What's the OS and libvirt/qemu version ? > > > > > > > > > > -Wei > > > > > > > > > > On Wed, 13 Dec 2023 at 09:30, Joan g wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > Recently below is getting logged in my cloudstack agent and the > > > manager > > > > > is > > > > > > not able to get any VM metrics. Can Someone help on below > > > > > > > > > > > > 2023-12-13 03:23:37,318 WARN [cloud.agent.Agent] > > > > > > (agentRequest-Handler-1:null) (logid:58ef1261) Caught: > > > > > > java.lang.ArrayIndexOutOfBoundsException: Index 7 out of bounds > for > > > > > length > > > > > > 7 > > > > > > at org.libvirt.DomainInfo.(Unknown Source) > > > > > > at org.libvirt.Domain.getInfo(Unknown Source) > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getVmStat(LibvirtComputingResource.java:4176) > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:53) > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:37) > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1848) > > > > > > at com.cloud.agent.Agent.processRequest(Agent.java:662) > > > > > > at > > > > > > com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1082) > > > > > > at com.cloud.utils.nio.Task.call(Task.java:83) > > > > > > at com.cloud.utils.nio.Task.call(Task.java:29) > > > > > > at > > > > > > > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > > > > > > at java.base/java.lang.Thread.run(Thread.java:829) > > > > > > > > > > > > Regards, > > > > > > Jon > > > > > > > > > > > > > > > > > > > > >
Re: Cloudstack Agent 4.18.1 metrics issue
yes. the domain state is not supported in libvirt-java I just created a PR https://gitlab.com/libvirt/libvirt-java/-/merge_requests/40 -Wei On Wed, 13 Dec 2023 at 11:36, Joan g wrote: > yes. restarting libvirtd does not help. > > "virsh list" command output. Is the "pmsuspended" causing issue ? > > root@kvm-1:~# virsh list > IdName State > --- > 11i-2-281-VMrunning > 25i-2-228-VMrunning > 27i-2-230-VMrunning > 52i-2-483-VMrunning > 121 i-2-585-VMpmsuspended > 122 i-2-584-VMrunning > 123 i-2-590-VMrunning > 139 i-2-635-VMrunning > 145 i-2-479-VMrunning > 164 i-8-703-VM running > > > Jon > > On Wed, Dec 13, 2023 at 3:48 PM Wei ZHOU wrote: > > > Can you restart libvirtd ? If it does not work, can you share the result > of > > "virsh list" command ? > > > > > > > > -Wei > > > > On Wed, 13 Dec 2023 at 09:48, Joan g wrote: > > > > > Hi Wei. > > > > > > OS : Ubuntu 20.04.6 LTS > > > > > > root@kvm-1:~# virsh version > > > Compiled against library: libvirt 6.0.0 > > > Using library: libvirt 6.0.0 > > > Using API: QEMU 6.0.0 > > > Running hypervisor: QEMU 4.2.1 > > > > > > Regards, > > > Jon > > > > > > On Wed, Dec 13, 2023 at 2:10 PM Wei ZHOU > wrote: > > > > > > > What's the OS and libvirt/qemu version ? > > > > > > > > -Wei > > > > > > > > On Wed, 13 Dec 2023 at 09:30, Joan g wrote: > > > > > > > > > Hi, > > > > > > > > > > Recently below is getting logged in my cloudstack agent and the > > manager > > > > is > > > > > not able to get any VM metrics. Can Someone help on below > > > > > > > > > > 2023-12-13 03:23:37,318 WARN [cloud.agent.Agent] > > > > > (agentRequest-Handler-1:null) (logid:58ef1261) Caught: > > > > > java.lang.ArrayIndexOutOfBoundsException: Index 7 out of bounds for > > > > length > > > > > 7 > > > > > at org.libvirt.DomainInfo.(Unknown Source) > > > > > at org.libvirt.Domain.getInfo(Unknown Source) > > > > > at > > > > > > > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getVmStat(LibvirtComputingResource.java:4176) > > > > > at > > > > > > > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:53) > > > > > at > > > > > > > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:37) > > > > > at > > > > > > > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) > > > > > at > > > > > > > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1848) > > > > > at com.cloud.agent.Agent.processRequest(Agent.java:662) > > > > > at > > > > > com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1082) > > > > > at com.cloud.utils.nio.Task.call(Task.java:83) > > > > > at com.cloud.utils.nio.Task.call(Task.java:29) > > > > > at > > > > > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > > > > > at > > > > > > > > > > > > > > > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > > > > > at > > > > > > > > > > > > > > > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > > > > > at java.base/java.lang.Thread.run(Thread.java:829) > > > > > > > > > > Regards, > > > > > Jon > > > > > > > > > > > > > > >
Re: Cloudstack Agent 4.18.1 metrics issue
yes. restarting libvirtd does not help. "virsh list" command output. Is the "pmsuspended" causing issue ? root@kvm-1:~# virsh list IdName State --- 11i-2-281-VMrunning 25i-2-228-VMrunning 27i-2-230-VMrunning 52i-2-483-VMrunning 121 i-2-585-VMpmsuspended 122 i-2-584-VMrunning 123 i-2-590-VMrunning 139 i-2-635-VMrunning 145 i-2-479-VMrunning 164 i-8-703-VM running Jon On Wed, Dec 13, 2023 at 3:48 PM Wei ZHOU wrote: > Can you restart libvirtd ? If it does not work, can you share the result of > "virsh list" command ? > > > > -Wei > > On Wed, 13 Dec 2023 at 09:48, Joan g wrote: > > > Hi Wei. > > > > OS : Ubuntu 20.04.6 LTS > > > > root@kvm-1:~# virsh version > > Compiled against library: libvirt 6.0.0 > > Using library: libvirt 6.0.0 > > Using API: QEMU 6.0.0 > > Running hypervisor: QEMU 4.2.1 > > > > Regards, > > Jon > > > > On Wed, Dec 13, 2023 at 2:10 PM Wei ZHOU wrote: > > > > > What's the OS and libvirt/qemu version ? > > > > > > -Wei > > > > > > On Wed, 13 Dec 2023 at 09:30, Joan g wrote: > > > > > > > Hi, > > > > > > > > Recently below is getting logged in my cloudstack agent and the > manager > > > is > > > > not able to get any VM metrics. Can Someone help on below > > > > > > > > 2023-12-13 03:23:37,318 WARN [cloud.agent.Agent] > > > > (agentRequest-Handler-1:null) (logid:58ef1261) Caught: > > > > java.lang.ArrayIndexOutOfBoundsException: Index 7 out of bounds for > > > length > > > > 7 > > > > at org.libvirt.DomainInfo.(Unknown Source) > > > > at org.libvirt.Domain.getInfo(Unknown Source) > > > > at > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getVmStat(LibvirtComputingResource.java:4176) > > > > at > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:53) > > > > at > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:37) > > > > at > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) > > > > at > > > > > > > > > > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1848) > > > > at com.cloud.agent.Agent.processRequest(Agent.java:662) > > > > at > > > > com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1082) > > > > at com.cloud.utils.nio.Task.call(Task.java:83) > > > > at com.cloud.utils.nio.Task.call(Task.java:29) > > > > at > > > > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > > > > at > > > > > > > > > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > > > > at > > > > > > > > > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > > > > at java.base/java.lang.Thread.run(Thread.java:829) > > > > > > > > Regards, > > > > Jon > > > > > > > > > >
Re: Cloudstack Agent 4.18.1 metrics issue
Can you restart libvirtd ? If it does not work, can you share the result of "virsh list" command ? -Wei On Wed, 13 Dec 2023 at 09:48, Joan g wrote: > Hi Wei. > > OS : Ubuntu 20.04.6 LTS > > root@kvm-1:~# virsh version > Compiled against library: libvirt 6.0.0 > Using library: libvirt 6.0.0 > Using API: QEMU 6.0.0 > Running hypervisor: QEMU 4.2.1 > > Regards, > Jon > > On Wed, Dec 13, 2023 at 2:10 PM Wei ZHOU wrote: > > > What's the OS and libvirt/qemu version ? > > > > -Wei > > > > On Wed, 13 Dec 2023 at 09:30, Joan g wrote: > > > > > Hi, > > > > > > Recently below is getting logged in my cloudstack agent and the manager > > is > > > not able to get any VM metrics. Can Someone help on below > > > > > > 2023-12-13 03:23:37,318 WARN [cloud.agent.Agent] > > > (agentRequest-Handler-1:null) (logid:58ef1261) Caught: > > > java.lang.ArrayIndexOutOfBoundsException: Index 7 out of bounds for > > length > > > 7 > > > at org.libvirt.DomainInfo.(Unknown Source) > > > at org.libvirt.Domain.getInfo(Unknown Source) > > > at > > > > > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getVmStat(LibvirtComputingResource.java:4176) > > > at > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:53) > > > at > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:37) > > > at > > > > > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) > > > at > > > > > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1848) > > > at com.cloud.agent.Agent.processRequest(Agent.java:662) > > > at > > > com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1082) > > > at com.cloud.utils.nio.Task.call(Task.java:83) > > > at com.cloud.utils.nio.Task.call(Task.java:29) > > > at > > > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > > > at > > > > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > > > at > > > > > > > > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > > > at java.base/java.lang.Thread.run(Thread.java:829) > > > > > > Regards, > > > Jon > > > > > >
Re: Cloudstack Agent 4.18.1 metrics issue
Hi Wei. OS : Ubuntu 20.04.6 LTS root@kvm-1:~# virsh version Compiled against library: libvirt 6.0.0 Using library: libvirt 6.0.0 Using API: QEMU 6.0.0 Running hypervisor: QEMU 4.2.1 Regards, Jon On Wed, Dec 13, 2023 at 2:10 PM Wei ZHOU wrote: > What's the OS and libvirt/qemu version ? > > -Wei > > On Wed, 13 Dec 2023 at 09:30, Joan g wrote: > > > Hi, > > > > Recently below is getting logged in my cloudstack agent and the manager > is > > not able to get any VM metrics. Can Someone help on below > > > > 2023-12-13 03:23:37,318 WARN [cloud.agent.Agent] > > (agentRequest-Handler-1:null) (logid:58ef1261) Caught: > > java.lang.ArrayIndexOutOfBoundsException: Index 7 out of bounds for > length > > 7 > > at org.libvirt.DomainInfo.(Unknown Source) > > at org.libvirt.Domain.getInfo(Unknown Source) > > at > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getVmStat(LibvirtComputingResource.java:4176) > > at > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:53) > > at > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:37) > > at > > > > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) > > at > > > > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1848) > > at com.cloud.agent.Agent.processRequest(Agent.java:662) > > at > > com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1082) > > at com.cloud.utils.nio.Task.call(Task.java:83) > > at com.cloud.utils.nio.Task.call(Task.java:29) > > at > > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > > at > > > > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > > at > > > > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > > at java.base/java.lang.Thread.run(Thread.java:829) > > > > Regards, > > Jon > > >
Re: Cloudstack Agent 4.18.1 metrics issue
What's the OS and libvirt/qemu version ? -Wei On Wed, 13 Dec 2023 at 09:30, Joan g wrote: > Hi, > > Recently below is getting logged in my cloudstack agent and the manager is > not able to get any VM metrics. Can Someone help on below > > 2023-12-13 03:23:37,318 WARN [cloud.agent.Agent] > (agentRequest-Handler-1:null) (logid:58ef1261) Caught: > java.lang.ArrayIndexOutOfBoundsException: Index 7 out of bounds for length > 7 > at org.libvirt.DomainInfo.(Unknown Source) > at org.libvirt.Domain.getInfo(Unknown Source) > at > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.getVmStat(LibvirtComputingResource.java:4176) > at > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:53) > at > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtGetVmStatsCommandWrapper.execute(LibvirtGetVmStatsCommandWrapper.java:37) > at > > com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:78) > at > > com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1848) > at com.cloud.agent.Agent.processRequest(Agent.java:662) > at > com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:1082) > at com.cloud.utils.nio.Task.call(Task.java:83) > at com.cloud.utils.nio.Task.call(Task.java:29) > at > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > at > > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at java.base/java.lang.Thread.run(Thread.java:829) > > Regards, > Jon >