GitHub user ong228 closed a discussion: Incorrect Network setup on agent (ACS
4.19.1 ; Ubuntu Server 24.04.1)
<!--
Verify first that your issue/request is not already reported on GitHub.
Also test if the latest release and main branch are affected too.
Always add information AFTER of these HTML comments, but no need to delete the
comments.
-->
##### ISSUE TYPE
<!-- Pick one below and delete the rest -->
* Other
##### COMPONENT NAME
<!--
Categorize the issue, e.g. API, VR, VPN, UI, etc.
-->
~~~
Initial setup adding KVM host to ACS MGMT
~~~
##### CLOUDSTACK VERSION
<!--
New line separated list of affected versions, commit ID for issues on main
branch.
-->
~~~
4.19.1
~~~
##### CONFIGURATION
<!--
Information about the configuration if relevant, e.g. basic network, advanced
networking, etc. N/A otherwise
-->
Picked Core -> Advanced Networking ; Has basic Zone, Pod and Cluster
##### OS / ENVIRONMENT
<!--
Information about the environment if relevant, N/A otherwise
-->
Ubuntu Server 24.04.1 LTS
##### SUMMARY
<!-- Explain the problem/feature briefly -->
##### STEPS TO REPRODUCE
<!--
For bugs, show exactly how to reproduce the problem, using a minimal test-case.
Use Screenshots if accurate.
For new features, show how the feature would be used.
-->
<!-- Paste example playbooks or commands between quotes below -->
~~~
Ubuntu Server KVM Host Netplan
network:
version: 2
renderer: networkd
ethernets:
ens3f0:
dhcp4: no
ens3f1:
dhcp4: no
ens3f2:
dhcp4: no
ens3f3:
dhcp4: no
ens3f4:
dhcp4: no
ens3f5:
dhcp4: no
ens3f6:
dhcp4: no
ens3f7:
dhcp4: no
bonds:
MgmtBond:
interfaces: [ens3f0,ens3f1]
parameters:
mode: 802.3ad
mtu: 9000
NFSBond:
interfaces: [ens3f2,ens3f3]
parameters:
mode: 802.3ad
mtu: 9000
TunnelsBond:
interfaces: [ens3f4,ens3f5]
parameters:
mode: 802.3ad
mtu: 9000
NotUsedBond:
interfaces: [ens3f6,ens3f7]
parameters:
mode: 802.3ad
mtu: 9000
bridges:
MgmtNetVLAN600:
addresses: [x.x.x.x/23]
dhcp4: no
routes:
- to: default
via: x.x.x.x
nameservers:
addresses: [xxxxxxxxxxxxx]
search: [xxxxxxxxxxxx]
interfaces:
- MgmtBond
NFSNetVLAN600:
addresses: [x.x.x.x]
dhcp4: no
nameservers:
addresses: [xxxxxxxx]
search: [xxxx]
interfaces:
- NFSBond
GuestNet:
dhcp4: no
interfaces:
- TunnelsBond
brctl show
bridge name bridge id STP enabled interfaces
GuestNet 8000.829f60077655 no TunnelsBond
MgmtNetVLAN600 8000.b678ceb71e00 no MgmtBond
NFSNetVLAN600 8000.6a708247dfd8 no NFSBond
cloud0 8000.929cbacc7081 no
~~~
<!-- You can also paste gist.github.com links for larger files -->
##### EXPECTED RESULTS
<!-- What did you expect to happen when running the steps above? -->
~~~
Adding host works i suppose. It's my first time setup.
Note :
It worked when i create only bonding in Netplan and use MgmtNetVLAN600 as name
with IP/GW/DNS etc (no bridges created in Netplan at all) but subsequent setup
with System VMs and Guest VMs did not work hence I am trying this route with
creating bonds, and then bridges too
~~~
##### ACTUAL RESULTS
<!-- What actually happened? -->
<!-- Paste verbatim command output between quotes below -->
~~~
ACS Management log ('error cannot find network MgmtNetVLAN600')
2024-10-08 03:57:55,584 INFO [c.c.r.ResourceManagerImpl]
(qtp1374754488-18:ctx-98530a47 ctx-f1ee4153) (logid:af1051eb) Trying to add a
new host at http://<KVM-Host-FQDN> in data center 2
2024-10-08 03:57:55,705 DEBUG [c.c.u.s.SSHCmdHelper]
(qtp1374754488-18:ctx-98530a47 ctx-f1ee4153) (logid:af1051eb) Executing cmd: ls
/dev/kvm
2024-10-08 03:57:56,580 DEBUG [o.a.c.h.HAManagerImpl]
(BackgroundTaskPollManager-6:ctx-26714a4f) (logid:622d07be) HA health check
task is running...
2024-10-08 03:57:57,046 DEBUG [c.c.u.s.SSHCmdHelper]
(qtp1374754488-18:ctx-98530a47 ctx-f1ee4153) (logid:af1051eb) SSH command: ls
/dev/kvm
SSH command output:/dev/kvm
2024-10-08 03:57:57,052 DEBUG [c.c.u.s.SSHCmdHelper]
(qtp1374754488-18:ctx-98530a47 ctx-f1ee4153) (logid:af1051eb) Executing cmd:
sudo /usr/share/cloudstack-common/scripts/util/keystore-setup
/etc/cloudstack/agent/agent.properties /etc/cloudstack/agent/
2024-10-08 03:57:58,024 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-198c6ffd) (logid:cadd9377) Skip capacity scan as there is
no Primary Storage in 'Up' state
2024-10-08 03:57:58,192 DEBUG [c.c.a.ApiServlet]
(qtp1374754488-20:ctx-5000b4f8) (logid:010e37e4) ===START=== x.x.x.x -- GET
command=readyForShutdown&response=json
2024-10-08 03:57:58,192 DEBUG [c.c.a.ApiServlet]
(qtp1374754488-20:ctx-5000b4f8) (logid:010e37e4) Two factor authentication is
already verified for the user 2, so skipping
2024-10-08 03:57:58,196 DEBUG [c.c.a.ApiServer] (qtp1374754488-20:ctx-5000b4f8
ctx-e2bb36a6) (logid:010e37e4) CIDRs from which account 'Account
[{"accountName":"admin","id":2,"uuid":"9995c13e-808d-11ef-8d51-005056a4ff14"}]'
is allowed to perform API calls: 0.0.0.0/0,::/0
2024-10-08 03:57:58,198 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker]
(qtp1374754488-20:ctx-5000b4f8 ctx-e2bb36a6) (logid:010e37e4) RoleService is
enabled. We will use it instead of StaticRoleBasedAPIAccessChecker.
2024-10-08 03:57:58,199 DEBUG [o.a.c.r.ApiRateLimitServiceImpl]
(qtp1374754488-20:ctx-5000b4f8 ctx-e2bb36a6) (logid:010e37e4) API rate limiting
is disabled. We will not use ApiRateLimitService.
2024-10-08 03:57:58,200 DEBUG [c.c.a.ApiServlet] (qtp1374754488-20:ctx-5000b4f8
ctx-e2bb36a6) (logid:010e37e4) ===END=== x.x.x.x -- GET
command=readyForShutdown&response=json
2024-10-08 03:57:58,226 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-a306536e) (logid:89796b84) Enabled non-edge zones available
for scan:
2024-10-08 03:57:58,755 DEBUG [c.c.u.s.SSHCmdHelper]
(qtp1374754488-18:ctx-98530a47 ctx-f1ee4153) (logid:af1051eb) SSH command: sudo
/usr/share/cloudstack-common/scripts/util/keystore-setup
/etc/cloudstack/agent/agent.properties /etc/cloudstack/agent/
SSH command output:Failed to generate CSR file, retrying after removing
existing settings
Reverting libvirtd to not listen on TLS
Removing cloud.* files in /etc/cloudstack/agent
Retrying to generate CSR file
Failed to generate CSR file while retrying
2024-10-08 03:57:58,755 DEBUG [c.c.u.s.SSHCmdHelper]
(qtp1374754488-18:ctx-98530a47 ctx-f1ee4153) (logid:af1051eb) Executing cmd:
sudo /usr/share/cloudstack-common/scripts/util/keystore-setup
/etc/cloudstack/agent/agent.properties /etc/cloudstack/agent/
2024-10-08 03:58:00,581 DEBUG [o.a.c.h.HAManagerImpl]
(BackgroundTaskPollManager-6:ctx-02918256) (logid:f9c1d55b) HA health check
task is running...
2024-10-08 03:58:00,899 DEBUG [c.c.u.s.SSHCmdHelper]
(qtp1374754488-18:ctx-98530a47 ctx-f1ee4153) (logid:af1051eb) SSH command: sudo
/usr/share/cloudstack-common/scripts/util/keystore-setup
/etc/cloudstack/agent/agent.properties /etc/cloudstack/agent/
SSH command output:
2024-10-08 03:58:00,924 DEBUG [c.c.u.s.SSHCmdHelper]
(qtp1374754488-18:ctx-98530a47 ctx-f1ee4153) (logid:af1051eb) Executing cmd:
sudo /usr/share/cloudstack-common/scripts/util/keystore-cert-import
/etc/cloudstack/agent/agent.properties qVyN7RKCCM6x48Gf /etc/cloudstack/agent/
2024-10-08 03:58:03,193 DEBUG [c.c.a.ApiServlet]
(qtp1374754488-20:ctx-6b786ed9) (logid:2b4b154c) ===START=== x.x.x.x -- GET
command=readyForShutdown&response=json
2024-10-08 03:58:03,193 DEBUG [c.c.a.ApiServlet]
(qtp1374754488-20:ctx-6b786ed9) (logid:2b4b154c) Two factor authentication is
already verified for the user 2, so skipping
2024-10-08 03:58:03,197 DEBUG [c.c.a.ApiServer] (qtp1374754488-20:ctx-6b786ed9
ctx-ad59e666) (logid:2b4b154c) CIDRs from which account 'Account
[{"accountName":"admin","id":2,"uuid":"9995c13e-808d-11ef-8d51-005056a4ff14"}]'
is allowed to perform API calls: 0.0.0.0/0,::/0
2024-10-08 03:58:03,200 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker]
(qtp1374754488-20:ctx-6b786ed9 ctx-ad59e666) (logid:2b4b154c) RoleService is
enabled. We will use it instead of StaticRoleBasedAPIAccessChecker.
2024-10-08 03:58:03,200 DEBUG [o.a.c.r.ApiRateLimitServiceImpl]
(qtp1374754488-20:ctx-6b786ed9 ctx-ad59e666) (logid:2b4b154c) API rate limiting
is disabled. We will not use ApiRateLimitService.
2024-10-08 03:58:03,201 DEBUG [c.c.a.ApiServlet] (qtp1374754488-20:ctx-6b786ed9
ctx-ad59e666) (logid:2b4b154c) ===END=== x.x.x.x -- GET
command=readyForShutdown&response=json
2024-10-08 03:58:03,252 DEBUG [c.c.h.k.d.LibvirtServerDiscoverer]
(qtp1374754488-18:ctx-98530a47 ctx-f1ee4153) (logid:af1051eb) Succeeded to
import certificate in the keystore for agent on the KVM host: <KVM-Host-IP>.
Agent secured and trusted.
2024-10-08 03:58:03,255 DEBUG [c.c.u.s.SSHCmdHelper]
(qtp1374754488-18:ctx-98530a47 ctx-f1ee4153) (logid:af1051eb) Executing cmd:
sudo cloudstack-setup-agent -m <MGMT-IP> -z 2 -p 2 -c 1 -g
e47f5b77-147f-36d8-8619-1e67f220d5e1 -a -s --pubNic=GuestNet
--prvNic=MgmtNetVLAN600 --guestNic=GuestNet --hypervisor=kvm
2024-10-08 03:58:04,582 DEBUG [o.a.c.h.HAManagerImpl]
(BackgroundTaskPollManager-6:ctx-6316354a) (logid:dadb8ac5) HA health check
task is running...
2024-10-08 03:58:05,696 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-9cfb730e) (logid:c92d1c32) Found 0 routers to update
status.
2024-10-08 03:58:05,697 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-9cfb730e) (logid:c92d1c32) Found 0 VPC's to update
Redundant State.
2024-10-08 03:58:05,698 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-9cfb730e) (logid:c92d1c32) Found 0 networks to
update RvR status.
2024-10-08 03:58:05,728 DEBUG [c.c.s.StatsCollector]
(StatsCollector-2:ctx-488064dd) (logid:0ad81c25) DbCollector is running...
2024-10-08 03:58:06,726 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) ManagementServerCollector is
running...
2024-10-08 03:58:06,727 DEBUG [c.c.u.d.DbProperties]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) DB properties were already
loaded
2024-10-08 03:58:06,727 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Executing command [/bin/bash -c systemctl status
cloudstack-usage | grep " Loaded:" ].
2024-10-08 03:58:06,734 WARN [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Execution of process [56039] for command [/bin/bash -c
systemctl status cloudstack-usage | grep " Loaded:" ] failed.
2024-10-08 03:58:06,734 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Exit value of process [56039] for command [/bin/bash -c
systemctl status cloudstack-usage | grep " Loaded:" ] is [1].
2024-10-08 03:58:06,734 WARN [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Process [56039] for command [/bin/bash -c systemctl status
cloudstack-usage | grep " Loaded:" ] encountered the error: [Unit
cloudstack-usage.service could not be found.].
2024-10-08 03:58:06,735 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) usage install: null
2024-10-08 03:58:06,735 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) Sessions active: 1
2024-10-08 03:58:06,736 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Executing command [/bin/bash -c cat /etc/os-release | grep
PRETTY_NAME | cut -f2 -d '=' | tr -d '"' ].
2024-10-08 03:58:06,739 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Successfully executed process [56042] for command [/bin/bash
-c cat /etc/os-release | grep PRETTY_NAME | cut -f2 -d '=' | tr -d '"' ].
2024-10-08 03:58:06,739 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Executing command [/bin/bash -c uname -r ].
2024-10-08 03:58:06,741 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Successfully executed process [56047] for command [/bin/bash
-c uname -r ].
2024-10-08 03:58:06,742 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Executing command [/bin/bash -c cat /proc/meminfo | grep
MemTotal | cut -f 2 -d ':' | tr -d 'a-zA-z ' ].
2024-10-08 03:58:06,744 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Successfully executed process [56048] for command [/bin/bash
-c cat /proc/meminfo | grep MemTotal | cut -f 2 -d ':' | tr -d 'a-zA-z ' ].
2024-10-08 03:58:06,744 INFO [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) system memory from /proc:
8326586368
2024-10-08 03:58:06,745 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Executing command [/bin/bash -c cat /proc/meminfo | grep
MemFree | cut -f 2 -d ':' | tr -d 'a-zA-z ' ].
2024-10-08 03:58:06,747 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Successfully executed process [56053] for command [/bin/bash
-c cat /proc/meminfo | grep MemFree | cut -f 2 -d ':' | tr -d 'a-zA-z ' ].
2024-10-08 03:58:06,747 INFO [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) free memory from /proc:
4363476992
2024-10-08 03:58:06,747 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Executing command [/bin/bash -c ps -o rss= 1175 ].
2024-10-08 03:58:06,755 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Successfully executed process [56058] for command [/bin/bash
-c ps -o rss= 1175 ].
2024-10-08 03:58:06,755 INFO [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) used memory from /proc: 816200
2024-10-08 03:58:06,755 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Executing command [/bin/bash -c uptime -s ].
2024-10-08 03:58:06,757 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Successfully executed process [56059] for command [/bin/bash
-c uptime -s ].
2024-10-08 03:58:06,758 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Executing command [/bin/bash -c ps -o vsz= 1175 ].
2024-10-08 03:58:06,764 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Successfully executed process [56060] for command [/bin/bash
-c ps -o vsz= 1175 ].
2024-10-08 03:58:06,765 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Executing command [/bin/bash -c cat /proc/cpuinfo | grep "cpu
MHz" | grep "cpu MHz" | cut -f 2 -d : | tr -d ' '| tr '\n' " " ].
2024-10-08 03:58:06,768 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Successfully executed process [56061] for command [/bin/bash
-c cat /proc/cpuinfo | grep "cpu MHz" | grep "cpu MHz" | cut -f 2 -d : | tr -d
' '| tr '\n' " " ].
2024-10-08 03:58:06,768 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Executing command [/bin/bash -c cat /proc/loadavg ].
2024-10-08 03:58:06,770 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Successfully executed process [56068] for command [/bin/bash
-c cat /proc/loadavg ].
2024-10-08 03:58:06,770 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Executing command [/bin/bash -c cat /proc/stat | grep "cpu " |
tr -d "cpu" ].
2024-10-08 03:58:06,773 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Successfully executed process [56069] for command [/bin/bash
-c cat /proc/stat | grep "cpu " | tr -d "cpu" ].
2024-10-08 03:58:06,773 DEBUG [c.c.u.LogUtils] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) file for FILE :
/var/log/cloudstack/management/management-server.log
2024-10-08 03:58:06,773 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Executing command [/bin/bash -c du -sh
/var/log/cloudstack/management/management-server.log | cut -f '1' ].
2024-10-08 03:58:06,775 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Successfully executed process [56073] for command [/bin/bash
-c du -sh /var/log/cloudstack/management/management-server.log | cut -f '1' ].
2024-10-08 03:58:06,776 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Executing command [/bin/bash -c df -h
/var/log/cloudstack/management/management-server.log | grep -v Filesystem | awk
'{print "on disk " $1 " mounted on " $6 " (" $5 " full)"}' ].
2024-10-08 03:58:06,779 DEBUG [c.c.u.s.Script] (StatsCollector-5:ctx-0579e6a1)
(logid:ed27ba51) Successfully executed process [56076] for command [/bin/bash
-c df -h /var/log/cloudstack/management/management-server.log | grep -v
Filesystem | awk '{print "on disk " $1 " mounted on " $6 " (" $5 " full)"}' ].
2024-10-08 03:58:06,780 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
buffersdirect.capacity, 242660
2024-10-08 03:58:06,780 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
buffersdirect.count, 20
2024-10-08 03:58:06,780 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
buffersdirect.used, 242660
2024-10-08 03:58:06,780 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
buffersmapped.capacity, 0
2024-10-08 03:58:06,780 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
buffersmapped.count, 0
2024-10-08 03:58:06,780 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
buffersmapped.used, 0
2024-10-08 03:58:06,780 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
gcPS-MarkSweep.count, 6
2024-10-08 03:58:06,780 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
gcPS-MarkSweep.time, 1180
2024-10-08 03:58:06,780 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
gcPS-Scavenge.count, 321
2024-10-08 03:58:06,780 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
gcPS-Scavenge.time, 1995
2024-10-08 03:58:06,780 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail jvmname,
1175@<MGMT-FQDN>
2024-10-08 03:58:06,780 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail jvmuptime,
73349034
2024-10-08 03:58:06,780 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail jvmvendor,
Ubuntu OpenJDK 64-Bit Server VM 11.0.24+8-post-Ubuntu-1ubuntu324.04.1 (11)
2024-10-08 03:58:06,780 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
memoryheap.committed, 346030080
2024-10-08 03:58:06,780 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
memoryheap.init, 132120576
2024-10-08 03:58:06,780 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
memoryheap.usage, 0.10579723933344849
2024-10-08 03:58:06,780 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
memorynon-heap.committed, 221929472
2024-10-08 03:58:06,781 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
memorynon-heap.init, 7667712
2024-10-08 03:58:06,781 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
memorynon-heap.max, -1
2024-10-08 03:58:06,781 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
memorynon-heap.usage, -2.14037608E8
2024-10-08 03:58:06,781 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
memorynon-heap.used, 214037608
2024-10-08 03:58:06,781 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
memorypools.CodeHeap-'non-nmethods'.usage, 0.3162087429775281
2024-10-08 03:58:06,781 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
memorypools.CodeHeap-'non-profiled-nmethods'.usage, 0.17123975273260464
2024-10-08 03:58:06,781 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
memorypools.CodeHeap-'profiled-nmethods'.usage, 0.36812579145561186
2024-10-08 03:58:06,781 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
memorypools.Compressed-Class-Space.usage, 0.011766009032726288
2024-10-08 03:58:06,781 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
memorypools.Metaspace.usage, 0.9737033644206021
2024-10-08 03:58:06,781 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
memorypools.PS-Eden-Space.usage, 0.011914277759919177
2024-10-08 03:58:06,781 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
memorypools.PS-Old-Gen.usage, 0.13457664732791616
2024-10-08 03:58:06,781 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
memorypools.PS-Survivor-Space.usage, 0.7812652587890625
2024-10-08 03:58:06,781 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
memorytotal.committed, 567959552
2024-10-08 03:58:06,781 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
memorytotal.init, 139788288
2024-10-08 03:58:06,781 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
memorytotal.max, 1908932607
2024-10-08 03:58:06,781 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
memorytotal.used, 416075200
2024-10-08 03:58:06,781 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
threadsdeadlocks, []
2024-10-08 03:58:06,781 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
threadsnew.count, 0
2024-10-08 03:58:06,782 DEBUG [c.c.s.StatsCollector]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) not storing detail
threadstimed_waiting.count, 80
2024-10-08 03:58:06,783 DEBUG [c.c.c.ClusterManagerImpl]
(StatsCollector-5:ctx-0579e6a1) (logid:ed27ba51) Forwarding
{"managementServerHostId":1,"managementServerHostUuid":"79a10d06-c488-41cf-969d-3f31b75435f9","collectionTime":"Oct
8, 2024, 3:58:06
AM","sessions":1,"cpuUtilization":0.0,"totalJvmMemoryBytes":346030080,"freeJvmMemoryBytes":145056344,"maxJvmMemoryBytes":1908932607,"processJvmMemoryBytes":0,"jvmUptime":73348988,"jvmStartTime":1728286537893,"availableProcessors":4,"loadAverage":0.01708984375,"totalInit":139788288,"totalUsed":415014000,"totalCommitted":567959552,"pid":1175,"jvmName":"1175@<MGMT-FQDN>","jvmVendor":"Ubuntu","jvmVersion":"11.0.24+8-post-Ubuntu-1ubuntu324.04.1","osDistribution":"Ubuntu
24.04.1
LTS","agentCount":0,"heapMemoryUsed":201959800,"heapMemoryTotal":1908932608,"threadsBlockedCount":0,"threadsDaemonCount":15,"threadsRunnableCount":9,"threadsTerminatedCount":0,"threadsTotalCount":218,"threadsWaitingCount":129,"systemMemoryTotal":8326586368,"systemMemory
Free":4363476992,"systemMemoryUsed":816200,"systemMemoryVirtualSize":6234066944,"logInfo":"/var/log/cloudstack/management/management-server.log
using: 6.2M\non disk /dev/mapper/ubuntu--vg-ubuntu--lv mounted on / (5%
full)","systemTotalCpuCycles":9577.496,"systemLoadAverages":[0.02,0.01,0.0],"systemCyclesUsage":[79069,40903,29083240],"dbLocal":true,"usageLocal":false,"systemBootTime":"Oct
7, 2024, 9:35:30 AM","kernelVersion":"6.8.0-45-generic"} to 345051037460
2024-10-08 03:58:06,783 DEBUG [c.c.c.ClusterManagerImpl]
(Cluster-Worker-4:ctx-f8aab4de) (logid:39e2a891) Cluster PDU 345051037460 ->
345051037460. agent: 0, pdu seq: 2445, pdu ack seq: 0, json:
{"managementServerHostId":1,"managementServerHostUuid":"79a10d06-c488-41cf-969d-3f31b75435f9","collectionTime":"Oct
8, 2024, 3:58:06
AM","sessions":1,"cpuUtilization":0.0,"totalJvmMemoryBytes":346030080,"freeJvmMemoryBytes":145056344,"maxJvmMemoryBytes":1908932607,"processJvmMemoryBytes":0,"jvmUptime":73348988,"jvmStartTime":1728286537893,"availableProcessors":4,"loadAverage":0.01708984375,"totalInit":139788288,"totalUsed":415014000,"totalCommitted":567959552,"pid":1175,"jvmName":"1175@<MGMT-FQDN>","jvmVendor":"Ubuntu","jvmVersion":"11.0.24+8-post-Ubuntu-1ubuntu324.04.1","osDistribution":"Ubuntu
24.04.1
LTS","agentCount":0,"heapMemoryUsed":201959800,"heapMemoryTotal":1908932608,"threadsBlockedCount":0,"threadsDaemonCount":15,"threadsRunnableCount":9,"threadsTerminatedCount":0,"threadsTotalCou
nt":218,"threadsWaitingCount":129,"systemMemoryTotal":8326586368,"systemMemoryFree":4363476992,"systemMemoryUsed":816200,"systemMemoryVirtualSize":6234066944,"logInfo":"/var/log/cloudstack/management/management-server.log
using: 6.2M\non disk /dev/mapper/ubuntu--vg-ubuntu--lv mounted on / (5%
full)","systemTotalCpuCycles":9577.496,"systemLoadAverages":[0.02,0.01,0.0],"systemCyclesUsage":[79069,40903,29083240],"dbLocal":true,"usageLocal":false,"systemBootTime":"Oct
7, 2024, 9:35:30 AM","kernelVersion":"6.8.0-45-generic"}
2024-10-08 03:58:06,783 DEBUG [c.c.c.ClusterServiceServletImpl]
(Cluster-Worker-4:ctx-f8aab4de) (logid:39e2a891) Executing ClusterServicePdu
with service URL: https://<MGMT-IP>:9090/clusterservice
2024-10-08 03:58:06,787 DEBUG [c.c.s.StatsCollector]
(Cluster-Worker-547:ctx-9bcae50b) (logid:b56f5716) StatusUpdate from
345051037460, json:
{"managementServerHostId":1,"managementServerHostUuid":"79a10d06-c488-41cf-969d-3f31b75435f9","collectionTime":"Oct
8, 2024, 3:58:06
AM","sessions":1,"cpuUtilization":0.0,"totalJvmMemoryBytes":346030080,"freeJvmMemoryBytes":145056344,"maxJvmMemoryBytes":1908932607,"processJvmMemoryBytes":0,"jvmUptime":73348988,"jvmStartTime":1728286537893,"availableProcessors":4,"loadAverage":0.01708984375,"totalInit":139788288,"totalUsed":415014000,"totalCommitted":567959552,"pid":1175,"jvmName":"1175@<MGMT-FQDN>","jvmVendor":"Ubuntu","jvmVersion":"11.0.24+8-post-Ubuntu-1ubuntu324.04.1","osDistribution":"Ubuntu
24.04.1
LTS","agentCount":0,"heapMemoryUsed":201959800,"heapMemoryTotal":1908932608,"threadsBlockedCount":0,"threadsDaemonCount":15,"threadsRunnableCount":9,"threadsTerminatedCount":0,"threadsTotalCount":218,"threadsWaitingCount":129,"systemMemoryTotal"
:8326586368,"systemMemoryFree":4363476992,"systemMemoryUsed":816200,"systemMemoryVirtualSize":6234066944,"logInfo":"/var/log/cloudstack/management/management-server.log
using: 6.2M\non disk /dev/mapper/ubuntu--vg-ubuntu--lv mounted on / (5%
full)","systemTotalCpuCycles":9577.496,"systemLoadAverages":[0.02,0.01,0.0],"systemCyclesUsage":[79069,40903,29083240],"dbLocal":true,"usageLocal":false,"systemBootTime":"Oct
7, 2024, 9:35:30 AM","kernelVersion":"6.8.0-45-generic"}
2024-10-08 03:58:06,788 DEBUG [c.c.c.ClusterServiceServletImpl]
(Cluster-Worker-4:ctx-f8aab4de) (logid:39e2a891) POST
https://<MGMT-IP>:9090/clusterservice response :true, responding time: 4 ms
2024-10-08 03:58:06,788 DEBUG [c.c.c.ClusterManagerImpl]
(Cluster-Worker-4:ctx-f8aab4de) (logid:39e2a891) Cluster PDU 345051037460 ->
345051037460 completed. time: 4ms. agent: 0, pdu seq: 2445, pdu ack seq: 0,
json:
{"managementServerHostId":1,"managementServerHostUuid":"79a10d06-c488-41cf-969d-3f31b75435f9","collectionTime":"Oct
8, 2024, 3:58:06
AM","sessions":1,"cpuUtilization":0.0,"totalJvmMemoryBytes":346030080,"freeJvmMemoryBytes":145056344,"maxJvmMemoryBytes":1908932607,"processJvmMemoryBytes":0,"jvmUptime":73348988,"jvmStartTime":1728286537893,"availableProcessors":4,"loadAverage":0.01708984375,"totalInit":139788288,"totalUsed":415014000,"totalCommitted":567959552,"pid":1175,"jvmName":"1175@<MGMT-FQDN>","jvmVendor":"Ubuntu","jvmVersion":"11.0.24+8-post-Ubuntu-1ubuntu324.04.1","osDistribution":"Ubuntu
24.04.1
LTS","agentCount":0,"heapMemoryUsed":201959800,"heapMemoryTotal":1908932608,"threadsBlockedCount":0,"threadsDaemonCount":15,"threadsRunnableCount":9,"threadsTerminatedCoun
t":0,"threadsTotalCount":218,"threadsWaitingCount":129,"systemMemoryTotal":8326586368,"systemMemoryFree":4363476992,"systemMemoryUsed":816200,"systemMemoryVirtualSize":6234066944,"logInfo":"/var/log/cloudstack/management/management-server.log
using: 6.2M\non disk /dev/mapper/ubuntu--vg-ubuntu--lv mounted on / (5%
full)","systemTotalCpuCycles":9577.496,"systemLoadAverages":[0.02,0.01,0.0],"systemCyclesUsage":[79069,40903,29083240],"dbLocal":true,"usageLocal":false,"systemBootTime":"Oct
7, 2024, 9:35:30 AM","kernelVersion":"6.8.0-45-generic"}
2024-10-08 03:58:08,194 DEBUG [c.c.a.ApiServlet]
(qtp1374754488-16:ctx-11ca45a7) (logid:5bcd6d11) ===START=== x.x.x.x -- GET
command=readyForShutdown&response=json
2024-10-08 03:58:08,194 DEBUG [c.c.a.ApiServlet]
(qtp1374754488-16:ctx-11ca45a7) (logid:5bcd6d11) Two factor authentication is
already verified for the user 2, so skipping
2024-10-08 03:58:08,199 DEBUG [c.c.a.ApiServer] (qtp1374754488-16:ctx-11ca45a7
ctx-2589f2f9) (logid:5bcd6d11) CIDRs from which account 'Account
[{"accountName":"admin","id":2,"uuid":"9995c13e-808d-11ef-8d51-005056a4ff14"}]'
is allowed to perform API calls: 0.0.0.0/0,::/0
2024-10-08 03:58:08,201 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker]
(qtp1374754488-16:ctx-11ca45a7 ctx-2589f2f9) (logid:5bcd6d11) RoleService is
enabled. We will use it instead of StaticRoleBasedAPIAccessChecker.
2024-10-08 03:58:08,201 DEBUG [o.a.c.r.ApiRateLimitServiceImpl]
(qtp1374754488-16:ctx-11ca45a7 ctx-2589f2f9) (logid:5bcd6d11) API rate limiting
is disabled. We will not use ApiRateLimitService.
2024-10-08 03:58:08,203 DEBUG [c.c.a.ApiServlet] (qtp1374754488-16:ctx-11ca45a7
ctx-2589f2f9) (logid:5bcd6d11) ===END=== x.x.x.x -- GET
command=readyForShutdown&response=json
2024-10-08 03:58:08,583 DEBUG [o.a.c.h.HAManagerImpl]
(BackgroundTaskPollManager-6:ctx-fb5dfdb9) (logid:ec45238d) HA health check
task is running...
2024-10-08 03:58:10,507 DEBUG [o.a.c.v.s.VMSchedulerImpl]
(VMSchedulerPollTask:ctx-26a3c9e8) (logid:fb4d56c5) VM scheduler.poll is being
called at 2024-10-08 03:58:00 GMT
2024-10-08 03:58:10,509 DEBUG [o.a.c.v.s.VMSchedulerImpl]
(VMSchedulerPollTask:ctx-26a3c9e8) (logid:fb4d56c5) Got 0 scheduled jobs to be
executed at 2024-10-08 03:58:00 GMT
2024-10-08 03:58:10,510 DEBUG [c.c.v.VirtualMachineManagerImpl]
(Vm-Operations-Cleanup-1:ctx-9bcf7c1d) (logid:37e7ebad) VM Operation Thread
Running
2024-10-08 03:58:10,510 INFO [o.a.c.v.s.VMSchedulerImpl]
(VMSchedulerPollTask:ctx-26a3c9e8) (logid:fb4d56c5) Cleaned up 0 VM scheduled
job entries
2024-10-08 03:58:10,864 DEBUG [o.a.c.c.ClusterDrsServiceImpl]
(VMSchedulerPollTask:ctx-c78b7db0) (logid:b6dd136a) ClusterDRS.poll is being
called at 2024-10-08 03:58:00 GMT
2024-10-08 03:58:10,869 DEBUG [o.a.c.c.ClusterDrsServiceImpl]
(VMSchedulerPollTask:ctx-c78b7db0) (logid:b6dd136a) Removed 0 old drs migration
plans
2024-10-08 03:58:12,584 DEBUG [o.a.c.h.HAManagerImpl]
(BackgroundTaskPollManager-6:ctx-049d2459) (logid:fa89dc81) HA health check
task is running...
2024-10-08 03:58:13,198 DEBUG [c.c.a.ApiServlet]
(qtp1374754488-20:ctx-38558ebf) (logid:d3629985) ===START=== x.x.x.x -- GET
command=readyForShutdown&response=json
2024-10-08 03:58:13,198 DEBUG [c.c.a.ApiServlet]
(qtp1374754488-20:ctx-38558ebf) (logid:d3629985) Two factor authentication is
already verified for the user 2, so skipping
2024-10-08 03:58:13,202 DEBUG [c.c.a.ApiServer] (qtp1374754488-20:ctx-38558ebf
ctx-473196ab) (logid:d3629985) CIDRs from which account 'Account
[{"accountName":"admin","id":2,"uuid":"9995c13e-808d-11ef-8d51-005056a4ff14"}]'
is allowed to perform API calls: 0.0.0.0/0,::/0
2024-10-08 03:58:13,205 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker]
(qtp1374754488-20:ctx-38558ebf ctx-473196ab) (logid:d3629985) RoleService is
enabled. We will use it instead of StaticRoleBasedAPIAccessChecker.
2024-10-08 03:58:13,205 DEBUG [o.a.c.r.ApiRateLimitServiceImpl]
(qtp1374754488-20:ctx-38558ebf ctx-473196ab) (logid:d3629985) API rate limiting
is disabled. We will not use ApiRateLimitService.
2024-10-08 03:58:13,206 DEBUG [c.c.a.ApiServlet] (qtp1374754488-20:ctx-38558ebf
ctx-473196ab) (logid:d3629985) ===END=== x.x.x.x -- GET
command=readyForShutdown&response=json
2024-10-08 03:58:16,585 DEBUG [o.a.c.h.HAManagerImpl]
(BackgroundTaskPollManager-1:ctx-3e103b21) (logid:ff5eb29b) HA health check
task is running...
2024-10-08 03:58:18,185 DEBUG [c.c.a.ApiServlet]
(qtp1374754488-20:ctx-8ca4f6cb) (logid:7f57b394) ===START=== x.x.x.x -- GET
command=readyForShutdown&response=json
2024-10-08 03:58:18,186 DEBUG [c.c.a.ApiServlet]
(qtp1374754488-20:ctx-8ca4f6cb) (logid:7f57b394) Two factor authentication is
already verified for the user 2, so skipping
2024-10-08 03:58:18,189 DEBUG [c.c.a.ApiServer] (qtp1374754488-20:ctx-8ca4f6cb
ctx-27cdc0e8) (logid:7f57b394) CIDRs from which account 'Account
[{"accountName":"admin","id":2,"uuid":"9995c13e-808d-11ef-8d51-005056a4ff14"}]'
is allowed to perform API calls: 0.0.0.0/0,::/0
2024-10-08 03:58:18,191 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker]
(qtp1374754488-20:ctx-8ca4f6cb ctx-27cdc0e8) (logid:7f57b394) RoleService is
enabled. We will use it instead of StaticRoleBasedAPIAccessChecker.
2024-10-08 03:58:18,191 DEBUG [o.a.c.r.ApiRateLimitServiceImpl]
(qtp1374754488-20:ctx-8ca4f6cb ctx-27cdc0e8) (logid:7f57b394) API rate limiting
is disabled. We will not use ApiRateLimitService.
2024-10-08 03:58:18,192 DEBUG [c.c.a.ApiServlet] (qtp1374754488-20:ctx-8ca4f6cb
ctx-27cdc0e8) (logid:7f57b394) ===END=== x.x.x.x -- GET
command=readyForShutdown&response=json
2024-10-08 03:58:20,586 DEBUG [o.a.c.h.HAManagerImpl]
(BackgroundTaskPollManager-1:ctx-6e808a10) (logid:54d5deba) HA health check
task is running...
2024-10-08 03:58:22,362 DEBUG [c.c.s.StatsCollector]
(StatsCollector-3:ctx-f5aa7a82) (logid:1e3291d5) VmStatsCollector is running to
process VMs across 0 UP hosts
2024-10-08 03:58:22,852 DEBUG [c.c.s.StatsCollector]
(StatsCollector-4:ctx-a5163f35) (logid:bc57c537) AutoScaling Monitor is
running...
2024-10-08 03:58:22,852 DEBUG [c.c.s.StatsCollector]
(StatsCollector-1:ctx-c9dbe77c) (logid:569e7a8b) StorageCollector is running...
2024-10-08 03:58:22,853 DEBUG [c.c.s.StatsCollector]
(StatsCollector-6:ctx-037ca908) (logid:4226b488) HostStatsCollector is running
to process 0 UP hosts
2024-10-08 03:58:23,186 DEBUG [c.c.a.ApiServlet]
(qtp1374754488-16:ctx-43402c73) (logid:f6c7d99c) ===START=== x.x.x.x -- GET
command=readyForShutdown&response=json
2024-10-08 03:58:23,186 DEBUG [c.c.a.ApiServlet]
(qtp1374754488-16:ctx-43402c73) (logid:f6c7d99c) Two factor authentication is
already verified for the user 2, so skipping
2024-10-08 03:58:23,190 DEBUG [c.c.a.ApiServer] (qtp1374754488-16:ctx-43402c73
ctx-628d3fa6) (logid:f6c7d99c) CIDRs from which account 'Account
[{"accountName":"admin","id":2,"uuid":"9995c13e-808d-11ef-8d51-005056a4ff14"}]'
is allowed to perform API calls: 0.0.0.0/0,::/0
2024-10-08 03:58:23,192 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker]
(qtp1374754488-16:ctx-43402c73 ctx-628d3fa6) (logid:f6c7d99c) RoleService is
enabled. We will use it instead of StaticRoleBasedAPIAccessChecker.
2024-10-08 03:58:23,192 DEBUG [o.a.c.r.ApiRateLimitServiceImpl]
(qtp1374754488-16:ctx-43402c73 ctx-628d3fa6) (logid:f6c7d99c) API rate limiting
is disabled. We will not use ApiRateLimitService.
2024-10-08 03:58:23,193 DEBUG [c.c.a.ApiServlet] (qtp1374754488-16:ctx-43402c73
ctx-628d3fa6) (logid:f6c7d99c) ===END=== x.x.x.x -- GET
command=readyForShutdown&response=json
2024-10-08 03:58:23,474 DEBUG [c.c.v.d.VmStatsDaoImpl]
(StatsCollector-5:ctx-cce905a0) (logid:d71b10bd) Starting to remove all
vm_stats rows older than [Mon Oct 07 15:58:23 UTC 2024].
2024-10-08 03:58:23,474 DEBUG [c.c.s.d.VolumeStatsDaoImpl]
(StatsCollector-2:ctx-45a5d6e5) (logid:2dcaa3bb) Starting to remove all
volume_stats rows older than [Mon Oct 07 15:58:23 UTC 2024].
2024-10-08 03:58:23,474 INFO [c.c.v.d.VmStatsDaoImpl]
(StatsCollector-5:ctx-cce905a0) (logid:d71b10bd) Removed a total of [0]
vm_stats rows older than [Mon Oct 07 15:58:23 UTC 2024].
2024-10-08 03:58:23,474 INFO [c.c.s.d.VolumeStatsDaoImpl]
(StatsCollector-2:ctx-45a5d6e5) (logid:2dcaa3bb) Removed a total of [0]
volume_stats rows older than [Mon Oct 07 15:58:23 UTC 2024].
2024-10-08 03:58:24,587 DEBUG [o.a.c.h.HAManagerImpl]
(BackgroundTaskPollManager-1:ctx-94d873b3) (logid:3fd55037) HA health check
task is running...
2024-10-08 03:58:28,024 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-e26978a2) (logid:21221808) Skip capacity scan as there is
no Primary Storage in 'Up' state
2024-10-08 03:58:28,188 DEBUG [c.c.a.ApiServlet]
(qtp1374754488-20:ctx-fce647a5) (logid:b0c9e2c9) ===START=== x.x.x.x -- GET
command=readyForShutdown&response=json
2024-10-08 03:58:28,188 DEBUG [c.c.a.ApiServlet]
(qtp1374754488-20:ctx-fce647a5) (logid:b0c9e2c9) Two factor authentication is
already verified for the user 2, so skipping
2024-10-08 03:58:28,192 DEBUG [c.c.a.ApiServer] (qtp1374754488-20:ctx-fce647a5
ctx-ed35d142) (logid:b0c9e2c9) CIDRs from which account 'Account
[{"accountName":"admin","id":2,"uuid":"9995c13e-808d-11ef-8d51-005056a4ff14"}]'
is allowed to perform API calls: 0.0.0.0/0,::/0
2024-10-08 03:58:28,194 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker]
(qtp1374754488-20:ctx-fce647a5 ctx-ed35d142) (logid:b0c9e2c9) RoleService is
enabled. We will use it instead of StaticRoleBasedAPIAccessChecker.
2024-10-08 03:58:28,194 DEBUG [o.a.c.r.ApiRateLimitServiceImpl]
(qtp1374754488-20:ctx-fce647a5 ctx-ed35d142) (logid:b0c9e2c9) API rate limiting
is disabled. We will not use ApiRateLimitService.
2024-10-08 03:58:28,195 DEBUG [c.c.a.ApiServlet] (qtp1374754488-20:ctx-fce647a5
ctx-ed35d142) (logid:b0c9e2c9) ===END=== x.x.x.x -- GET
command=readyForShutdown&response=json
2024-10-08 03:58:28,226 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-a2b4b642) (logid:5e54b63d) Enabled non-edge zones available
for scan:
2024-10-08 03:58:28,587 DEBUG [o.a.c.h.HAManagerImpl]
(BackgroundTaskPollManager-1:ctx-4ea857f0) (logid:c3de6d33) HA health check
task is running...
2024-10-08 03:58:32,588 DEBUG [o.a.c.h.HAManagerImpl]
(BackgroundTaskPollManager-1:ctx-ec6a2c91) (logid:7c6d84e2) HA health check
task is running...
2024-10-08 03:58:33,195 DEBUG [c.c.a.ApiServlet]
(qtp1374754488-16:ctx-aadf2ecb) (logid:a0b7bb76) ===START=== x.x.x.x -- GET
command=readyForShutdown&response=json
2024-10-08 03:58:33,195 DEBUG [c.c.a.ApiServlet]
(qtp1374754488-16:ctx-aadf2ecb) (logid:a0b7bb76) Two factor authentication is
already verified for the user 2, so skipping
2024-10-08 03:58:33,199 DEBUG [c.c.a.ApiServer] (qtp1374754488-16:ctx-aadf2ecb
ctx-03b1b2a0) (logid:a0b7bb76) CIDRs from which account 'Account
[{"accountName":"admin","id":2,"uuid":"9995c13e-808d-11ef-8d51-005056a4ff14"}]'
is allowed to perform API calls: 0.0.0.0/0,::/0
2024-10-08 03:58:33,201 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker]
(qtp1374754488-16:ctx-aadf2ecb ctx-03b1b2a0) (logid:a0b7bb76) RoleService is
enabled. We will use it instead of StaticRoleBasedAPIAccessChecker.
2024-10-08 03:58:33,202 DEBUG [o.a.c.r.ApiRateLimitServiceImpl]
(qtp1374754488-16:ctx-aadf2ecb ctx-03b1b2a0) (logid:a0b7bb76) API rate limiting
is disabled. We will not use ApiRateLimitService.
2024-10-08 03:58:33,203 DEBUG [c.c.a.ApiServlet] (qtp1374754488-16:ctx-aadf2ecb
ctx-03b1b2a0) (logid:a0b7bb76) ===END=== x.x.x.x -- GET
command=readyForShutdown&response=json
2024-10-08 03:58:35,697 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-cf97d28b) (logid:4a97aae5) Found 0 routers to update
status.
2024-10-08 03:58:35,697 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-cf97d28b) (logid:4a97aae5) Found 0 VPC's to update
Redundant State.
2024-10-08 03:58:35,698 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-cf97d28b) (logid:4a97aae5) Found 0 networks to
update RvR status.
2024-10-08 03:58:36,589 DEBUG [o.a.c.h.HAManagerImpl]
(BackgroundTaskPollManager-1:ctx-bce0b9da) (logid:7177fb9f) HA health check
task is running...
2024-10-08 03:58:37,647 DEBUG [c.c.u.s.SSHCmdHelper]
(qtp1374754488-18:ctx-98530a47 ctx-f1ee4153) (logid:af1051eb) SSH command: sudo
cloudstack-setup-agent -m <MGMT-IP> -z 2 -p 2 -c 1 -g
e47f5b77-147f-36d8-8619-1e67f220d5e1 -a -s --pubNic=GuestNet
--prvNic=MgmtNetVLAN600 --guestNic=GuestNet --hypervisor=kvm
SSH command output:Starting to configure your system:
Configure Host ... [OK]
Configure Apparmor ... [OK]
Configure Network ... [OK]
Configure Libvirt ... [OK]
Configure Firewall ... [OK]
Configure Nfs ... [OK]
Configure cloudAgent ... [OK]
CloudStack Agent setup is done!
2024-10-08 03:58:38,192 DEBUG [c.c.a.ApiServlet]
(qtp1374754488-20:ctx-11e6b25e) (logid:1e13b99b) ===START=== x.x.x.x -- GET
command=readyForShutdown&response=json
2024-10-08 03:58:38,192 DEBUG [c.c.a.ApiServlet]
(qtp1374754488-20:ctx-11e6b25e) (logid:1e13b99b) Two factor authentication is
already verified for the user 2, so skipping
2024-10-08 03:58:38,196 DEBUG [c.c.a.ApiServer] (qtp1374754488-20:ctx-11e6b25e
ctx-f122db45) (logid:1e13b99b) CIDRs from which account 'Account
[{"accountName":"admin","id":2,"uuid":"9995c13e-808d-11ef-8d51-005056a4ff14"}]'
is allowed to perform API calls: 0.0.0.0/0,::/0
2024-10-08 03:58:38,198 DEBUG [o.a.c.a.StaticRoleBasedAPIAccessChecker]
(qtp1374754488-20:ctx-11e6b25e ctx-f122db45) (logid:1e13b99b) RoleService is
enabled. We will use it instead of StaticRoleBasedAPIAccessChecker.
2024-10-08 03:58:38,198 DEBUG [o.a.c.r.ApiRateLimitServiceImpl]
(qtp1374754488-20:ctx-11e6b25e ctx-f122db45) (logid:1e13b99b) API rate limiting
is disabled. We will not use ApiRateLimitService.
2024-10-08 03:58:38,200 DEBUG [c.c.a.ApiServlet] (qtp1374754488-20:ctx-11e6b25e
ctx-f122db45) (logid:1e13b99b) ===END=== x.x.x.x -- GET
command=readyForShutdown&response=json
2024-10-08 03:58:38,842 DEBUG [o.a.c.c.p.RootCACustomTrustManager]
(pool-462-thread-1:null) (logid:) A client/agent attempting connection from
address=<KVM-Host-IP> has presented these certificate(s):
Certificate [1] :
Serial: 79f82e8eb188419f
Not Before:Mon Oct 07 15:58:00 UTC 2024
Not After:Wed Oct 08 03:58:00 UTC 2025
Signature Algorithm:SHA256withRSA
Version:3
Subject DN:C=cloudstack, O=cloudstack, OU=cloudstack, CN=<KVM-Host-FQDN>
Issuer DN:CN=ca.cloudstack.apache.org
Alternative Names:[[7, x.x.x.x], [7, <KVM-Host-IP>], [2, <KVM-Host-IP>], [2,
<KVM-Host-FQDN>]]
Certificate [2] :
Serial: 1466753dbfeb19c6
Not Before:Tue Oct 01 19:11:59 UTC 2024
Not After:Fri Sep 25 07:11:59 UTC 2054
Signature Algorithm:SHA256withRSA
Version:3
Subject DN:CN=ca.cloudstack.apache.org
Issuer DN:CN=ca.cloudstack.apache.org
Alternative Names:null
2024-10-08 03:58:38,844 DEBUG [o.a.c.c.p.RootCACustomTrustManager]
(pool-462-thread-1:null) (logid:) Client/agent connection from ip=<KVM-Host-IP>
has been validated and trusted.
2024-10-08 03:58:39,067 DEBUG [c.c.a.t.Request] (AgentManager-Handler-12:null)
(logid:) Seq -1-0: Scheduling the first command { Cmd , MgmtId: -1, via: -1,
Ver: v1, Flags: 1,
[{"com.cloud.agent.api.StartupRoutingCommand":{"cpuSockets":"4","cpus":"96","speed":"2400","memory":"810159267840","dom0MinMemory":"1073741824","poolSync":"false","supportsClonedVolumes":"false","caps":"hvm,snapshot","pool":"/root","hypervisorType":"KVM","hostDetails":{"Host.OS.Kernel.Version":"6.8.0-45-generic","com.cloud.network.Networks.RouterPrivateIpStrategy":"HostLocal","Host.OS.Version":"24.04","host.volume.encryption":"true","host.instance.conversion":"true","secured":"false","Host.OS":"Ubuntu"},"hostTags":[],"groupDetails":{},"type":"Routing","dataCenter":"2","pod":"2","cluster":"1","guid":"e47f5b77-147f-36d8-8619-1e67f220d5e1-LibvirtComputingResource","name":"<KVM-Host-FQDN>","version":"4.19.1.1","iqn":"iqn.2004-10.com.ubuntu:01:cf82f14213f1","privateIpAddress":"<KVM-Host-IP>","privateMacAddress":"b6:
78:ce:b7:1e:00","privateNetmask":"255.255.254.0","storageIpAddress":"<KVM-Host-IP>","storageNetmask":"255.255.254.0","storageMacAddress":"b6:78:ce:b7:1e:00","resourceName":"LibvirtComputingResource","gatewayIpAddress":"x.x.x.x","msHostList":"<MGMT-IP>","wait":"0","bypassHostMaintenance":"false"}},{"com.cloud.agent.api.StartupStorageCommand":{"totalSize":"(0
bytes)
0","poolInfo":{"uuid":"1e6b4fd8-7ce4-4237-8b37-f18962e18afc","host":"<KVM-Host-IP>","localPath":"/var/lib/libvirt/images","hostPath":"/var/lib/libvirt/images","poolType":"Filesystem","capacityBytes":"(436.01
GB) 468157972480","availableBytes":"(423.06 GB)
454255984640"},"resourceType":"STORAGE_POOL","hostDetails":{},"type":"Storage","dataCenter":"2","pod":"2","guid":"e47f5b77-147f-36d8-8619-1e67f220d5e1-LibvirtComputingResource","name":"<KVM-Host-FQDN>","version":"4.19.1.1","resourceName":"LibvirtComputingResource","msHostList":"<MGMT-IP>","wait":"0","bypassHostMaintenance":"false"}}]
}
2024-10-08 03:58:39,069 DEBUG [c.c.a.t.Request]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Seq -1-0: Processing
the first command { Cmd , MgmtId: -1, via: -1, Ver: v1, Flags: 1,
[{"com.cloud.agent.api.StartupRoutingCommand":{"cpuSockets":"4","cpus":"96","speed":"2400","memory":"810159267840","dom0MinMemory":"1073741824","poolSync":"false","supportsClonedVolumes":"false","caps":"hvm,snapshot","pool":"/root","hypervisorType":"KVM","hostDetails":{"Host.OS.Kernel.Version":"6.8.0-45-generic","com.cloud.network.Networks.RouterPrivateIpStrategy":"HostLocal","Host.OS.Version":"24.04","host.volume.encryption":"true","host.instance.conversion":"true","secured":"false","Host.OS":"Ubuntu"},"hostTags":[],"groupDetails":{},"type":"Routing","dataCenter":"2","pod":"2","cluster":"1","guid":"e47f5b77-147f-36d8-8619-1e67f220d5e1-LibvirtComputingResource","name":"<KVM-Host-FQDN>","version":"4.19.1.1","iqn":"iqn.2004-10.com.ubuntu:01:cf82f14213f1","privateIpAddress":"<KVM-Host-IP>","privat
eMacAddress":"b6:78:ce:b7:1e:00","privateNetmask":"255.255.254.0","storageIpAddress":"<KVM-Host-IP>","storageNetmask":"255.255.254.0","storageMacAddress":"b6:78:ce:b7:1e:00","resourceName":"LibvirtComputingResource","gatewayIpAddress":"x.x.x.x","msHostList":"<MGMT-IP>","wait":"0","bypassHostMaintenance":"false"}},{"com.cloud.agent.api.StartupStorageCommand":{"totalSize":"(0
bytes)
0","poolInfo":{"uuid":"1e6b4fd8-7ce4-4237-8b37-f18962e18afc","host":"<KVM-Host-IP>","localPath":"/var/lib/libvirt/images","hostPath":"/var/lib/libvirt/images","poolType":"Filesystem","capacityBytes":"(436.01
GB) 468157972480","availableBytes":"(423.06 GB)
454255984640"},"resourceType":"STORAGE_POOL","hostDetails":{},"type":"Storage","dataCenter":"2","pod":"2","guid":"e47f5b77-147f-36d8-8619-1e67f220d5e1-LibvirtComputingResource","name":"<KVM-Host-FQDN>","version":"4.19.1.1","resourceName":"LibvirtComputingResource","msHostList":"<MGMT-IP>","wait":"0","bypassHostMaintenance":"false"}}]
}
2024-10-08 03:58:39,071 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Dispatching resource
state event CREATE_HOST_VO_FOR_CONNECTED to CiscoVnmcElement
2024-10-08 03:58:39,071 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Dispatching resource
state event CREATE_HOST_VO_FOR_CONNECTED to BareMetalDiscoverer
2024-10-08 03:58:39,071 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Dispatching resource
state event CREATE_HOST_VO_FOR_CONNECTED to NetscalerElement
2024-10-08 03:58:39,071 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Dispatching resource
state event CREATE_HOST_VO_FOR_CONNECTED to HypervServerDiscoverer
2024-10-08 03:58:39,071 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Dispatching resource
state event CREATE_HOST_VO_FOR_CONNECTED to BaremetalPxeManagerImpl
2024-10-08 03:58:39,071 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Dispatching resource
state event CREATE_HOST_VO_FOR_CONNECTED to XcpServerDiscoverer
2024-10-08 03:58:39,071 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Dispatching resource
state event CREATE_HOST_VO_FOR_CONNECTED to NiciraNvp
2024-10-08 03:58:39,071 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Dispatching resource
state event CREATE_HOST_VO_FOR_CONNECTED to BrocadeVcsElement
2024-10-08 03:58:39,071 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Dispatching resource
state event CREATE_HOST_VO_FOR_CONNECTED to Ovm3Discoverer
2024-10-08 03:58:39,071 DEBUG [c.c.h.o.r.Ovm3Discoverer]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43)
createHostVOForConnectedAgent: Host
{"id":0,"name":"<KVM-Host-FQDN>","type":null,"uuid":"d5022935-187b-4714-8067-4d39cdf0345b"}
2024-10-08 03:58:39,072 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Dispatching resource
state event CREATE_HOST_VO_FOR_CONNECTED to LxcServerDiscoverer
2024-10-08 03:58:39,072 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Dispatching resource
state event CREATE_HOST_VO_FOR_CONNECTED to NetworkUsageManagerImpl
2024-10-08 03:58:39,072 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Dispatching resource
state event CREATE_HOST_VO_FOR_CONNECTED to PremiumSecondaryStorageManagerImpl
2024-10-08 03:58:39,072 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Dispatching resource
state event CREATE_HOST_VO_FOR_CONNECTED to Ovs
2024-10-08 03:58:39,072 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Dispatching resource
state event CREATE_HOST_VO_FOR_CONNECTED to VmwareServerDiscoverer
2024-10-08 03:58:39,072 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Dispatching resource
state event CREATE_HOST_VO_FOR_CONNECTED to ConsoleProxyManagerImpl
2024-10-08 03:58:39,072 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Dispatching resource
state event CREATE_HOST_VO_FOR_CONNECTED to OvmDiscoverer
2024-10-08 03:58:39,072 DEBUG [c.c.r.ResourceManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Dispatching resource
state event CREATE_HOST_VO_FOR_CONNECTED to KvmServerDiscoverer
2024-10-08 03:58:39,088 DEBUG [c.c.r.ResourceState]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Resource state update:
[id = 15; name = <KVM-Host-FQDN>; old state = Creating; event =
InternalCreated; new state = Enabled]
2024-10-08 03:58:39,088 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Transition:[Resource
state = Enabled, Agent event = AgentConnected, Host id = 15, name =
<KVM-Host-FQDN>]
2024-10-08 03:58:39,096 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Agent's management
server host list is not up to date, sending list update:[<MGMT-IP>]
2024-10-08 03:58:39,096 DEBUG [c.c.a.m.ClusteredAgentManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) create
ClusteredAgentAttache for 15
2024-10-08 03:58:39,096 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Sending Connect to
listener: XcpServerDiscoverer
2024-10-08 03:58:39,096 DEBUG [c.c.h.x.d.XcpServerDiscoverer]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Not XenServer so
moving on.
2024-10-08 03:58:39,097 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Sending Connect to
listener: HypervServerDiscoverer
2024-10-08 03:58:39,097 DEBUG [c.c.h.h.d.HypervServerDiscoverer]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Not Hyper-V
hypervisor, so moving on.
2024-10-08 03:58:39,097 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Sending Connect to
listener: TungstenElement
2024-10-08 03:58:39,097 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Sending Connect to
listener: SecondaryStorageListener
2024-10-08 03:58:39,097 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Sending Connect to
listener: SecurityGroupListener
2024-10-08 03:58:39,097 INFO [c.c.n.s.SecurityGroupListener]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Received a host
startup notification
2024-10-08 03:58:39,098 DEBUG [c.c.a.t.Request]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Seq
15-5367727805872209921: Sending { Cmd , MgmtId: 345051037460, via:
15(<KVM-Host-FQDN>), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.CleanupNetworkRulesCmd":{"interval":"2529","wait":"0","bypassHostMaintenance":"false"}}]
}
2024-10-08 03:58:39,098 INFO [c.c.n.s.SecurityGroupListener]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Scheduled network
rules cleanup, interval=2529
2024-10-08 03:58:39,098 INFO [c.c.n.s.SecurityGroupListener]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Received a host
startup notification
2024-10-08 03:58:39,098 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Sending Connect to
listener: StoragePoolMonitor
2024-10-08 03:58:39,101 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Sending Connect to
listener: NetworkOrchestrator
2024-10-08 03:58:39,102 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Host's hypervisorType
is: KVM
2024-10-08 03:58:39,106 DEBUG [o.a.c.e.o.NetworkOrchestrator]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Sending
CheckNetworkCommand to check the Network is setup correctly on Agent
2024-10-08 03:58:39,108 DEBUG [c.c.a.t.Request]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Seq
15-5367727805872209922: Sending { Cmd , MgmtId: 345051037460, via:
15(<KVM-Host-FQDN>), Ver: v1, Flags: 100111,
[{"com.cloud.agent.api.CheckNetworkCommand":{"networkInfoList":[{"physicalNetworkId":"203","privateNetworkName":"MgmtNetVLAN600"},{"physicalNetworkId":"204","storageNetworkName":"NFSNetVLAN602"},{"physicalNetworkId":"210","publicNetworkName":"GuestNet","guestNetworkName":"GuestNet"}],"wait":"0","bypassHostMaintenance":"false"}}]
}
2024-10-08 03:58:39,109 DEBUG [c.c.a.t.Request] (AgentManager-Handler-14:null)
(logid:) Seq 15-5367727805872209921: Processing: { Ans: , MgmtId:
345051037460, via: 15, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":"true","wait":"0","bypassHostMaintenance":"false"}}]
}
2024-10-08 03:58:39,220 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-13:null) (logid:) Ping from Routing host
15(<KVM-Host-FQDN>)
2024-10-08 03:58:39,220 DEBUG [c.c.a.t.Request] (AgentManager-Handler-15:null)
(logid:) Seq 15-5367727805872209922: Processing: { Ans: , MgmtId:
345051037460, via: 15, Ver: v1, Flags: 110,
[{"com.cloud.agent.api.CheckNetworkAnswer":{"_reconnect":"false","result":"false","details":"Can
not find network:
MgmtNetVLAN600","wait":"0","bypassHostMaintenance":"false"}}] }
2024-10-08 03:58:39,220 DEBUG [c.c.a.t.Request]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Seq
15-5367727805872209922: Received: { Ans: , MgmtId: 345051037460, via:
15(<KVM-Host-FQDN>), Ver: v1, Flags: 110, { CheckNetworkAnswer } }
2024-10-08 03:58:39,221 DEBUG [c.c.a.m.AgentAttache]
(AgentManager-Handler-15:null) (logid:) Seq 15-5367727805872209922: No more
commands found
2024-10-08 03:58:39,221 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-13:null) (logid:) Process host VM state report from ping
process. host: 15
2024-10-08 03:58:39,221 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Details from executing
class com.cloud.agent.api.CheckNetworkCommand: Can not find network:
MgmtNetVLAN600
2024-10-08 03:58:39,221 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-13:null) (logid:) Process VM state report. host: 15,
number of records in report: 0
2024-10-08 03:58:39,221 WARN [o.a.c.e.o.NetworkOrchestrator]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Unable to setup agent
15 due to Can not find network: MgmtNetVLAN600
2024-10-08 03:58:39,221 WARN [c.c.a.AlertManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) alertType=[7]
dataCenterId=[2] podId=[2] clusterId=[null] message=[Incorrect Network setup on
agent, Reinitialize agent after network names are setup, details : Can not find
network: MgmtNetVLAN600].
2024-10-08 03:58:39,222 DEBUG [c.c.v.VirtualMachinePowerStateSyncImpl]
(AgentManager-Handler-13:null) (logid:) Done with process of VM state report.
host: 15
2024-10-08 03:58:39,223 INFO [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-13:null) (logid:) PingMap for agent: 15 will not be
updated because agent is no longer in the PingMap
2024-10-08 03:58:39,226 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-13:null) (logid:) Not processing PingRoutingCommand for
agent id=0; can't find the host in the DB
2024-10-08 03:58:39,228 WARN [c.c.a.AlertManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) No recipients set in
global setting 'alert.email.addresses', skipping sending alert with subject
[Incorrect Network setup on agent, Reinitialize agent after network names are
setup, details : Can not find network: MgmtNetVLAN600] and content [Incorrect
Network setup on agent, Reinitialize agent after network names are setup,
details : Can not find network: MgmtNetVLAN600].
2024-10-08 03:58:39,228 INFO [c.c.u.e.CSExceptionErrorCode]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Could not find
exception: com.cloud.exception.ConnectionException in error code list for
exceptions
2024-10-08 03:58:39,228 WARN [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-456:ctx-ac8f76a7) (logid:ee7c4c43) Monitor
NetworkOrchestrator says there is an error in the connect process for 15 due to
Incorrect Network setup on agent, Reinitialize agent after network names are
setup, details : Can not find network: MgmtNetVLAN600
~~~
GitHub link: https://github.com/apache/cloudstack/discussions/9781
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
