Is it safe to force remove the host? On Thu, May 20, 2021 at 4:12 PM Fariborz Navidan <mdvlinqu...@gmail.com> wrote:
> All VMs show as stopped but "virsh list" show dp,aoms rimmomh > > > On Thu, May 20, 2021 at 1:20 PM Yordan Kostov <yord...@nsogroup.com> > wrote: > >> Hello Fariborz, >> >> I am not experienced with ACS + KVM but here is something that >> may help. >> If host is empty of VMs, can you force-remove it and add it again? >> >> Regards, >> Jordan >> >> -----Original Message----- >> From: Fariborz Navidan <mdvlinqu...@gmail.com> >> Sent: Thursday, May 20, 2021 11:23 AM >> To: users@cloudstack.apache.org >> Subject: Re: Change host IP >> >> >> [X] This message came from outside your organization >> >> >> When I restart agent the old IP gets back to agent.properties >> >> Please help[ me/ >> >> On Thu, May 20, 2021 at 2:42 AM Fariborz Navidan <mdvlinqu...@gmail.com> >> wrote: >> >> > I use cloudstack 4.14.0 >> > >> > Please help me >> > >> > On Thu, May 20, 2021 at 2:34 AM Fariborz Navidan >> > <mdvlinqu...@gmail.com> >> > wrote: >> > >> >> Hello, >> >> >> >> We have migrated our server and as a result, we need to change the IP >> >> address. I have modified agent.properties and all ip address fields >> >> of host record to match the new IP address. But unfortunately, >> >> management server is unable to connect to it. I forced the host to go >> >> for maintenance checking "force". Now when I disable maintenance mode >> >> I get error " Failed to prepare host for maintenance due to: Unable >> >> to prepare for maintenance host 1". >> >> >> >> Bellow is the agent log: >> >> >> >> 2021-05-19 23:57:41,985 WARN [utils.nio.NioConnection] (main:null) >> >> (logid:) Unable to connect to remote: is there a server running on >> >> port 8250 >> >> 2021-05-19 23:57:46,986 INFO [cloud.agent.Agent] (main:null) >> >> (logid:) Connecting to host:79.143.86.149 >> >> 2021-05-19 23:57:46,987 INFO [utils.nio.NioClient] (ma >> >> >> >> Bellow is management server log: >> >> >> >> ] (Cluster-Heartbeat-1:ctx-f4e82537) (logid:c1947b66) Management node >> >> 11 is detected inactive by timestamp but is pingable >> >> 2021-05-20 00:02:52,857 DEBUG [c.c.a.ApiServlet] >> >> (qtp1609124502-279:ctx-17009f08) (logid:7e8b8325) ===START=== >> >> 2.190.171.255 -- GET >> >> >> >> command=prepareHostForMaintenance&id=49bbd35c-9e33-48a5-ba38-41452429 >> >> 986b&response=json&_=1621461469563 >> >> 2021-05-20 00:02:52,870 DEBUG [c.c.a.ApiServer] >> >> (qtp1609124502-279:ctx-17009f08 ctx-e209aafe) (logid:7e8b8325) CIDRs >> >> from which account 'Acct[97f4b3a5-a631-11ea-84fb-002590c1a7d4-admin]' >> >> is allowed to perform API calls: 0.0.0.0/0,::/0 >> >> 2021-05-20 00:02:52,893 INFO [o.a.c.f.j.i.AsyncJobMonitor] >> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:176b8386) Add >> >> job-594 into job monitoring >> >> 2021-05-20 00:02:52,897 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> >> (qtp1609124502-279:ctx-17009f08 ctx-e209aafe) (logid:7e8b8325) submit >> >> async job-594, details: AsyncJobVO {id:594, userId: 2, accountId: 2, >> >> instanceType: Host, instanceId: 1, cmd: >> >> org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd >> >> , >> >> cmdInfo: >> >> {"response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventI >> >> d":"23778","id":"49bbd35c-9e33-48a5-ba38-41452429986b","ctxDetails":" >> >> {\"interface >> >> com.cloud.host.Host\":\"49bbd35c-9e33-48a5-ba38-41452429986b\"}","ctx >> >> AccountId":"2","uuid":"49bbd35c-9e33-48a5-ba38-41452429986b","cmdEven >> >> tType":"MAINT.PREPARE","_":"1621461469563"}, >> >> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, >> >> result: null, initMsid: 279278805467174, completeMsid: null, >> lastUpdated: >> >> null, lastPolled: null, created: null, removed: null} >> >> 2021-05-20 00:02:52,899 DEBUG [c.c.a.ApiServlet] >> >> (qtp1609124502-279:ctx-17009f08 ctx-e209aafe) (logid:7e8b8325) >> >> ===END=== >> >> 2.190.171.255 -- GET >> >> >> >> command=prepareHostForMaintenance&id=49bbd35c-9e33-48a5-ba38-41452429 >> >> 986b&response=json&_=1621461469563 >> >> 2021-05-20 00:02:52,899 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Executing >> >> AsyncJobVO {id:594, userId: 2, accountId: 2, instanceType: Host, >> >> instanceId: 1, cmd: >> >> org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd >> >> , >> >> cmdInfo: >> >> {"response":"json","ctxUserId":"2","httpmethod":"GET","ctxStartEventI >> >> d":"23778","id":"49bbd35c-9e33-48a5-ba38-41452429986b","ctxDetails":" >> >> {\"interface >> >> com.cloud.host.Host\":\"49bbd35c-9e33-48a5-ba38-41452429986b\"}","ctx >> >> AccountId":"2","uuid":"49bbd35c-9e33-48a5-ba38-41452429986b","cmdEven >> >> tType":"MAINT.PREPARE","_":"1621461469563"}, >> >> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, >> >> result: null, initMsid: 279278805467174, completeMsid: null, >> lastUpdated: >> >> null, lastPolled: null, created: null, removed: null} >> >> 2021-05-20 00:02:52,924 INFO [c.c.r.ResourceManagerImpl] >> >> (API-Job-Executor-8:ctx-4599c29b job-594 ctx-50530646) >> >> (logid:33d9ad1e) >> >> Maintenance: attempting maintenance of host >> >> 49bbd35c-9e33-48a5-ba38-41452429986b >> >> 2021-05-20 00:02:52,926 DEBUG [c.c.a.m.AgentManagerImpl] >> >> (API-Job-Executor-8:ctx-4599c29b job-594 ctx-50530646) >> >> (logid:33d9ad1e) Can not send command >> >> com.cloud.agent.api.MaintainCommand due to Host 1 is not up >> >> 2021-05-20 00:02:52,926 WARN [c.c.r.ResourceManagerImpl] >> >> (API-Job-Executor-8:ctx-4599c29b job-594 ctx-50530646) >> >> (logid:33d9ad1e) Unable to send MaintainCommand to host: 1 >> >> 2021-05-20 00:02:52,928 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Complete >> >> async job-594, jobStatus: FAILED, resultCode: 530, result: >> >> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList" >> >> :[],"errorcode":530,"errortext":"Failed >> >> to prepare host for maintenance due to: Unable to prepare for >> >> maintenance host 1"} >> >> 2021-05-20 00:02:52,930 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Publish >> >> async >> >> job-594 complete on message bus >> >> 2021-05-20 00:02:52,930 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Wake up >> >> jobs related to job-594 >> >> 2021-05-20 00:02:52,930 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Update db >> >> status for job-594 >> >> 2021-05-20 00:02:52,932 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Wake up >> >> jobs joined with job-594 and disjoin all subjobs created from job- >> >> 594 >> >> 2021-05-20 00:02:52,939 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] >> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Done >> >> executing >> >> org.apache.cloudstack.api.command.admin.host.PrepareForMaintenanceCmd >> >> for >> >> job-594 >> >> 2021-05-20 00:02:52,939 INFO [o.a.c.f.j.i.AsyncJobMonitor] >> >> (API-Job-Executor-8:ctx-4599c29b job-594) (logid:33d9ad1e) Remove >> >> job-594 from job monitoring >> >> 2021-05-20 00:02:53,709 DEBUG [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-2481f4ba) (logid:946491f3) Detected >> >> management node left, id:11, nodeIP:169.254.0.1 >> >> 2021-05-20 00:02:53,709 INFO [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-2481f4ba) (logid:946491f3) Trying to connect >> >> to >> >> 169.254.0.1 >> >> 2021-05-20 00:02:53,709 INFO [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-2481f4ba) (logid:946491f3) Management node >> >> 11 is detected inactive by timestamp but is pingable >> >> 2021-05-20 00:02:55,209 DEBUG [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-dfca10b1) (logid:43b23998) Detected >> >> management node left, id:11, nodeIP:169.254.0.1 >> >> 2021-05-20 00:02:55,209 INFO [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-dfca10b1) (logid:43b23998) Trying to connect >> >> to >> >> 169.254.0.1 >> >> 2021-05-20 00:02:55,210 INFO [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-dfca10b1) (logid:43b23998) Management node >> >> 11 is detected inactive by timestamp but is pingable >> >> 2021-05-20 00:02:55,968 DEBUG [c.c.a.ApiServlet] >> >> (qtp1609124502-308:ctx-ef5fd0c0) (logid:ac2a9f7e) ===START=== >> >> 2.190.171.255 -- GET >> >> >> >> command=queryAsyncJobResult&jobId=33d9ad1e-40fb-43db-bfb2-4047d386e9a >> >> d&response=json&_=1621461469564 >> >> 2021-05-20 00:02:55,981 DEBUG [c.c.a.ApiServer] >> >> (qtp1609124502-308:ctx-ef5fd0c0 ctx-1484d35d) (logid:ac2a9f7e) CIDRs >> >> from which account 'Acct[97f4b3a5-a631-11ea-84fb-002590c1a7d4-admin]' >> >> is allowed to perform API calls: 0.0.0.0/0,::/0 >> >> 2021-05-20 00:02:56,005 DEBUG [c.c.a.ApiServlet] >> >> (qtp1609124502-308:ctx-ef5fd0c0 ctx-1484d35d) (logid:ac2a9f7e) >> >> ===END=== >> >> 2.190.171.255 -- GET >> >> >> >> command=queryAsyncJobResult&jobId=33d9ad1e-40fb-43db-bfb2-4047d386e9a >> >> d&response=json&_=1621461469564 >> >> 2021-05-20 00:02:56,708 DEBUG [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-474baae1) (logid:2eea24af) Detected >> >> management node left, id:11, nodeIP:169.254.0.1 >> >> 2021-05-20 00:02:56,708 INFO [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-474baae1) (logid:2eea24af) Trying to connect >> >> to >> >> 169.254.0.1 >> >> 2021-05-20 00:02:56,708 INFO [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-474baae1) (logid:2eea24af) Management node >> >> 11 is detected inactive by timestamp but is pingable >> >> 2021-05-20 00:02:56,948 DEBUG [c.c.s.StatsCollector] >> >> (StatsCollector-3:ctx-a25c34d9) (logid:a111f62a) AutoScaling Monitor >> >> is running... >> >> 2021-05-20 00:02:56,971 DEBUG [c.c.s.StatsCollector] >> >> (StatsCollector-5:ctx-43fd4a09) (logid:2ea9d48e) HostStatsCollector >> >> is running... >> >> 2021-05-20 00:02:57,173 DEBUG [c.c.s.StatsCollector] >> >> (StatsCollector-4:ctx-191f8643) (logid:c0c4dc4d) StorageCollector is >> >> running... >> >> 2021-05-20 00:02:57,182 DEBUG [c.c.s.StatsCollector] >> >> (StatsCollector-4:ctx-191f8643) (logid:c0c4dc4d) There is no >> >> secondary storage VM for secondary storage host Secondary1 >> >> 2021-05-20 00:02:58,208 DEBUG [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-3a4a4f30) (logid:ffd0426d) Detected >> >> management node left, id:11, nodeIP:169.254.0.1 >> >> 2021-05-20 00:02:58,208 INFO [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-3a4a4f30) (logid:ffd0426d) Trying to connect >> >> to >> >> 169.254.0.1 >> >> 2021-05-20 00:02:58,208 INFO [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-3a4a4f30) (logid:ffd0426d) Management node >> >> 11 is detected inactive by timestamp but is pingable >> >> 2021-05-20 00:02:59,707 DEBUG [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-71c53c5d) (logid:62a78da5) Detected >> >> management node left, id:11, nodeIP:169.254.0.1 >> >> 2021-05-20 00:02:59,707 INFO [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-71c53c5d) (logid:62a78da5) Trying to connect >> >> to >> >> 169.254.0.1 >> >> 2021-05-20 00:02:59,707 INFO [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-71c53c5d) (logid:62a78da5) Management node >> >> 11 is detected inactive by timestamp but is pingable >> >> 2021-05-20 00:03:01,052 DEBUG [c.c.c.ConsoleProxyManagerImpl] >> >> (consoleproxy-1:ctx-5d2e0d48) (logid:1252bec0) Zone 2 is not ready to >> >> launch console proxy yet >> >> 2021-05-20 00:03:01,064 DEBUG [o.a.c.s.SecondaryStorageManagerImpl] >> >> (secstorage-1:ctx-e131be81) (logid:15af42a6) Zone 2 is not ready to >> >> launch secondary storage VM yet >> >> 2021-05-20 00:03:01,207 DEBUG [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-3b1d4af2) (logid:d2f407cc) Detected >> >> management node left, id:11, nodeIP:169.254.0.1 >> >> 2021-05-20 00:03:01,207 INFO [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-3b1d4af2) (logid:d2f407cc) Trying to connect >> >> to >> >> 169.254.0.1 >> >> 2021-05-20 00:03:01,208 INFO [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-3b1d4af2) (logid:d2f407cc) Management node >> >> 11 is detected inactive by timestamp but is pingable >> >> 2021-05-20 00:03:02,707 DEBUG [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-905dc01e) (logid:9043c478) Detected >> >> management node left, id:11, nodeIP:169.254.0.1 >> >> 2021-05-20 00:03:02,708 INFO [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-905dc01e) (logid:9043c478) Trying to connect >> >> to >> >> 169.254.0.1 >> >> 2021-05-20 00:03:02,708 INFO [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-905dc01e) (logid:9043c478) Management node >> >> 11 is detected inactive by timestamp but is pingable >> >> 2021-05-20 00:03:04,206 DEBUG [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-29bd1b3b) (logid:0823e425) Detected >> >> management node left, id:11, nodeIP:169.254.0.1 >> >> 2021-05-20 00:03:04,206 INFO [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-29bd1b3b) (logid:0823e425) Trying to connect >> >> to >> >> 169.254.0.1 >> >> 2021-05-20 00:03:04,206 INFO [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-29bd1b3b) (logid:0823e425) Management node >> >> 11 is detected inactive by timestamp but is pingable >> >> 2021-05-20 00:03:05,708 DEBUG [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-a85c2e4c) (logid:2b511c6a) Detected >> >> management node left, id:11, nodeIP:169.254.0.1 >> >> 2021-05-20 00:03:05,708 INFO [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-a85c2e4c) (logid:2b511c6a) Trying to connect >> >> to >> >> 169.254.0.1 >> >> 2021-05-20 00:03:05,708 INFO [c.c.c.ClusterManagerImpl] >> >> (Cluster-Heartbeat-1:ctx-a85c2e4c) (logid:2b511c6a) Management node >> >> 11 is detected inactive by timestamp but is pingable >> >> [root@ir-kvm1 management]# >> >> >> >> >> >