Steps to remove the host: * If this is the only one host in the cluster, add one more host so the existing vms can migrate there * put the host you want to remove into Maintenance. * Once maintenance is completed, delete the host.
-Alena. On 7/16/12 8:09 PM, "Evan Miller" <[email protected]> wrote: >FYI: From the GUI, I noticed that the cluster for the only host has an >undefined State. >FYI: From the GUI, the only host has a State of Up. > >First ... > >I stopped the cloud-management service. >I dumped the cloud DB: > >[root@cumulus management]# mysqldump -u root -p cloud > /tmp/cloud.sql >Enter password: >[root@cumulus management]# ls -alg /tmp/cloud.sql >-rw-r--r--. 1 root root 401072 Jul 16 19:35 /tmp/cloud.sql >[root@cumulus management]# > >Got the ids for the VMs ... > >mysql> Select * from vm_instance where state in ('Running', 'Starting', >'Stopping') AND host_id in (select id from host where removed is not >null); >+----+--------+--------------------------------------+---------------+---- >-----+----------------+-------------+---------------------+--------------- >-----+--------+----------------+---------+--------------+----------+------ >-------------+----------------------------------------------+------------+ >---------------+--------------+---------------------+--------------------- >+---------+--------------------+--------------------+------------+-------- >---+---------------------+--------------------------------------+--------- >--------+ >| id | name | uuid | instance_name | >state | vm_template_id | guest_os_id | private_mac_address | >private_ip_address | pod_id | data_center_id | host_id | last_host_id | >proxy_id | proxy_assign_time | vnc_password > | ha_enabled | limit_cpu_use | update_count | update_time | >created | removed | type | vm_type | >account_id | domain_id | service_offering_id | reservation_id > | hypervisor_type | >+----+--------+--------------------------------------+---------------+---- >-----+----------------+-------------+---------------------+--------------- >-----+--------+----------------+---------+--------------+----------+------ >-------------+----------------------------------------------+------------+ >---------------+--------------+---------------------+--------------------- >+---------+--------------------+--------------------+------------+-------- >---+---------------------+--------------------------------------+--------- >--------+ >| 1 | s-1-VM | 4780c7ba-dd55-45a2-a2e5-ed2974c070b5 | s-1-VM | >Running | 1 | 15 | 06:9e:ec:00:00:07 | >10.217.5.199 | 1 | 1 | 1 | 1 | > NULL | NULL | caqFANzAxnvRMnVMJeajvIY4x3W8NMjFUkj/RkQmhSU= >| 0 | 0 | 3 | 2012-07-14 00:02:27 | >2012-07-13 23:59:42 | NULL | SecondaryStorageVm | SecondaryStorageVm | > 1 | 1 | 6 | >e62a93bd-9bb0-4640-8a89-dc1049f300fa | XenServer | >| 2 | v-2-VM | 79924f6e-66e2-429f-b38a-9d3bd3f6a575 | v-2-VM | >Running | 1 | 15 | 06:31:80:00:00:06 | >10.217.5.198 | 1 | 1 | 1 | 1 | > NULL | NULL | cF8VnXqcjTMS8dUfURYlShHJhV55iUFKMyyEKA/SNhw= >| 0 | 0 | 3 | 2012-07-14 00:04:19 | >2012-07-13 23:59:42 | NULL | ConsoleProxy | ConsoleProxy | > 1 | 1 | 8 | >c1e4d053-95c6-4804-b73f-c38e7e4e887a | XenServer | >| 4 | r-4-VM | 593d5b00-9c73-485f-b091-121b69171e65 | r-4-VM | >Running | 1 | 15 | 0e:00:a9:fe:00:88 | >169.254.0.136 | 1 | 1 | 1 | 1 | > NULL | NULL | AZ69gd1Mp6IO4gt0AWAeAQawEXZtjh9mA7cQ6ZCDcJg= >| 1 | 0 | 3 | 2012-07-14 01:53:39 | >2012-07-14 01:52:48 | NULL | DomainRouter | DomainRouter | > 1 | 1 | 7 | >4f69471a-c0f0-48d2-a39e-3dda20b20c56 | XenServer | >+----+--------+--------------------------------------+---------------+---- >-----+----------------+-------------+---------------------+--------------- >-----+--------+----------------+---------+--------------+----------+------ >-------------+----------------------------------------------+------------+ >---------------+--------------+---------------------+--------------------- >+---------+--------------------+--------------------+------------+-------- >---+---------------------+--------------------------------------+--------- >--------+ >3 rows in set (0.00 sec) > >mysql> Select id, name, state from vm_instance where state in ('Running', >'Starting', 'Stopping') AND host_id in (select id from host where removed >is not null); >+----+--------+---------+ >| id | name | state | >+----+--------+---------+ >| 1 | s-1-VM | Running | >| 2 | v-2-VM | Running | >| 4 | r-4-VM | Running | >+----+--------+---------+ >3 rows in set (0.00 sec) > >mysql> > >Next ... > >Update vm_instance set state='Stopped', host_id=null where id in('1', >'2', '4'); > >mysql> Update vm_instance set state='Stopped', host_id=null where id >in('1', '2', '4'); >Query OK, 3 rows affected (0.06 sec) >Rows matched: 3 Changed: 3 Warnings: 0 > >mysql> > >Restarted CSMS ... > >[root@cumulus management]# service cloud-management start >Starting cloud-management: [ OK ] >[root@cumulus management]# > >From the GUI, started the two system VMs ... > >From the API (no other way), started the Router ... > >http://10.217.5.192:8080/client/api?apikey=iFl88lw1Pk6gKqUIFPN8vzZbJNsUVdY >GIJKBTEXtrymcIH5UWp9VHjgnpP_zCmaucmi8XmwK75TR70z-2ayjGA&command=startR >outer&id=593d5b00-9c73-485f-b091-121b69171e65&response=json&signature=sx8e >tg9c1W5o%2BV7L2JUr1Bn%2BfEk%3D > >{ "listroutersresponse" : { "count":1 ,"router" : [ >{"id":"593d5b00-9c73-485f-b091-121b69171e65","zoneid":"9bf663f5-167a-4152- >b80c-a1fb905e01ef ... etc} ] } } > >From the API, verified the Router had started ... > >http://10.217.5.192:8080/client/api?apikey=iFl88lw1Pk6gKqUIFPN8vzZbJNsUVdY >GIJKBTEXtrymcIH5UWp9VHjgnpP_zCmaucmi8XmwK75TR70z-2ayjGA&command=listRouter >s&listall=true&response=json&signature=aQWLi1pJXK2wKF5D2ljFR6rbsDA%3D > >{ "listroutersresponse" : { "count":1 ,"router" : [ >{"id":"593d5b00-9c73-485f-b091-121b69171e65","zoneid":"9bf663f5-167a-4152- >b80c-a1fb905e01ef","zonename":"LS_ZONE1","dns1":"10.217.120.2","dns2":"10. >217.120.3","networkdomain":"cs1cloud.internal","name":"r-4-VM","podid":"3d >6a0b64-94c7-40e1-865b-c62469ca2bda","hostid":"7b966f69-e792-4fb5-a5d7-2c3a >cb294f79","hostname":"xenserver1-cs","linklocalip":"169.254.2.164","linklo >calmacaddress":"0e:00:a9:fe:02:a4","linklocalnetmask":"255.255.0.0","linkl >ocalnetworkid":"83214711-3a39-46ac-a482-95368cbd6950","guestipaddress":"10 >.217.5.214","guestmacaddress":"06:0e:d6:00:00:0d","guestnetmask":"255.255. >255.0","guestnetworkid":"003cef18-fb0c-4b5b-8f4d-1b89eda55580","templateid >":"d6260123-7192-4122-beda-d810a7c596b0","created":"2012-07-13T18:52:48-07 >00","state":"Running","account":"system","domainid":"b8d53f75-99be-432f-9a >49-853509c461a3","domain":"ROOT","serviceofferingid":"43713ed5-61fe-4dbc-b >676-f21f0d0ebbb0","serviceofferingname":"System Offering For Software >Router","isredundantrouter":false,"redundantstate":"UNKNOWN"} ] } } > >The three VMs are now Running ... > >From the GUI, successfully added a VM to the only host. > >Thanks. > >Could you please give the sequence of API steps to deleting a host? >deleteHost reports success when devices other than the host have been >adversely affected. > >Regards, >Evan > > > >-----Original Message----- >From: Alena Prokharchyk [mailto:[email protected]] >Sent: Monday, July 16, 2012 5:26 PM >To: [email protected] >Subject: Re: Problem: Unable to create a deployment for VM[User|i-2-8-VM] > >How did you remove the host id=1, through the API or UI? It should have >been resulted with error because it has vms running on it - your Virtual >router r-4-VM. > >Looks like the system is in a pretty bad state. I can think of one way to >recover the vms: > >1) stop the Management server, save the cloud DB dump. >2) In the DB find all Running vms referring the Removed hosts: > >Select * from vm_instance where state in (Running,Starting, Stopping) and >host_id in (select id from host where removed is not null). > >3) update all these vms with the following info: > >Update vm_instance set state='Stopped', host_id=null where id in (<ids >found on prev step>) > >4) start the management server >5) start the vms updated on step 3). > > >-Alena. > > > >On 7/16/12 5:16 PM, "Evan Miller" <[email protected]> wrote: > >>Hi Alena: >> >>It looks like the host table in the database needs to be cleaned up: >> >>mysql> select id, name, uuid, status from host; >>+----+-----------------------------------------------------+----------- >>+----+-----------------------------------------------------+--- >>------------------------+--------------+ >>| id | name | uuid >> | status | >>+----+-----------------------------------------------------+----------- >>+----+-----------------------------------------------------+--- >>------------------------+--------------+ >>| 1 | xenserver1-cs | >>5da2e723-c41a-45bb-9f87-6048c5487ea2 | Removed | >>| 2 | nfs://cumulus.eng.citrite.net/home/export/secondary | >>7bf58f58-0cf5-4487-9346-9481fa28ad7a | Alert | >>| 3 | s-1-VM | >>780d68d6-f6d7-45de-bbd6-766d8e17ee48 | Disconnected | >>| 4 | v-2-VM | >>ca98385f-90f5-49ba-8d88-50af5587e7a9 | Disconnected | >>| 5 | xenserver1-cs | >>dff5e16e-7d8b-47d1-a9a3-8c8e18ca8d23 | Removed | >>| 6 | xenserver1-cs | >>7b966f69-e792-4fb5-a5d7-2c3acb294f79 | Up | >>+----+-----------------------------------------------------+----------- >>+----+-----------------------------------------------------+--- >>------------------------+--------------+ >>6 rows in set (0.00 sec) >> >>mysql> >> >>id 1 and id 5 should be deleted. >>id 6 has the correct state and uuid. >>id 2, 3 and 4 have the wrong state. >> >>I can fix directly with SQL. >> >>Any other tables that need fixing besides host? >> >>Regards, >>Evan >> >> >> >>-----Original Message----- >>From: Alena Prokharchyk [mailto:[email protected]] >>Sent: Monday, July 16, 2012 4:55 PM >>To: [email protected] >>Subject: Re: Problem: Unable to create a deployment for >>VM[User|i-2-8-VM] >> >>The log states that the host doesn't exist: >> >>com.cloud.exception.AgentUnavailableException: Resource [Host:1] is >>unreachable: Host 1: Can't find the host >> >> >> >>Can you check the database for host id=1? Make sure its not removed. >> >> >>On 7/16/12 4:45 PM, "Evan Miller" <[email protected]> wrote: >> >>>From CSMS GUI, the State is Up for the host, xenserver1-cs. >>> >>>Regards, >>>Evan >>> >>> >>>-----Original Message----- >>>From: Alena Prokharchyk [mailto:[email protected]] >>>Sent: Monday, July 16, 2012 4:45 PM >>>To: [email protected] >>>Subject: Re: Problem: Unable to create a deployment for >>>VM[User|i-2-8-VM] >>> >>>Looks like the Xen host your Virtual Router r-4-VM is running on, is >>>in not in UP state. Can you please confirm. If this is the case, then >>>the failure is expected. The Xen host should be in UP state. >>> >>> >>>On 7/16/12 4:16 PM, "Evan Miller" <[email protected]> wrote: >>> >>>>Running CloudStack Management Server: >>>> v3.0.2.1 >>>>On: >>>> [root@cumulus management]# uname -a >>>> Linux cumulus.eng.citrite.net 2.6.32-220.el6.x86_64 #1 SMP Tue Dec >>>>6 >>>>19:48:22 GMT 2011 x86_64 x86_64 x86_64 GNU/Linux >>>> [root@cumulus management]# >>>>Hypervisor : >>>> XenServer v6.02 >>>> >>>>Hi: >>>> >>>>Can¹t add VM instance. >>>>Getting this error message from the GUI: >>>> Unable to create a deployment for VM[User|i-2-8-VM] >>>> >>>>In the attached log file, there are a few ³apparent² issues when >>>>trying to create the instance : >>>> >>>>1) >>>>2012-07-16 15:56:19,928 DEBUG >>>>[network.router.VirtualNetworkApplianceManagerImpl] >>>>(Job-Executor-3:job-33) Applying dhcp entry in network >>>>Ntwk[204|Guest|5] >>>>2012-07-16 15:56:19,943 WARN >>>>[network.router.VirtualNetworkApplianceManagerImpl] >>>>(Job-Executor-3:job-33) Unable to apply dhcp entry on disconnected >>>>router r-4-VM >>>>com.cloud.exception.AgentUnavailableException: Resource [Host:1] is >>>>unreachable: Host 1: Can't find the host >>>> >>>>This one occurred at the beginning of the async job. Not sure if it >>>>is an error or not. >>>> >>>>2) >>>>2012-07-16 15:56:20,521 DEBUG [cloud.capacity.CapacityManagerImpl] >>>>(Job-Executor-3:job-33) VM state transitted from :Stopped to Error >>>>with >>>>event: OperationFailedToErrorvm's original host id: null new host id: >>>>null host id before state transition: >>>> null >>>>2012-07-16 15:56:21,192 INFO [api.commands.DeployVMCmd] >>>>(Job-Executor-3:job-33) >>>>com.cloud.exception.InsufficientServerCapacityException: Unable to >>>>create a deployment for VM[User|i-2-8-VM]Scope=interface >>>>com.cloud.dc.DataCenter; id=1 >>>>2012-07-16 15:56:21,192 WARN [cloud.api.ApiDispatcher] >>>>(Job-Executor-3:job-33) class com.cloud.api.ServerApiException : >>>>Unable to create a deployment for VM[User|i-2-8-VM] >>>> >>>>That one doesn¹t look good. >>>> >>>>3) >>>>2012-07-16 15:56:30,263 DEBUG [cloud.server.StatsCollector] >>>>(StatsCollector-1:null) There is no secondary storage VM for >>>>secondary storage host >>>>nfs://cumulus.eng.citrite.net/home/export/secondary >>>> >>>> >>>>From CloudStack GUI, secondary storage is defined and enabled. >>>> >>>>What¹s important, which needs to be fixed and how to fix? >>>> >>>>Thanks. >>>> >>>>Regards, >>>> >>>>Evan Miller >>>> >>>>Citrix Systems. Inc. >>>>Desktop and Cloud Engineering Infrastructure >>>>4988 Great America Parkway >>>>Santa Clara, CA 95054 >>>> >>>> >>>> >>> >>> >>> >> >> >> > > >
