Elad, When I try to detach, these errors appear from engine telling that the domain is not pool :
2015-04-01 12:37:51,726 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand] (org.ovirt.thread.pool-8-thread-23) [394eef4c] START, DetachStorageDomainVDSCommand( storagePoolId = f422de63-8869-41ef-a782-8b0c9ee03c41, ignoreFailoverLimit = false, storageDomainId = d7b9d7cc-f7d6-43c7-ae13-e720951657c9, masterDomainId = 00000000-0000-0000-0000-000000000000, masterVersion = 29, force = false), log id: 50a504cc 2015-04-01 12:37:52,296 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand] (org.ovirt.thread.pool-8-thread-23) [394eef4c] Command DetachStorageDomainVDSCommand( storagePoolId = f422de63-8869-41ef-a782-8b0c9ee03c41, ignoreFailoverLimit = false, storageDomainId = d7b9d7cc-f7d6-43c7-ae13-e720951657c9, masterDomainId = 00000000-0000-0000-0000-000000000000, masterVersion = 29, force = false) execution failed. Exception: IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Storage domain not in pool: 'domain=d7b9d7cc-f7d6-43c7-ae13-e720951657c9, pool=f422de63-8869-41ef-a782-8b0c9ee03c41' 2015-04-01 12:37:52,304 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand] (org.ovirt.thread.pool-8-thread-23) [394eef4c] FINISH, DetachStorageDomainVDSCommand, log id: 50a504cc 2015-04-01 12:37:52,309 ERROR [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-8-thread-23) [394eef4c] Command org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.irsbroker.IrsOperationFailedNoFailoverException: IRSGenericException: IRSErrorException: Storage domain not in pool: 'domain=d7b9d7cc-f7d6-43c7-ae13-e720951657c9, pool=f422de63-8869-41ef-a782-8b0c9ee03c41' (Failed with error StorageDomainNotInPool and code 353) 2015-04-01 12:37:52,323 INFO [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-8-thread-23) [394eef4c] Command [id=04581afc-9388-4d6e-ab00-868f261ea07f]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap; snapshot: EntityStatusSnapshot [id=storagePoolId = f422de63-8869-41ef-a782-8b0c9ee03c41, storageId = d7b9d7cc-f7d6-43c7-ae13-e720951657c9, status=Maintenance]. 2015-04-01 12:37:52,365 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-23) [394eef4c] Correlation ID: 394eef4c, Job ID: 97a92d14-2eca-41b6-840d-d189d7398882, Call Stack: null, Custom Event ID: -1, Message: Failed to detach Storage Domain VOL-UNC-PROD-02 to Data Center Unicef. (User: admin) 2015-04-01 12:37:52,390 INFO [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-8-thread-23) [394eef4c] Lock freed to object EngineLock [exclusiveLocks= key: d7b9d7cc-f7d6-43c7-ae13-e720951657c9 value: STORAGE , sharedLocks= ] Thanks De : users-boun...@ovirt.org [mailto:users-boun...@ovirt.org] De la part de VONDRA Alain Envoyé : mercredi 1 avril 2015 12:19 À : Elad Ben Aharon Cc : users@ovirt.org Objet : Re: [ovirt-users] Storage domain not in pool issue Elad, As I told before, the Storage domain is in Maintenance state, and It’s impossible to Detach or Activate it. It can just stay in Maintenance state... Thanks ________________________________________ Alain VONDRA Chargé d'exploitation des Systèmes d'Information Direction Administrative et Financière +33 1 44 39 77 76 UNICEF France 3 rue Duguay Trouin 75006 PARIS www.unicef.fr Alain VONDRA Chargé d'exploitation des Systèmes d'Information Direction Administrative et Financière +33 1 44 39 77 76 UNICEF France 3 rue Duguay Trouin 75006 PARIS www.unicef.fr ________________________________________ De : Elad Ben Aharon [mailto:ebena...@redhat.com] Envoyé : mercredi 1 avril 2015 10:44 À : VONDRA Alain Cc : Adam Litke; users@ovirt.org; Allon Mureinik; Tal Nisan Objet : Re: [ovirt-users] Storage domain not in pool issue Can you please try to detach them from the current DC? If it will work, please try to re-attach them to the desired DC and to activate them. ________________________________________ From: "VONDRA Alain" <avon...@unicef.fr> To: "Elad Ben Aharon" <ebena...@redhat.com> Cc: "Adam Litke" <ali...@redhat.com>, users@ovirt.org, "Allon Mureinik" <amure...@redhat.com>, "Tal Nisan" <tni...@redhat.com> Sent: Wednesday, 1 April, 2015 11:39:13 AM Subject: RE: [ovirt-users] Storage domain not in pool issue These domains were previously imported to another DC after a crash disaster using “Import Domain” in oVirt. Thanks ________________________________________ Alain VONDRA Chargé d'exploitation des Systèmes d'Information Direction Administrative et Financière +33 1 44 39 77 76 UNICEF France 3 rue Duguay Trouin 75006 PARIS www.unicef.fr ________________________________________ De : Elad Ben Aharon [mailto:ebena...@redhat.com] Envoyé : mercredi 1 avril 2015 10:36 À : VONDRA Alain Cc : Adam Litke; users@ovirt.org; Allon Mureinik; Tal Nisan Objet : Re: [ovirt-users] Storage domain not in pool issue Were these domains attached to another DC previously? ________________________________________ From: "VONDRA Alain" <avon...@unicef.fr> To: "Elad Ben Aharon" <ebena...@redhat.com>, "Adam Litke" <ali...@redhat.com> Cc: users@ovirt.org, "Allon Mureinik" <amure...@redhat.com> Sent: Wednesday, 1 April, 2015 10:52:19 AM Subject: RE: [ovirt-users] Storage domain not in pool issue Hi, Any more news ??? Thanks Alain De : Elad Ben Aharon [mailto:ebena...@redhat.com] Envoyé : mardi 31 mars 2015 14:44 À : VONDRA Alain; Adam Litke Cc : users@ovirt.org; Allon Mureinik Objet : Re: [ovirt-users] Storage domain not in pool issue Hi Adam, Can you please take a look? By the provided information, there seems to be an issue of synchronization failure between engine and vdsm regarding the storage pool which the 2 domains are attached to. Thanks Alain VONDRA Chargé d'exploitation des Systèmes d'Information Direction Administrative et Financière +33 1 44 39 77 76 UNICEF France 3 rue Duguay Trouin 75006 PARIS www.unicef.fr ________________________________________ From: "VONDRA Alain" <avon...@unicef.fr> To: "Elad Ben Aharon" <ebena...@redhat.com> Cc: "Idan Shaby" <ish...@redhat.com>, users@ovirt.org, "Allon Mureinik" <amure...@redhat.com> Sent: Tuesday, 31 March, 2015 11:43:39 AM Subject: RE: [ovirt-users] Storage domain not in pool issue Hi, Here is the logs. Thanks ________________________________________ Alain VONDRA Chargé d'exploitation des Systèmes d'Information Direction Administrative et Financière +33 1 44 39 77 76 UNICEF France 3 rue Duguay Trouin 75006 PARIS www.unicef.fr ________________________________________ De : Elad Ben Aharon [mailto:ebena...@redhat.com] Envoyé : mardi 31 mars 2015 08:19 À : VONDRA Alain Cc : Idan Shaby; users@ovirt.org; Allon Mureinik Objet : Re: [ovirt-users] Storage domain not in pool issue Please attach /var/log/ovirt-engine/engine.log from engine and /var/log/vdsm/vdsm.log from the host(s). ________________________________________ From: "VONDRA Alain" <avon...@unicef.fr> To: "Elad Ben Aharon" <ebena...@redhat.com>, "Idan Shaby" <ish...@redhat.com> Cc: users@ovirt.org Sent: Tuesday, 31 March, 2015 1:49:42 AM Subject: RE:[ovirt-users] Storage domain not in pool issue Hi, Is there anybody who can help me ? I have more than 6 month of work in these two storage domains... Thanks, Alain Alain VONDRA Chargé d'exploitation des Systèmes d'Information Direction Administrative et Financière +33 1 44 39 77 76 UNICEF France 3 rue Duguay Trouin 75006 PARIS www.unicef.fr ________________________________________ De : users-boun...@ovirt.org [users-boun...@ovirt.org] de la part de VONDRA Alain [avon...@unicef.fr] Envoyé : lundi 30 mars 2015 17:33 À : Elad Ben Aharon; Idan Shaby Cc : users@ovirt.org Objet : Re: [ovirt-users] Storage domain not in pool issue Hi, Is there anyother way to synchronize the engine and vdsm ? Thanks Alain VONDRA Chargé d'exploitation des Systèmes d'Information Direction Administrative et Financière +33 1 44 39 77 76 UNICEF France 3 rue Duguay Trouin 75006 PARIS www.unicef.fr -----Message d'origine----- De : users-boun...@ovirt.org [mailto:users-boun...@ovirt.org] De la part de VONDRA Alain Envoyé : lundi 30 mars 2015 16:17 À : Elad Ben Aharon; Idan Shaby Cc : users@ovirt.org Objet : Re: [ovirt-users] Storage domain not in pool issue Elad, I’ve tried many times to restart the engine without success. Thanks De : Elad Ben Aharon [mailto:ebena...@redhat.com] Envoyé : lundi 30 mars 2015 16:00 À : VONDRA Alain; Idan Shaby Cc : users@ovirt.org Objet : Re: [ovirt-users] Storage domain not in pool issue Thanks for the input. From the input, it's clear that engine is not synchronized with vdsm regarding the storage pool which these 2 domains are attached to. I guess that ovirt-engine service restart would solve it, but I would like someone from DEV to take a look before doing so. Idan, can you please take a look? Thanks Alain VONDRA Chargé d'exploitation des Systèmes d'Information Direction Administrative et Financière +33 1 44 39 77 76 UNICEF France 3 rue Duguay Trouin 75006 PARIS www.unicef.fr ________________________________________ From: "VONDRA Alain" <avon...@unicef.fr> To: "Elad Ben Aharon" <ebena...@redhat.com> Cc: users@ovirt.org Sent: Monday, 30 March, 2015 4:18:08 PM Subject: RE: [ovirt-users] Storage domain not in pool issue Thank you for your help ! I attempt to activate using oVirt GUI, in fact this SD was previously attached to the Data Center, and I had a crash on it, after a recover à my manager oVirt, I see all the VMs and SD up, but the VMs contained in the 2 bad Storage Domain didn’t go up, so I’ve tried to put the SD in maintenance, but now I can’t even activate this domain. I didn’t yet try to put the second bad SD in maintenance after that. id | storage_pool_id --------------------------------------+--------------------------------- --------------------------------------+----- 072fbaa1-08f3-4a40-9f34-a5ca22dd1d74 | 37a291a4-6108-4bb9-af33-da88555ab3a9 | f422de63-8869-41ef-a782-8b0c9ee03c41 d7b9d7cc-f7d6-43c7-ae13-e720951657c9 | f422de63-8869-41ef-a782-8b0c9ee03c41 7e40772a-fe94-4fb2-94c4-6198bed04a6a | f422de63-8869-41ef-a782-8b0c9ee03c41 7e137867-953f-4fd5-a77a-f3bc18cd140e | f422de63-8869-41ef-a782-8b0c9ee03c41 1f6dec51-12a6-41ed-9d14-8f0ad4e062d2 | f422de63-8869-41ef-a782-8b0c9ee03c41 69ddb676-f423-43fc-af38-909bbedeef61 | f422de63-8869-41ef-a782-8b0c9ee03c41 (7 lignes) Thanks De : Elad Ben Aharon [mailto:ebena...@redhat.com] Envoyé : lundi 30 mars 2015 15:09 À : VONDRA Alain Cc : users@ovirt.org Objet : Re: [ovirt-users] Storage domain not in pool issue Thanks. As you can see, the domain is not attached to pool f422de63-8869-41ef-a782-8b0c9ee03c41, which is the pool you've tried to activate the domain in. Via which API did you attempt to activate the domain in this pool? Also, please execute the following command on engine and paste its output here. su - postgres -c "psql -U postgres engine -c 'select id,storage_pool_id from storage_domains;'" | less -S Alain VONDRA Chargé d'exploitation des Systèmes d'Information Direction Administrative et Financière +33 1 44 39 77 76 UNICEF France 3 rue Duguay Trouin 75006 PARIS www.unicef.fr ________________________________________ From: "VONDRA Alain" <avon...@unicef.fr> To: "Elad Ben Aharon" <ebena...@redhat.com> Cc: users@ovirt.org Sent: Monday, 30 March, 2015 3:43:35 PM Subject: RE: [ovirt-users] Storage domain not in pool issue My oVirt version is 3.5.0.1 [root@unc-srv-hyp2 ~]$ vdsClient -s 0 getStorageDomainInfo d7b9d7cc-f7d6-43c7-ae13-e720951657c9 uuid = d7b9d7cc-f7d6-43c7-ae13-e720951657c9 vguuid = zyiwLa-STgp-v3r8-KJqj-1AIP-rC2k-4NuXqf state = OK version = 3 role = Regular type = ISCSI class = Data pool = ['c58a44b1-1c98-450e-97e1-3347eeb28f86'] name = VOL-UNC-PROD-02 De : Elad Ben Aharon [mailto:ebena...@redhat.com] Envoyé : lundi 30 mars 2015 14:41 À : VONDRA Alain Cc : users@ovirt.org Objet : Re: [ovirt-users] Storage domain not in pool issue Which oVirt version are you using? Also, please execute the following command and paste its output here. vdsClient -s 0 getStorageDomainInfo d7b9d7cc-f7d6-43c7-ae13-e720951657c9 Alain VONDRA Chargé d'exploitation des Systèmes d'Information Direction Administrative et Financière +33 1 44 39 77 76 UNICEF France 3 rue Duguay Trouin 75006 PARIS www.unicef.fr ________________________________________ From: "VONDRA Alain" <avon...@unicef.fr> To: "Elad Ben Aharon" <ebena...@redhat.com> Cc: users@ovirt.org Sent: Monday, 30 March, 2015 3:05:49 PM Subject: RE: [ovirt-users] Storage domain not in pool issue Hi, Yes I have an active storage domain Thanks De : Elad Ben Aharon [mailto:ebena...@redhat.com] Envoyé : lundi 30 mars 2015 13:59 À : VONDRA Alain Cc : users@ovirt.org Objet : Re: [ovirt-users] Storage domain not in pool issue Hi, Besides from these 2 storage domains, do you have an active master storage domain in the pool? Elad Ben Aharon RHEV-QE storage Alain VONDRA Chargé d'exploitation des Systèmes d'Information Direction Administrative et Financière +33 1 44 39 77 76 UNICEF France 3 rue Duguay Trouin 75006 PARIS www.unicef.fr ________________________________________ From: "VONDRA Alain" <avon...@unicef.fr> To: users@ovirt.org Sent: Monday, 30 March, 2015 12:18:26 PM Subject: [ovirt-users] Storage domain not in pool issue Hi, I can’t activate two Storage domain after a Data Center disaster, I have these errors in my vdsm.log : Thread-76::ERROR::2015-03-30 10:38:36,521::task::866::Storage.TaskManager.Task::(_setError) Task=`48e52e40-423b-4e9a-ac6b-e5b423bf9b99`::Unexpected error Thread-76::ERROR::2015-03-30 10:38:36,525::dispatcher::76::Storage.Dispatcher::(wrapper) {'status': {'message': "Storage domain not in pool: 'domain=d7b9d7cc-f7d6-43c7-ae13-e720951657c9, pool=f422de63-8869-41ef-a782-8b0c9ee03c41'", 'code': 353}} How can I re-activate them ? Thanks ________________________________________ Alain VONDRA Chargé d'exploitation des Systèmes d'Information Direction Administrative et Financière +33 1 44 39 77 76 UNICEF France 3 rue Duguay Trouin 75006 PARIS www.unicef.fr ________________________________________ _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users