Hallo all, I'm currently unable to import/activate VM due to error above. It has been a long series of unfortunate events. Long story short: - VM running on storage on iscsi failing disks - put storage and server in maint. Shut down server - restart server. - server has a misconfiguration on lvm/multipath rules (not due to ovirt) - put storage and server active - cluster goes down. Put storage in maintenance again. Detached storage from cluster (ach! That was unfortunate). - cluster now ok. Storage ok. Activate storage. - trying to import VM. Ovirt replies with error in subject and attached.
How can I recover from this situation? It would be great to recover the entire VM, but I'm more interested in disks which are in unregistered state than VM itself. Thank you all
2016-08-14 17:49:32,414 INFO [org.ovirt.engine.core.bll.ImportVmFromConfigurationCommand] (ajp--127.0.0.1-8702-9) [446f3b0e] Lock Acquired to object EngineLock [exclus iveLocks= key: Hertz value: VM_NAME , sharedLocks= key: 49950ad2-05cd-4c5a-bde7-b702199fa250 value: REMOTE_VM ] 2016-08-14 17:49:32,446 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DoesImageExistVDSCommand] (ajp--127.0.0.1-8702-9) [446f3b0e] START, DoesImageExistVDSCommand( s toragePoolId = 00000002-0002-0002-0002-00000000009c, ignoreFailoverLimit = false, storageDomainId = 2103720d-5821-482e-a265-2152ec97eb0f, imageGroupId = 29b97d45-e481-4 35d-a50a-f2a890a678bf, imageId = 0225fa05-be1e-44a6-9d9e-1203b8144933), log id: 3d5b900c 2016-08-14 17:49:32,604 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DoesImageExistVDSCommand] (ajp--127.0.0.1-8702-9) [446f3b0e] FINISH, DoesImageExistVDSCommand, return: true, log id: 3d5b900c 2016-08-14 17:49:32,605 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DoesImageExistVDSCommand] (ajp--127.0.0.1-8702-9) [446f3b0e] START, DoesImageExistVDSCommand( s toragePoolId = 00000002-0002-0002-0002-00000000009c, ignoreFailoverLimit = false, storageDomainId = a074b2a8-0cd3-4d28-bc29-ad37587a8777, imageGroupId = bba3bd45-d02d-4 032-8bb0-e723b7acf23b, imageId = 20f9d5eb-b4d9-4a5d-b75e-3af013c56604), log id: 464e234c 2016-08-14 17:49:32,792 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DoesImageExistVDSCommand] (ajp--127.0.0.1-8702-9) [446f3b0e] FINISH, DoesImageExistVDSCommand, return: true, log id: 464e234c 2016-08-14 17:49:32,792 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DoesImageExistVDSCommand] (ajp--127.0.0.1-8702-9) [446f3b0e] START, DoesImageExistVDSCommand( s toragePoolId = 00000002-0002-0002-0002-00000000009c, ignoreFailoverLimit = false, storageDomainId = 2103720d-5821-482e-a265-2152ec97eb0f, imageGroupId = 6804229e-68f7-4 6a8-90d3-c065529f9624, imageId = 76b4f302-5182-478d-a7d3-cf4c8b6abe40), log id: 254ca74a 2016-08-14 17:49:32,815 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DoesImageExistVDSCommand] (ajp--127.0.0.1-8702-9) [446f3b0e] FINISH, DoesImageExistVDSCommand, return: true, log id: 254ca74a 2016-08-14 17:49:32,815 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DoesImageExistVDSCommand] (ajp--127.0.0.1-8702-9) [446f3b0e] START, DoesImageExistVDSCommand( s toragePoolId = 00000002-0002-0002-0002-00000000009c, ignoreFailoverLimit = false, storageDomainId = 2103720d-5821-482e-a265-2152ec97eb0f, imageGroupId = 217a34f9-ca8c-4a13-a2a2-9dde4c329c55, imageId = 621c0b0c-c041-46e0-8c00-e69896fcf554), log id: 4b7afe67 2016-08-14 17:49:32,842 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DoesImageExistVDSCommand] (ajp--127.0.0.1-8702-9) [446f3b0e] FINISH, DoesImageExistVDSCommand, return: true, log id: 4b7afe67 2016-08-14 17:49:32,843 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DoesImageExistVDSCommand] (ajp--127.0.0.1-8702-9) [446f3b0e] START, DoesImageExistVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000009c, ignoreFailoverLimit = false, storageDomainId = 2103720d-5821-482e-a265-2152ec97eb0f, imageGroupId = efb67e0b-c592-406a-949c-121102eef796, imageId = 44420cae-7c90-461a-bee1-7552758ecee9), log id: 77d442e7 2016-08-14 17:49:32,865 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DoesImageExistVDSCommand] (ajp--127.0.0.1-8702-9) [446f3b0e] FINISH, DoesImageExistVDSCommand, 2016-08-14 17:49:32,908 INFO [org.ovirt.engine.core.bll.ImportVmFromConfigurationCommand] (org.ovirt.thread.pool-8-thread-42) [446f3b0e] Running command: ImportVmFromConfigurationCommand internal: false. Entities affected : ID: a074b2a8-0cd3-4d28-bc29-ad37587a8777 Type: StorageAction group IMPORT_EXPORT_VM with role type ADMIN, ID: 2103720d-5821-482e-a265-2152ec97eb0f Type: StorageAction group IMPORT_EXPORT_VM with role type ADMIN, ID: 2103720d-5821-482e-a265-2152ec97eb0f Type: StorageAction group IMPORT_EXPORT_VM with role type ADMIN, ID: 2103720d-5821-482e-a265-2152ec97eb0f Type: StorageAction group IMPORT_EXPORT_VM with role type ADMIN, ID: 2103720d-5821-482e-a265-2152ec97eb0f Type: StorageAction group IMPORT_EXPORT_VM with role type ADMIN 2016-08-14 17:49:33,079 INFO [org.ovirt.engine.core.utils.transaction.TransactionSupport] (org.ovirt.thread.pool-8-thread-42) [446f3b0e] transaction rolled back 2016-08-14 17:49:33,080 ERROR [org.ovirt.engine.core.bll.ImportVmFromConfigurationCommand] (org.ovirt.thread.pool-8-thread-42) [446f3b0e] Command org.ovirt.engine.core.bll.ImportVmFromConfigurationCommand throw exception: org.springframework.dao.DuplicateKeyException: CallableStatementCallback; SQL [{call insertimage(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)}]; ERROR: duplicate key value violates unique constraint "pk_images" Where: SQL statement "INSERT INTO images( creation_date, image_guid, it_guid, size, ParentId, imageStatus, lastModified, vm_snapshot_id, volume_type, image_group_id, volume_format, active) VALUES( $1 , $2 , $3 , $4 , $5 , $6 , $7 , $8 , $9 , $10 , $11 , $12 )" PL/pgSQL function "insertimage" line 2 at SQL statement; nested exception is org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "pk_images" Where: SQL statement "INSERT INTO images( creation_date, image_guid, it_guid, size, ParentId, imageStatus, lastModified, vm_snapshot_id, volume_type, image_group_id, volume_format, active) VALUES( $1 , $2 , $3 , $4 , $5 , $6 , $7 , $8 , $9 , $10 , $11 , $12 )" PL/pgSQL function "insertimage" line 2 at SQL statement at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:241) [spring-jdbc.jar:3.1.1.RELEASE] at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) [spring-jdbc.jar:3.1.1.RELEASE] at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1030) [spring-jdbc.jar:3.1.1.RELEASE] at org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:1064) [spring-jdbc.jar:3.1.1.RELEASE] at org.springframework.jdbc.core.simple.AbstractJdbcCall.executeCallInternal(AbstractJdbcCall.java:388) [spring-jdbc.jar:3.1.1.RELEASE] at org.springframework.jdbc.core.simple.AbstractJdbcCall.doExecute(AbstractJdbcCall.java:351) [spring-jdbc.jar:3.1.1.RELEASE] at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:141) [dal.jar:] at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:136) [dal.jar:] at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeModification(SimpleJdbcCallsHandler.java:74) [dal.jar:] at org.ovirt.engine.core.dao.DefaultGenericDaoDbFacade.save(DefaultGenericDaoDbFacade.java:93) [dal.jar:] at org.ovirt.engine.core.bll.BaseImagesCommand.saveImage(BaseImagesCommand.java:405) [bll.jar:] at org.ovirt.engine.core.bll.ImportVmCommand.saveImage(ImportVmCommand.java:952) [bll.jar:] at org.ovirt.engine.core.bll.ImportVmCommand.addVmImagesAndSnapshots(ImportVmCommand.java:917) [bll.jar:] at org.ovirt.engine.core.bll.ImportVmCommand$4.runInTransaction(ImportVmCommand.java:737) [bll.jar:] at org.ovirt.engine.core.bll.ImportVmCommand$4.runInTransaction(ImportVmCommand.java:733) [bll.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:210) [utils.jar:] at org.ovirt.engine.core.bll.ImportVmCommand.processImages(ImportVmCommand.java:733) [bll.jar:] at org.ovirt.engine.core.bll.ImportVmCommand.executeCommand(ImportVmCommand.java:702) [bll.jar:] at org.ovirt.engine.core.bll.ImportVmFromConfigurationCommand.executeCommand(ImportVmFromConfigurationCommand.java:129) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1199) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1338) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1967) [bll.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1362) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:359) [bll.jar:] at org.ovirt.engine.core.bll.MultipleActionsRunner.executeValidatedCommand(MultipleActionsRunner.java:193) [bll.jar:] at org.ovirt.engine.core.bll.MultipleActionsRunner.runCommands(MultipleActionsRunner.java:160) [bll.jar:] at org.ovirt.engine.core.bll.MultipleActionsRunner$2.run(MultipleActionsRunner.java:169) [bll.jar:] at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_111] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_111] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_111] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_111] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_111] Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "pk_images" Where: SQL statement "INSERT INTO images( creation_date, image_guid, it_guid, size, ParentId, imageStatus, lastModified, vm_snapshot_id, volume_type, image_group_id, volume_format, active) VALUES( $1 , $2 , $3 , $4 , $5 , $6 , $7 , $8 , $9 , $10 , $11 , $12 )" PL/pgSQL function "insertimage" line 2 at SQL statement at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2103) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1836) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:512) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:388) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:381) at org.jboss.jca.adapters.jdbc.CachedPreparedStatement.execute(CachedPreparedStatement.java:297) at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.execute(WrappedPreparedStatement.java:404) at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1066) [spring-jdbc.jar:3.1.1.RELEASE] at org.springframework.jdbc.core.JdbcTemplate$6.doInCallableStatement(JdbcTemplate.java:1) [spring-jdbc.jar:3.1.1.RELEASE] at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:1014) [spring-jdbc.jar:3.1.1.RELEASE] ... 33 more 2016-08-14 17:49:33,094 INFO [org.ovirt.engine.core.bll.ImportVmFromConfigurationCommand] (org.ovirt.thread.pool-8-thread-42) [446f3b0e] Command [id=aa68938d-49ec-4094-ac8d-9bc06376d222]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.VmStatistics; snapshot: 49950ad2-05cd-4c5a-bde7-b702199fa250. 2016-08-14 17:49:33,095 INFO [org.ovirt.engine.core.bll.ImportVmFromConfigurationCommand] (org.ovirt.thread.pool-8-thread-42) [446f3b0e] Command [id=aa68938d-49ec-4094-ac8d-9bc06376d222]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.network.VmNetworkStatistics; snapshot: ee0be571-554e-4a2f-9822-efc134f068fb. 2016-08-14 17:49:33,097 INFO [org.ovirt.engine.core.bll.ImportVmFromConfigurationCommand] (org.ovirt.thread.pool-8-thread-42) [446f3b0e] Command [id=aa68938d-49ec-4094-ac8d-9bc06376d222]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.network.VmNetworkInterface; snapshot: ee0be571-554e-4a2f-9822-efc134f068fb. 2016-08-14 17:49:33,098 INFO [org.ovirt.engine.core.bll.ImportVmFromConfigurationCommand] (org.ovirt.thread.pool-8-thread-42) [446f3b0e] Command [id=aa68938d-49ec-4094-ac8d-9bc06376d222]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.VmDynamic; snapshot: 49950ad2-05cd-4c5a-bde7-b702199fa250. 2016-08-14 17:49:33,099 INFO [org.ovirt.engine.core.bll.ImportVmFromConfigurationCommand] (org.ovirt.thread.pool-8-thread-42) [446f3b0e] Command [id=aa68938d-49ec-4094-ac8d-9bc06376d222]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.VmStatic; snapshot: 49950ad2-05cd-4c5a-bde7-b702199fa250. 2016-08-14 17:49:33,108 INFO [org.ovirt.engine.core.bll.ImportVmFromConfigurationCommand] (org.ovirt.thread.pool-8-thread-42) [446f3b0e] Lock freed to object EngineLock [exclusiveLocks= key: Hertz value: VM_NAME , sharedLocks= key: 49950ad2-05cd-4c5a-bde7-b702199fa250 value: REMOTE_VM ] 2016-08-14 17:50:06,441 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-26) starting processDomainRecovery for domain 2103720d-5821-482e-a265-2152ec97eb0f:Hertz-dstore1-L 2016-08-14 17:50:06,551 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-26) vds primitivo reported domain 2103720d-5821-482e-a265-2152ec97eb0f:Hertz-dstore1-L as in problem, attempting to move the vds to status NonOperational 2016-08-14 17:50:06,552 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-26) vds sovana reported domain 2103720d-5821-482e-a265-2152ec97eb0f:Hertz-dstore1-L as in problem, attempting to move the vds to status NonOperational 2016-08-14 17:50:06,552 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-26) vds Lambrusco reported domain 2103720d-5821-482e-a265-2152ec97eb0f:Hertz-dstore1-L as in problem, attempting to move the vds to status NonOperational 2016-08-14 17:50:06,553 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-26) Removing vds [6d37d71e-c8c5-47c8-a896-5112298bb83f, 89254f23-8748-402a-afc9-08438dca0975, a21e6b4f-7d06-49c5-8463-680d78517c73] from the domain in maintenance cache 2016-08-14 17:50:06,554 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-26) Removing host(s) [6d37d71e-c8c5-47c8-a896-5112298bb83f, 89254f23-8748-402a-afc9-08438dca0975, a21e6b4f-7d06-49c5-8463-680d78517c73] from hosts unseen domain report cache 2016-08-14 17:50:06,595 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (org.ovirt.thread.pool-8-thread-38) [11443d31] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 89254f23-8748-402a-afc9-08438dca0975 Type: VDS 2016-08-14 17:50:06,607 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (org.ovirt.thread.pool-8-thread-34) [170d64d2] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: a21e6b4f-7d06-49c5-8463-680d78517c73 Type: VDS 2016-08-14 17:50:06,610 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [11443d31] START, SetVdsStatusVDSCommand(HostName = sovana, HostId = 89254f23-8748-402a-afc9-08438dca0975, status=NonOperational, nonOperationalReason=STORAGE_DOMAIN_UNREACHABLE, stopSpmFailureLogged=false), log id: 6463356e 2016-08-14 17:50:06,621 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (org.ovirt.thread.pool-8-thread-26) [4449c225] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 6d37d71e-c8c5-47c8-a896-5112298bb83f Type: VDS 2016-08-14 17:50:06,623 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-34) [170d64d2] START, SetVdsStatusVDSCommand(HostName = Lambrusco, HostId = a21e6b4f-7d06-49c5-8463-680d78517c73, status=NonOperational, nonOperationalReason=STORAGE_DOMAIN_UNREACHABLE, stopSpmFailureLogged=false), log id: 704153b7 2016-08-14 17:50:06,623 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-38) [11443d31] FINISH, SetVdsStatusVDSCommand, log id: 6463356e 2016-08-14 17:50:06,635 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-34) [170d64d2] FINISH, SetVdsStatusVDSCommand, log id: 704153b7 2016-08-14 17:50:06,644 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-8-thread-26) [4449c225] START, SetVdsStatusVDSCommand(HostName = primitivo, HostId = 6d37d71e-c8c5-47c8-a896-5112298bb83f, status=NonOperational, nonOperationalReason=STORAGE_DOMAIN_UNREACHABLE, stopSpmFailureLogged=false), log id: 522d70ef 2016-08-14 17:50:06,657 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-38) [11443d31] Correlation ID: 11443d31, Job ID: 144d8e4d-3560-4a13-9307-2c08977b6c0c, Call Stack: null, Custom Event ID: -1, Message: Host sovana cannot access the Storage Domain(s) Hertz-dstore1-L attached to the Data Center IRDC-INTEL. Setting Host state to Non-Operational. 2016-08-14 17:50:06,666 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-34) [170d64d2] Correlation ID: 170d64d2, Job ID: 91f007cf-2d72-4f69-a7a0-023f150a1c59, Call Stack: null, Custom Event ID: -1, Message: Host Lambrusco cannot access the Storage Domain(s) Hertz-dstore1-L attached to the Data Center IRDC-INTEL. Setting Host state to Non-Operational.
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users