Re: [Users] Backup functionality via Acronis and extracting snapshots
Thanks for your reply Itamar. or copying the images directly from one of the nodes already connected to the storage by ovirt engine How would I copy the image directly? On the node that contains the vm I see the following mounted: /rhev/data-center/mnt/blockSD/0e6991ae-6238-4c61-96d2-ca8fed35161e/master and inside it I see the .ovf file: vms/23b9212c-1e25-4003-aa18-b1e819bf6bb1/23b9212c-1e25-4003-aa18-b1e819bf6bb1.ovf Please excuse my ignorance but where do I find the guest's file system image and copy it out? Get important Linux and industry-related news at: facebook.com/dcdata http://facebook.com/dcdata Kind regards, David Wilson CNS,CLS, LINUX+, CLA, DCTS, LPIC3 *LinuxTech CC t/a DcData* CK number: 2001/058368/23 *Website:* http://www.dcdata.co.za *Support:* +27(0)860-1-LINUX *Mobile:* +27(0)824147413 *Tel:* +27(0)333446100 *Fax:* +27(0)866878971 On 06/19/2012 08:17 AM, Itamar Heim wrote: On 06/19/2012 09:13 AM, David Wilson wrote: Thank you for your reply Itamar. you can export a VM to an NFS export domain, then back it up. you can stop the VM and backup the data from NFS/Block directly. Ok, am I am guessing that for automated backups I should be able to script this using vdsClient? either this, or copying the images directly from one of the nodes already connected to the storage by ovirt engine. in 3.1, you should be able to live snapshot, then backup the snapshot from storage Unfortunately I've not had success with the live snapshots. My message with subject oVirt live snapshot problem to users@ovirt.org provides more information about the problem. How would one go about backing up the snapshot from storage? same as a regular vm... via command line for now. Get important Linux and industry-related news at: facebook.com/dcdata http://facebook.com/dcdata Kind regards, David Wilson CNS,CLS, LINUX+, CLA, DCTS, LPIC3 *LinuxTech CC t/a DcData* CK number: 2001/058368/23 *Website:* http://www.dcdata.co.za *Support:* +27(0)860-1-LINUX *Mobile:* +27(0)824147413 *Tel:* +27(0)333446100 *Fax:* +27(0)866878971 On 06/18/2012 03:36 PM, Itamar Heim wrote: On 06/18/2012 03:41 PM, David Wilson wrote: Hi guys, I am looking for a simple way to backup guest images. What is the recommended/best method to copy vm snapshots from oVirt? I am hoping to copy to an NFS share (export domain) if possible? From what I've seen it is possible when using qemu-img when not using oVirt is it possible to do something similar with oVirt? you can export a VM to an NFS export domain, then back it up. you can stop the VM and backup the data from NFS/Block directly. in 3.1, you should be able to live snapshot, then backup the snapshot from storage Get important Linux and industry-related news at: facebook.com/dcdata http://facebook.com/dcdata Kind regards, David Wilson CNS,CLS, LINUX+, CLA, DCTS, LPIC3 *LinuxTech CC t/a DcData* CK number: 2001/058368/23 *Website:* http://www.dcdata.co.za *Support:* +27(0)860-1-LINUX *Mobile:* +27(0)824147413 *Tel:* +27(0)333446100 *Fax:* +27(0)866878971 On 06/15/2012 03:47 PM, David Wilson wrote: Hi everyone, A quick check up on this one? Has anyone used Acronis with oVirt? Get important Linux and industry-related news at: facebook.com/dcdata http://facebook.com/dcdata Kind regards, David Wilson CNS,CLS, LINUX+, CLA, DCTS, LPIC3 *LinuxTech CC t/a DcData* CK number: 2001/058368/23 *Website:* http://www.dcdata.co.za *Support:* +27(0)860-1-LINUX *Mobile:* +27(0)824147413 *Tel:* +27(0)333446100 *Fax:* +27(0)866878971 On 06/13/2012 06:57 AM, David Wilson wrote: Hi guys, First off well done to the oVirt contributors - it is an excellent product! Has anyone tested Acronis Backup and Recovery with oVirt? I am looking for a reliable way to backup guest oVirt disk images, config and state. So far I've tried the popular script by Daniel Berteaud 'virt-backup.pl' which works very well for our standard KVM systems but I found that I had to disable libvirtd's SASL authentication which caused oVirt to lose connectivity to the node and oVirt to fence the node. I thought I'd try a manual 'virsh save' but that causes oVirt to believe that the guest has gone down which causes the guest to be automatically migrated (I am running 2 nodes). What experience have you had with backing up guests? I am not sure if Acronis is the best way forward? -- Get important Linux and industry-related news at: facebook.com/dcdata http://facebook.com/dcdata Kind regards, David Wilson CNS,CLS, LINUX+, CLA, DCTS, LPIC3 *LinuxTech CC t/a DcData* CK number: 2001/058368/23 *Website:* http://www.dcdata.co.za *Support:* +27(0)860-1-LINUX *Mobile:* +27(0)824147413 *Tel:* +27(0)333446100 *Fax:* +27(0)866878971 Time to evaluate your email security provider? Watch the video http://www.mimecast.com/emailsecurity/ and take advantage of Mimecast’s first ever limited promotion.
[Users] live snapshots ovirt 3.1
It looks like it is possible now to create live snapshots; that is: on a running system. Creating fails however: [root@ovirtm ovirt-engine]# tail -f engine.log | grep -i snapshot 2012-06-19 09:50:07,046 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (ajp--0.0.0.0-8009-7) [62433299] Lock Acquired to object EngineLock [exclusiveLocks= key: 88321641-ae26-4460-ab3e-30daa5a7fc1a value: VM 2012-06-19 09:50:07,213 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (ajp--0.0.0.0-8009-7) [62433299] Running command: CreateAllSnapshotsFromVmCommand internal: false. Entities affected : ID: 88321641-ae26-4460-ab3e-30daa5a7fc1a Type: VM 2012-06-19 09:50:07,260 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (ajp--0.0.0.0-8009-7) [62433299] Lock freed to object EngineLock [exclusiveLocks= key: 88321641-ae26-4460-ab3e-30daa5a7fc1a value: VM 2012-06-19 09:50:07,265 INFO [org.ovirt.engine.core.bll.CreateSnapshotCommand] (ajp--0.0.0.0-8009-7) [50705e1b] Running command: CreateSnapshotCommand internal: true. Entities affected : ID: ---- Type: Storage 2012-06-19 09:50:07,334 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (ajp--0.0.0.0-8009-7) [50705e1b] START, CreateSnapshotVDSCommand(storagePoolId = 29decfa5-6776-45f2-be4c-5aa975e7decf, ignoreFailoverLimit = false, compatabilityVersion = 3.1, storageDomainId = 573aae44-b52c-48c4-a588-6338baedced7, imageGroupId = a5fe1b85-af8d-444b-be70-b0dfe9355593, imageSizeInBytes = 8589934592, volumeFormat = COW, newImageId = cc9b435b-f9c0-4bd4-bb01-1d8f92016448, newImageDescription = , imageId = 50d53c99-ffe7-4ae0-938b-c3f79720e13e, sourceImageGroupId = a5fe1b85-af8d-444b-be70-b0dfe9355593), log id: 7b7a0fb2 2012-06-19 09:50:07,336 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (ajp--0.0.0.0-8009-7) [50705e1b] -- CreateSnapshotVDSCommand::ExecuteIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID 2012-06-19 09:50:07,336 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (ajp--0.0.0.0-8009-7) [50705e1b] -- createVolume parameters: 2012-06-19 09:50:07,529 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateSnapshotVDSCommand] (ajp--0.0.0.0-8009-7) [50705e1b] FINISH, CreateSnapshotVDSCommand, return: cc9b435b-f9c0-4bd4-bb01-1d8f92016448, log id: 7b7a0fb2 2012-06-19 09:50:07,572 INFO [org.ovirt.engine.core.bll.AsyncTaskManager] (ajp--0.0.0.0-8009-7) [50705e1b] AsyncTaskManager::AddTaskToManager: Adding task 2dcdc253-87f8-47cf-ba9f-80deaf131ce4 (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. 2012-06-19 09:50:07,630 INFO [org.ovirt.engine.core.bll.EntityMultiAsyncTasks] (ajp--0.0.0.0-8009-7) [50705e1b] EntityMultiAsyncTasks::StartPollingTask: Current Action Type for entity 88321641-ae26-4460-ab3e-30daa5a7fc1a is CreateAllSnapshotsFromVm (determined by task 2dcdc253-87f8-47cf-ba9f-80deaf131ce4) 2012-06-19 09:50:13,628 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (QuartzScheduler_Worker-76) SPMAsyncTask::PollTask: Polling task 2dcdc253-87f8-47cf-ba9f-80deaf131ce4 (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. 2012-06-19 09:50:13,713 INFO [org.ovirt.engine.core.bll.SPMAsyncTask] (QuartzScheduler_Worker-76) BaseAsyncTask::OnTaskEndSuccess: Task 2dcdc253-87f8-47cf-ba9f-80deaf131ce4 (Parent Command CreateAllSnapshotsFromVm, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. 2012-06-19 09:50:13,717 INFO [org.ovirt.engine.core.bll.EntityAsyncTask] (QuartzScheduler_Worker-76) EntityAsyncTask::EndAction: Ending action for 1 tasks (entity ID: 88321641-ae26-4460-ab3e-30daa5a7fc1a): calling EndAction for action type CreateAllSnapshotsFromVm. 2012-06-19 09:50:13,719 INFO [org.ovirt.engine.core.bll.EntityAsyncTask] (pool-3-thread-48) EntityAsyncTask::EndCommandAction [within thread]context: Attempting to EndAction CreateAllSnapshotsFromVm 2012-06-19 09:50:13,746 INFO [org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand] (pool-3-thread-48) [62433299] Ending command successfully: org.ovirt.engine.core.bll.CreateAllSnapshotsFromVmCommand 2012-06-19 09:50:13,758 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SnapshotVDSCommand] (pool-3-thread-48) [62433299] START, SnapshotVDSCommand(vdsId = c7bfb9f6-b926-11e1-ba3b-d7d59418f024, vmId=88321641-ae26-4460-ab3e-30daa5a7fc1a), log id: 2e483a2e 2012-06-19 09:50:13,832 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-3-thread-48) [62433299] Failed in SnapshotVDS method 2012-06-19 09:50:13,833 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (pool-3-thread-48) [62433299] Error code SNAPSHOT_FAILED and error message VDSGenericException:
[Users] creating template fails ovirt 3.1
Creating a template fails using oivrt 3.1: 2012-06-19 10:16:31,887 ERROR [org.ovirt.engine.core.bll.EntityAsyncTask] (pool-3-thread-47) EntityAsyncTask::EndCommandAction [within thread]: EndAction for action type AddVmTemplate threw an exception: org.ovirt.engine.core.compat.CompatException: XML document structures must start and end within the same entity. at org.ovirt.engine.core.compat.backendcompat.XmlDocument.LoadXml(XmlDocument.java:48) [engine-compat.jar:] at org.ovirt.engine.core.compat.backendcompat.XmlDocument.Load(XmlDocument.java:112) [engine-compat.jar:] at org.ovirt.engine.core.utils.ovf.OvfWriter.Dispose(OvfWriter.java:281) [engine-utils.jar:] at org.ovirt.engine.core.utils.ovf.OvfWriter.dispose(OvfWriter.java:297) [engine-utils.jar:] at org.ovirt.engine.core.utils.ovf.OvfManager.ExportTemplate(OvfManager.java:39) [engine-utils.jar:] at org.ovirt.engine.core.bll.VmTemplateCommand.UpdateTemplateInSpm(VmTemplateCommand.java:212) [engine-bll.jar:] at org.ovirt.engine.core.bll.VmTemplateCommand.UpdateTemplateInSpm(VmTemplateCommand.java:187) [engine-bll.jar:] at org.ovirt.engine.core.bll.AddVmTemplateCommand.EndSuccessfully(AddVmTemplateCommand.java:401) [engine-bll.jar:] at org.ovirt.engine.core.bll.CommandBase.InternalEndSuccessfully(CommandBase.java:466) [engine-bll.jar:] at org.ovirt.engine.core.bll.CommandBase.endActionInTransactionScope(CommandBase.java:421) [engine-bll.jar:] at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1151) [engine-bll.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:204) [engine-utils.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:142) [engine-utils.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:109) [engine-utils.jar:] at org.ovirt.engine.core.bll.CommandBase.EndAction(CommandBase.java:367) [engine-bll.jar:] at org.ovirt.engine.core.bll.Backend.endAction(Backend.java:356) [engine-bll.jar:] at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) [:1.7.0_03-icedtea] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_03-icedtea] at java.lang.reflect.Method.invoke(Method.java:601) [rt.jar:1.7.0_03-icedtea] at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final] at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.1.Final] at org.ovirt.engine.core.utils.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:11) [engine-utils.jar:] at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_03-icedtea] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_03-icedtea] at java.lang.reflect.Method.invoke(Method.java:601) [rt.jar:1.7.0_03-icedtea] at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final] at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.1.Final] at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final] at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final] at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.1.Final] at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53)
Re: [Users] engine-setup fails on NFS ISO Domain
On 06/19/2012 01:56 AM, Trey Dockendorf wrote: My first attempt installing engine failed on the ISO domain so I skipped it and continued on. I've now gone back to try and troubleshoot this and so far this is what I'm getting... This is the failure log information 2012-06-18 18:26:09::DEBUG::common_utils::335::root:: output = Starting NFS services: [ OK ] Starting NFS mountd: [ OK ] Starting NFS daemon: [ OK ] 2012-06-18 18:26:09::DEBUG::common_utils::336::root:: stderr = 2012-06-18 18:26:09::DEBUG::common_utils::337::root:: retcode = 0 2012-06-18 18:26:09::DEBUG::nfsutils::101::root:: Generating unique uuid 2012-06-18 18:26:09::DEBUG::nfsutils::106::root:: creating iso domain for /isodomain. uuid: 58ee5a2c-13dc-48bb-8fa2-ad4b8efefb9e 2012-06-18 18:26:09::DEBUG::nfsutils::115::root:: creating empty files 2012-06-18 18:26:09::DEBUG::nfsutils::122::root:: writing metadata 2012-06-18 18:26:09::DEBUG::nfsutils::65::root:: generating metadata 2012-06-18 18:26:09::DEBUG::nfsutils::77::root:: checksum of metadata is 2030813e3089682bd753c1776ad88f9b13e406f7 2012-06-18 18:26:09::DEBUG::nfsutils::80::root:: writing metadata file (/isodomain/58ee5a2c-13dc-48bb-8fa2-ad4b8efefb9e/dom_md/metadata) 2012-06-18 18:26:10::DEBUG::nfsutils::128::root:: setting directories files permissions to 36:36 2012-06-18 18:26:10::DEBUG::engine-setup::1614::root:: Adding iso domain into DB 2012-06-18 18:26:10::DEBUG::common_utils::350::root:: running sql query 'select inst_add_iso_storage_domain ('58ee5a2c-13dc-48bb-8fa2-ad4b8efefb9e', 'ISODomain', 'dc-vmarchitect.tamu.edu:/isodomain', 0, 0)' on db server: 'localhost'. 2012-06-18 18:26:10::DEBUG::common_utils::286::root:: cmd = /usr/bin/psql -h localhost -p 5432 -U engine -d engine -c select inst_add_iso_storage_domain ('58ee5a2c-13dc-48bb-8fa2-ad4b8efefb9e', 'ISODomain', 'dc-vmarchitect.tamu.edu:/isodomain', 0, 0) 2012-06-18 18:26:10::DEBUG::common_utils::291::root:: output = 2012-06-18 18:26:10::DEBUG::common_utils::292::root:: stderr = ERROR: NUM:42883, DETAILS:function insertstorage_server_connections(character varying, character varying, unknown, unknown, unknown, unknown, integer, unknown, unknown, unknown) does not exist 2012-06-18 18:26:10::DEBUG::common_utils::293::root:: retcode = 1 2012-06-18 18:26:10::ERROR::engine-setup::1587::root:: Traceback (most recent call last): File /usr/bin/engine-setup, line 1582, in _configNfsShare _addIsoDomaintoDB(controller.CONF[sd_uuid], controller.CONF[ISO_DOMAIN_NAME]) File /usr/bin/engine-setup, line 1616, in _addIsoDomaintoDB utils.execRemoteSqlCommand(getDbUser(), getDbHostName(), getDbPort(), basedefs.DB_NAME, sqlQuery, True, output_messages.ERR_FAILED_INSERT_ISO_DOMAIN%(basedefs.DB_NAME)) File /usr/share/ovirt-engine/scripts/common_utils.py, line 352, in execRemoteSqlCommand return execExternalCmd(cmd, failOnError, errMsg) File /usr/share/ovirt-engine/scripts/common_utils.py, line 296, in execExternalCmd raise Exception(msg) Exception: Failed inserting ISO domain into engine db 2012-06-18 18:26:10::DEBUG::setup_sequences::62::root:: Traceback (most recent call last): File /usr/share/ovirt-engine/scripts/setup_sequences.py, line 60, in run function() File /usr/bin/engine-setup, line 1588, in _configNfsShare raise Exception(output_messages.ERR_FAILED_CFG_NFS_SHARE) Exception: Failed to configure NFS share on this host 2012-06-18 18:26:10::DEBUG::engine-setup::1814::root:: *** The following params were used as user input: 2012-06-18 18:26:10::DEBUG::engine-setup::1819::root:: override-httpd-config: yes 2012-06-18 18:26:10::DEBUG::engine-setup::1819::root:: http-port: 80 2012-06-18 18:26:10::DEBUG::engine-setup::1819::root:: https-port: 443 2012-06-18 18:26:10::DEBUG::engine-setup::1819::root:: mac-range: 00:1A:4A:C2:4C:00-00:1A:4A:C2:4C:FF 2012-06-18 18:26:10::DEBUG::engine-setup::1819::root:: host-fqdn: dc-vmarchitect.tamu.edu 2012-06-18 18:26:10::DEBUG::engine-setup::1819::root:: auth-pass: 2012-06-18 18:26:10::DEBUG::engine-setup::1819::root:: org-name: Academy for Advanced Telecommunications and Learning Technology 2012-06-18 18:26:10::DEBUG::engine-setup::1819::root:: default-dc-type: ISCSI 2012-06-18 18:26:10::DEBUG::engine-setup::1819::root:: db-remote-install: local 2012-06-18 18:26:10::DEBUG::engine-setup::1819::root:: db-host: localhost 2012-06-18 18:26:10::DEBUG::engine-setup::1819::root:: db-local-pass: 2012-06-18 18:26:10::DEBUG::engine-setup::1819::root:: nfs-mp: /isodomain 2012-06-18 18:26:10::DEBUG::engine-setup::1819::root:: iso-domain-name: ISODomain 2012-06-18 18:26:10::DEBUG::engine-setup::1819::root:: config-nfs: yes 2012-06-18 18:26:10::DEBUG::engine-setup::1819::root:: override-iptables: yes 2012-06-18 18:26:10::ERROR::engine-setup::2447::root:: Traceback (most recent call last): File /usr/bin/engine-setup, line 2441, inmodule main(confFile) File /usr/bin/engine-setup, line 2230, in main runSequences() File
Re: [Users] creating template fails ovirt 3.1
looks like https://bugzilla.redhat.com/show_bug.cgi?id=832106 to me, feel free to comment there. Thanks for the input, Ofer. - Original Message - Creating a template fails using oivrt 3.1: 2012-06-19 10:16:31,887 ERROR [org.ovirt.engine.core.bll.EntityAsyncTask] (pool-3-thread-47) EntityAsyncTask::EndCommandAction [within thread]: EndAction for action type AddVmTemplate threw an exception : org.ovirt.engine.core.compat.CompatException: XML document structures must start and end within the same entity. at org.ovirt.engine.core.compat.backendcompat.XmlDocument.LoadXml(XmlDocument.java:48) [engine-compat.jar:] at org.ovirt.engine.core.compat.backendcompat.XmlDocument.Load(XmlDocument.java:112) [engine-compat.jar:] at org.ovirt.engine.core.utils.ovf.OvfWriter.Dispose(OvfWriter.java:281) [engine-utils.jar:] at org.ovirt.engine.core.utils.ovf.OvfWriter.dispose(OvfWriter.java:297) [engine-utils.jar:] at org.ovirt.engine.core.utils.ovf.OvfManager.ExportTemplate(OvfManager.java:39) [engine-utils.jar:] at org.ovirt.engine.core.bll.VmTemplateCommand.UpdateTemplateInSpm(VmTemplateCommand.java:212) [engine-bll.jar:] at org.ovirt.engine.core.bll.VmTemplateCommand.UpdateTemplateInSpm(VmTemplateCommand.java:187) [engine-bll.jar:] at org.ovirt.engine.core.bll.AddVmTemplateCommand.EndSuccessfully(AddVmTemplateCommand.java:401) [engine-bll.jar:] at org.ovirt.engine.core.bll.CommandBase.InternalEndSuccessfully(CommandBase.java:466) [engine-bll.jar:] at org.ovirt.engine.core.bll.CommandBase.endActionInTransactionScope(CommandBase.java:421) [engine-bll.jar:] at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1151) [engine-bll.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:204) [engine-utils.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:142) [engine-utils.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:109) [engine-utils.jar:] at org.ovirt.engine.core.bll.CommandBase.EndAction(CommandBase.java:367) [engine-bll.jar:] at org.ovirt.engine.core.bll.Backend.endAction(Backend.java:356) [engine-bll.jar:] at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) [:1.7.0_03-icedtea] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_03-icedtea] at java.lang.reflect.Method.invoke(Method.java:601) [rt.jar:1.7.0_03-icedtea] at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final] at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.1.Final] at org.ovirt.engine.core.utils.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:11) [engine-utils.jar:] at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source) [:1.7.0_03-icedtea] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_03-icedtea] at java.lang.reflect.Method.invoke(Method.java:601) [rt.jar:1.7.0_03-icedtea] at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final] at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.1.Final] at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final] at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final] at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.1.Final] at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.1.1.Final.jar:7.1.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.1.Final] at
Re: [Users] How can remove the existing isos from engine
On 06/19/2012 12:47 AM, Itamar Heim wrote: On 06/19/2012 06:42 AM, Shu Ming wrote: Hi, I uploaded several isos to engine by engine-iso-uploader. But, I found that there is no tools to remove the iso from engine or there is no option in engine to remove them. Any clue to remove them? Just remove the iso from the directory where iso domain is mapped? I think it is a bit rude to do that. Is there any graceful way to remove them? the iso uploader just helps you add them to that directory. i guess it can help you delete them. but not sure deleting them won't be a shorter command no graceful way yet I had plans to do that. Just ran out of time in the initial implementation. I suppose I can put together a feature page for it. Not too difficult. Cheers, Keith ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] Ovirt 3.1 and gluster (creation in ovirt)
On 06/19/2012 11:34 AM, зоррыч wrote: I do not understand you The catalog, which is mounted Gloucester storage available for recording and vdsm successfully creates the necessary files to it. can you please try the NFS troubleshooting approach on this first to try and diagnose the issue? http://www.ovirt.org/wiki/Troubleshooting_NFS_Storage_Issues -Original Message- From: Itamar Heim [mailto:ih...@redhat.com] Sent: Monday, June 18, 2012 7:07 PM To: зоррыч Cc: 'Vijay Bellur'; 'Robert Middleswarth'; users@ovirt.org; 'Daniel Paikov' Subject: Re: [Users] Ovirt 3.1 and gluster (creation in ovirt) On 06/18/2012 06:03 PM, зоррыч wrote: Posix FS storage and you can mount this from vdsm via sudo with same mount options and use it? -Original Message- From: Itamar Heim [mailto:ih...@redhat.com] Sent: Monday, June 18, 2012 6:29 PM To: зоррыч Cc: 'Vijay Bellur'; 'Robert Middleswarth'; users@ovirt.org; 'Daniel Paikov' Subject: Re: [Users] Ovirt 3.1 and gluster (creation in ovirt) On 06/18/2012 04:50 PM, зоррыч wrote: Any ideas for solutions? Is this a bug? *From:*users-boun...@ovirt.org [mailto:users-boun...@ovirt.org] *On Behalf Of *зоррыч *Sent:* Sunday, June 17, 2012 12:04 AM *To:* 'Vijay Bellur'; 'Robert Middleswarth' *Cc:* users@ovirt.org; 'Daniel Paikov' *Subject:* Re: [Users] Ovirt 3.1 and gluster (creation in ovirt) I have updated GlusterFS and volume successfully created Thank you! But I was not able to mount a storage domain. an NFS or Posix FS storage domain? Vdsm.log: Thread-21025::DEBUG::2012-06-16 15:43:21,495::BindingXMLRPC::160::vds::(wrapper) [10.1.20.2] Thread-21025::DEBUG::2012-06-16 15:43:21,495::task::588::TaskManager.Task::(_updateState) Task=`8d841c96-43e3-4d4b-b115-a36c4adf695a`::moving from state init - state preparing Thread-21025::INFO::2012-06-16 15:43:21,503::logUtils::37::dispatcher::(wrapper) Run and protect: validateStorageServerConnection(domType=6, spUUID='----', conList=[{'port': '', 'connection': '10.1.20.7:/sd2', 'iqn': '', 'portal': '', 'user': '', 'vfs_type': 'glusterfs', 'password': '**', 'id': '----'}], options=None) Thread-21025::INFO::2012-06-16 15:43:21,503::logUtils::39::dispatcher::(wrapper) Run and protect: validateStorageServerConnection, Return response: {'statuslist': [{'status': 0, 'id': '----'}]} Thread-21025::DEBUG::2012-06-16 15:43:21,503::task::1172::TaskManager.Task::(prepare) Task=`8d841c96-43e3-4d4b-b115-a36c4adf695a`::finished: {'statuslist': [{'status': 0, 'id': '----'}]} Thread-21025::DEBUG::2012-06-16 15:43:21,503::task::588::TaskManager.Task::(_updateState) Task=`8d841c96-43e3-4d4b-b115-a36c4adf695a`::moving from state preparing - state finished Thread-21025::DEBUG::2012-06-16 15:43:21,503::resourceManager::809::ResourceManager.Owner::(releaseAl l ) Owner.releaseAll requests {} resources {} Thread-21025::DEBUG::2012-06-16 15:43:21,504::resourceManager::844::ResourceManager.Owner::(cancelAll ) Owner.cancelAll requests {} Thread-21025::DEBUG::2012-06-16 15:43:21,504::task::978::TaskManager.Task::(_decref) Task=`8d841c96-43e3-4d4b-b115-a36c4adf695a`::ref 0 aborting False Thread-21026::DEBUG::2012-06-16 15:43:21,526::BindingXMLRPC::160::vds::(wrapper) [10.1.20.2] Thread-21026::DEBUG::2012-06-16 15:43:21,526::task::588::TaskManager.Task::(_updateState) Task=`2a6538e5-b961-478a-bce6-f5ded1a62bca`::moving from state init - state preparing Thread-21026::INFO::2012-06-16 15:43:21,527::logUtils::37::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=6, spUUID='----', conList=[{'port': '', 'connection': '10.1.20.7:/sd2', 'iqn': '', 'portal': '', 'user': '', 'vfs_type': 'glusterfs', 'password': '**', 'id': 'e7766e1d-f2c6-45ee-900e-00c6689649cd'}], options=None) Thread-21026::DEBUG::2012-06-16 15:43:21,530::lvm::460::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex Thread-21026::DEBUG::2012-06-16 15:43:21,531::lvm::462::OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex Thread-21026::DEBUG::2012-06-16 15:43:21,531::lvm::472::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex Thread-21026::DEBUG::2012-06-16 15:43:21,531::lvm::474::OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex Thread-21026::DEBUG::2012-06-16 15:43:21,531::lvm::493::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex Thread-21026::DEBUG::2012-06-16 15:43:21,531::lvm::495::OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex Thread-21026::INFO::2012-06-16 15:43:21,532::logUtils::39::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response:
Re: [Users] SELinux denials on Sanlock
On 06/19/2012 12:50 AM, Trey Dockendorf wrote: I don't know if this is the wrong place to ask this question, but I just started seeing SELinux denials after adding an iSCSI storage domain in oVirt using my first node. The node is reporting that selinux is preventing /usr/sbin/wdmd from accessing the pid file for that service. david? Here are the denials type=AVC msg=audit(1340053766.745:7): avc: denied { open } for pid=1908 comm=wdmd name=wdmd.pid dev=dm-0 ino=1574530 scontext=system_u:system_r:wdmd_t:s0 tcontext=system_u:object_r:initrc_var_run_t:s0 tclass=file type=AVC msg=audit(1340053766.746:8): avc: denied { lock } for pid=1908 comm=wdmd path=/var/run/wdmd/wdmd.pid dev=dm-0 ino=1574530 scontext=system_u:system_r:wdmd_t:s0 tcontext=system_u:object_r:initrc_var_run_t:s0 tclass=file type=SYSCALL msg=audit(1340053766.746:8): arch=x86_64 syscall=fcntl success=yes exit=0 a0=4 a1=6 a2=7fffae656290 a3=fff3 items=0 ppid=1 pid=1908 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm=wdmd exe=/usr/sbin/wdmd subj=system_u:system_r:wdmd_t:s0 key=(null) I ran restorecon on /var/run/wdmd and it changed from var_run_t to wdmd_var_run_t. I'm not sure how to make sanlock run again to see if that solved the issue. Is there a way to determine it's resolved? Thanks - Trey ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] SELinux denials on Sanlock
On Tue, Jun 19, 2012 at 01:29:30PM -0400, Daniel J Walsh wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 06/19/2012 12:13 PM, David Teigland wrote: type=AVC msg=audit(1340053766.745:7): avc: denied { open } for pid=1908 comm=wdmd name=wdmd.pid dev=dm-0 ino=1574530 scontext=system_u:system_r:wdmd_t:s0 tcontext=system_u:object_r:initrc_var_run_t:s0 tclass=file type=AVC msg=audit(1340053766.746:8): avc: denied { lock } for pid=1908 comm=wdmd path=/var/run/wdmd/wdmd.pid dev=dm-0 ino=1574530 scontext=system_u:system_r:wdmd_t:s0 tcontext=system_u:object_r:initrc_var_run_t:s0 tclass=file type=SYSCALL msg=audit(1340053766.746:8): arch=x86_64 syscall=fcntl success=yes exit=0 a0=4 a1=6 a2=7fffae656290 a3=fff3 items=0 ppid=1 pid=1908 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm=wdmd exe=/usr/sbin/wdmd subj=system_u:system_r:wdmd_t:s0 key=(null) THis is caused by a bug in sanlock, the init script for wdmd was creating the /var/run/wdmd file but not running restorecon on it. Thanks, that was fixed back in Feb, so some old/wrong packages must be in use. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[Users] Sanlock issue when trying to start vm
Hi, after getting the 3.1 beta engine and a host set up I now get an error when trying to start a vm. Engine reports this: VM myvm is down. Exit message: internal error Failed to open socket to sanlock daemon: No such file or directory. On the node no sanlock daemon is running. When I try to start the service I get this: Jun 19 20:22:31 node systemd-sanlock[13607]: Starting sanlock: [ OK ] Jun 19 20:22:31 node sanlock[13621]: 2012-06-19 20:22:31+0200 910 [13621]: sanlock daemon started 2.3 aio 1 10 renew 20 80 host 93bea910-1d9d-4203-8333-beb3d7b92c10.node.local time 1340130151 Jun 19 20:22:31 node sanlock[13621]: 2012-06-19 20:22:31+0200 910 [13621]: set scheduler RR|RESET_ON_FORK priority 99 failed: Operation not permitted Jun 19 20:22:31 node sanlock[13621]: 2012-06-19 20:22:31+0200 910 [13621]: wdmd connect failed for watchdog handling Jun 19 20:22:31 node systemd[1]: sanlock.service: main process exited, code=exited, status=255 Jun 19 20:22:31 node systemd[1]: Unit sanlock.service entered failed state. I followed the SELinux and sanlock threads but I disabled SELinux on the node so that cannot be a reason for this to fail. Regards, Dennis ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] oVirt 3.1 - Error adding host , incorrect vdsm version
On Tue, Jun 19, 2012 at 12:08:09PM -0400, Ofer Schreiber wrote: Just checked this issue with latest ovirt-engine on F17 (with vdsm 4.10), and didn't encounter this issue at all. Danken - any idea why? Hey! I was the one soliciting ideas on how come this issue was not seen until Trey bumped on it. ;-) Maybe, just maybe, you host sees BOTH 4.10 and 4.9 ? what does the following print on your python interpreter? import yum my = yum.YumBase() my.pkgSack.searchNevra(name='rpm') - Original Message - On Mon, Jun 18, 2012 at 04:08:06PM -0500, Trey Dockendorf wrote: On Mon, Jun 18, 2012 at 3:25 PM, Dan Kenigsberg dan...@redhat.com wrote: On Mon, Jun 18, 2012 at 03:01:10PM -0500, Trey Dockendorf wrote: snip You are right. And it is truly surprising that this bug hasn't been open since we've bumped vdsm version to 4.10. Would you open up this bug, and make it block Bug 822145 - Tracker: oVirt 3.1 release ? The bootstrapping code is a mess. It evolves by adding one ugly patch over the other. In that spirit, would the following patch solve your bootstrapping pains? I actually really really hate the code there, and spent some time suggesting a less bad approach. Would you be kind enough to test http://gerrit.ovirt.org/#/c/5469/ and the 2 patches below it? diff --git a/vds_bootstrap/vds_bootstrap.py b/vds_bootstrap/vds_bootstrap.py index 0df5023..4fcc040 100755 --- a/vds_bootstrap/vds_bootstrap.py +++ b/vds_bootstrap/vds_bootstrap.py @@ -76,11 +76,11 @@ rhel6based = deployUtil.versionCompare(deployUtil.getOSVersion(), 6.0) = 0 # TODO this is an infra-hackish heuristic for identifying Fedora # drop as soon as possible -fedorabased = deployUtil.versionCompare(deployUtil.getOSVersion(), 16) = 0 +fedorabased = deployUtil.versionCompare(deployUtil.getOSVersion(), 17) = 0 if rhel6based: VDSM_NAME = vdsm -VDSM_MIN_VER = VDSM_NAME + -4.9 +VDSM_MIN_VER = VDSM_NAME + -4.10 KERNEL_VER = 2.6.32-.*.el6 KERNEL_MIN_VER = 150 MINIMAL_SUPPORTED_PLATFORM = 6.0 Those changes worked, the install is now past that point and installing packages. Bug filed, https://bugzilla.redhat.com/show_bug.cgi?id=833201 Thanks! ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] oVirt 3.1 - Error adding host , incorrect vdsm version
On Tue, Jun 19, 2012 at 02:43:14PM -0500, Trey Dockendorf wrote: On Tue, Jun 19, 2012 at 1:40 PM, Dan Kenigsberg dan...@redhat.com wrote: On Tue, Jun 19, 2012 at 12:08:09PM -0400, Ofer Schreiber wrote: Just checked this issue with latest ovirt-engine on F17 (with vdsm 4.10), and didn't encounter this issue at all. Danken - any idea why? Hey! I was the one soliciting ideas on how come this issue was not seen until Trey bumped on it. ;-) Maybe, just maybe, you host sees BOTH 4.10 and 4.9 ? what does the following print on your python interpreter? import yum my = yum.YumBase() my.pkgSack.searchNevra(name='rpm') From the python , you want to output of name='rpm' or name='vdsm' ? I figured the later but included both arghh s/rpm/vdsm/ Actually, this question was directed at Ofer. I was wodering how come the untouched code was working in his environment. On VDSM node import yum my = yum.YumBase() my.pkgSack.searchNevra(name='rpm') Loaded plugins: fastestmirror, presto, priorities Loading mirror speeds from cached hostfile * base: mirror.raystedman.net * epel: mirror.utexas.edu * extras: mirror.unl.edu * updates: mirror.raystedman.net [YumAvailablePackageSqlite : rpm-4.8.0-19.el6.x86_64 (0x28c49d0), YumAvailablePackageSqlite : rpm-4.8.0-19.el6_2.1.x86_64 (0x28c4b90)] my.pkgSack.searchNevra(name='vdsm') [YumAvailablePackageSqlite : vdsm-4.10.0-0.33.git40b2d55.el6.x86_64 (0x28c4cd0)] On the ovirt-engine host import yum my = yum.YumBase() my.pkgSack.searchNevra(name='rpm') Loaded plugins: fastestmirror, priorities, versionlock Loading mirror speeds from cached hostfile * base: mirror.steadfast.net * epel: mirror.steadfast.net * extras: mirror.steadfast.net * updates: mirror.anl.gov [YumAvailablePackageSqlite : rpm-4.8.0-19.el6.x86_64 (0x7f60110cedd0), YumAvailablePackageSqlite : rpm-4.8.0-19.el6_2.1.x86_64 (0x7f60110dde90)] my.pkgSack.searchNevra(name='vdsm') [YumAvailablePackageSqlite : vdsm-4.10.0-0.33.git40b2d55.el6.x86_64 (0x7f60110dd490)] Let me know if anything else would be helpful. Well, I could use yet another verification/review of my http://gerrit.ovirt.org/#/c/5469/ Thanks, Dan. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] I am using the CentOS build from the 12th. I the engine is dieing with the following error.
On 06/20/2012 04:09 AM, Robert Middleswarth wrote: On 06/18/2012 04:47 AM, Juan Hernandez wrote: On 06/18/2012 04:25 AM, Robert Middleswarth wrote: 2012-06-17 05:21:24,682 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (ajp--0.0.0.0-8009-9) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,121 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-88) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 05:21:26,130 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-88) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 15:49:08,577 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-28) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:13,943 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-64) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 20:21:15,949 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-65) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:00:15,133 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-98) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:07:00,037 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-5) File /usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:07:00,115 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File /usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,686 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-6) File /usr/share/vdsm-bootstrap/deployUtil.py is 47867 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:32:10,752 WARN [org.ovirt.engine.core.ServletUtils] (ajp--0.0.0.0-8009-1) File /usr/share/vdsm-bootstrap/vds_bootstrap.py is 33884 bytes long. Please reconsider using this servlet for files larger than 8192 bytes. 2012-06-17 21:44:34,028 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-23) XML RPC error in command ListVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files 2012-06-17 21:44:36,034 ERROR [org.ovirt.engine.core.engineencryptutils.EncryptionUtils] (QuartzScheduler_Worker-26) Failed to decryptjava.io.FileNotFoundException: /etc/pki/ovirt-engine/.keystore (Too many open files) 2012-06-17 21:44:36,045 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (QuartzScheduler_Worker-26) XML RPC error in command GetCapabilitiesVDS ( Vds: ann100ovirt01 ), the error was: java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetException, SocketException: Too many open files I restarted ovirt-engine and ran lsof -p 3557 I see hundreds of the same open file. java 3557 ovirt 839r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 840r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 841r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 842r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 843r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 844r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 845r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 846r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 847r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 848r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 849r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 850r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 851r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 852r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 853r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore java 3557 ovirt 854r REG 253,0 3230 7865608 /etc/pki/ovirt-engine/.keystore Is anyone else seeing this? There are two issues that might be affecting you here: 1. We may have a leak of file descriptors on the logic