[Engine-devel] Usage of Dynamic Queries
Hi All, As part of an ongoing effort to improve oVirt performance, we're trying to reduce the number of Dynamic Queries usage in the code, and replace it with Postgres stored procedures. You wouldn't believe the performance gain. Please consider before making use of the Dynamic Queries feature - it should ONLY be used for places where users can enter search queries - and not for any other usage. Thanks. Liran ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] oVirt 3.4.0 branch creation
Hi, we're going to create ovirt-engine-3.4.0 branch tomorrow morning 09:00 UTC. oVirt 3.4.1 development will continue on ovirt-engine-3.4 branch. Critical fixes for oVirt 3.4.0 GA will need to be pushed to 3.4.0 branch too starting tomorrow morning 09:00 UTC. Thanks, -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Asynchronous tasks for live merge
On Fri, Feb 28, 2014 at 09:30:16AM -0500, Adam Litke wrote: Hi all, As part of our plan to support live merging of VM disk snapshots it seems we will need a new form of asynchronous task in ovirt-engine. I am aware of AsyncTaskManager but it seems to be limited to managing SPM tasks. For live merge, we are going to need something called VmTasks since the async command can be run only on the host that currently runs the VM. The way I see this working from an engine perspective is: 1. RemoveSnapshotCommand in bll is invoked as usual but since the VM is found to be up, we activate an alternative live merge flow. 2. We submit a LiveMerge VDS Command for each impacted disk. This is an asynchronous command which we need to monitor for completion. 3. A VmJob is inserted into the DB so we'll remember to handle it. 4. The VDS Broker monitors the operation via an extension to the already collected VmStatistics data. Vdsm will report active Block Jobs only. Once the job stops (in error or success) it will cease to be reported by vdsm and engine will know to proceed. You describe a reasonable way for Vdsm to report whether an async operation has finished. However, may we instead use the oportunity to introduce generic hsm tasks? I suggest to have something loosely modeled on posix fork/wait. - Engine asks Vdsm to start an API verb asynchronously and supplies a uuid. This is unlike fork(2), where the system chooses the pid, but that's required so that Engine could tell if the command has reached Vdsm in case of a network error. - Engine may monitor the task (a-la wait(WNOHANG)) - When the task is finished, Engine may collect its result (a-la wait). Until that happens, Vdsm must report the task forever; restart or upgrade are no excuses. On reboot, though, all tasks are forgotten, so Engine may stop monitoring tasks on a fenced host. This may be an over kill for your use case, but it would come useful for other cases. In particular, setupNetwork returns before it is completely done, since dhcp address acquisition may take too much time. Engine may poll getVdsCaps to see when it's done (or timeout), but it would be nicer to have a generic mechanism that can serve us all. Note that I'm suggesting a completely new task framwork, at least on Vdsm side, as the current one (with its broken persistence, arcane states and never-reliable rollback) is beyond redemption, imho. 5. When the job has completed, VDS Broker raises an event up to bll. Maybe this could be done via VmJobDAO on the stored VmJob? 6. Bll receives the event and issues a series of VDS commands to complete the operation: a) Verify the new image chain matches our expectations (the snap is no longer present in the chain). b) Delete the snapshot volume c) Remove the VmJob from the DB Could you guys review this proposed flow for sanity? The main conceptual gaps I am left with concern #5 and #6. What is the appropriate way for VDSBroker to communicate with BLL? Is there an event mechanism I can explore or should I use the database? I am leaning toward the database because it is persistent and will ensure #6 gets completed even if engine is restarted somewhere in the middle. For #6, is there an existing polling / event loop in bll that I can plug into? Thanks in advance for taking the time to think about this flow and for providing your insights! ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Asynchronous tasks for live merge
On 03/03/2014 04:28 PM, Dan Kenigsberg wrote: On Fri, Feb 28, 2014 at 09:30:16AM -0500, Adam Litke wrote: Hi all, As part of our plan to support live merging of VM disk snapshots it seems we will need a new form of asynchronous task in ovirt-engine. I am aware of AsyncTaskManager but it seems to be limited to managing SPM tasks. For live merge, we are going to need something called VmTasks since the async command can be run only on the host that currently runs the VM. The way I see this working from an engine perspective is: 1. RemoveSnapshotCommand in bll is invoked as usual but since the VM is found to be up, we activate an alternative live merge flow. 2. We submit a LiveMerge VDS Command for each impacted disk. This is an asynchronous command which we need to monitor for completion. 3. A VmJob is inserted into the DB so we'll remember to handle it. 4. The VDS Broker monitors the operation via an extension to the already collected VmStatistics data. Vdsm will report active Block Jobs only. Once the job stops (in error or success) it will cease to be reported by vdsm and engine will know to proceed. You describe a reasonable way for Vdsm to report whether an async operation has finished. However, may we instead use the oportunity to introduce generic hsm tasks? I suggest to have something loosely modeled on posix fork/wait. - Engine asks Vdsm to start an API verb asynchronously and supplies a uuid. This is unlike fork(2), where the system chooses the pid, but that's required so that Engine could tell if the command has reached Vdsm in case of a network error. - Engine may monitor the task (a-la wait(WNOHANG)) - When the task is finished, Engine may collect its result (a-la wait). Until that happens, Vdsm must report the task forever; restart or upgrade are no excuses. On reboot, though, all tasks are forgotten, so Engine may stop monitoring tasks on a fenced host. This may be an over kill for your use case, but it would come useful for other cases. In particular, setupNetwork returns before it is completely done, since dhcp address acquisition may take too much time. Engine may poll getVdsCaps to see when it's done (or timeout), but it would be nicer to have a generic mechanism that can serve us all. Note that I'm suggesting a completely new task framwork, at least on Vdsm side, as the current one (with its broken persistence, arcane states and never-reliable rollback) is beyond redemption, imho. 5. When the job has completed, VDS Broker raises an event up to bll. Maybe this could be done via VmJobDAO on the stored VmJob? 6. Bll receives the event and issues a series of VDS commands to complete the operation: a) Verify the new image chain matches our expectations (the snap is no longer present in the chain). b) Delete the snapshot volume c) Remove the VmJob from the DB Could you guys review this proposed flow for sanity? The main conceptual gaps I am left with concern #5 and #6. What is the appropriate way for VDSBroker to communicate with BLL? Is there an event mechanism I can explore or should I use the database? I am leaning toward the database because it is persistent and will ensure #6 gets completed even if engine is restarted somewhere in the middle. For #6, is there an existing polling / event loop in bll that I can plug into? Thanks in advance for taking the time to think about this flow and for providing your insights! ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel the way i read Adam's proposal, there is no task entity at vdsm side to monitor, rather the state of the object the operation is performed on (similar to CreateVM, where the engine monitors the state of the VM, rather than the CreateVM request). ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Install rhevm-guest-agent on SLES guest
On 03/02/2014 10:46 PM, Itamar Heim wrote: On 02/21/2014 03:43 PM, Alexander Wels wrote: Doesn't look like the package exists yet, there is an open bug for it: https://bugzilla.redhat.com/show_bug.cgi?id=1043471 On Friday, February 21, 2014 11:00:40 AM Vikas Kokare wrote: We want to install the rhevm-guest-agent package on a virtual machine with Suse Linux as the guest OS. The RHEVM 3.2 documentation provides install instructions for RHEL and Windows guests but not for Suse Linux. This agent is important to monitor virtual resources on guest machines, hence the need. How can this be done? ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel is the suse one relevant for this? http://www.ovirt.org/Feature/GuestAgentOpenSUSE Only if it is openSUSE, SLES won't currently work though. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel -- Regards, Vinzenz Feenstra | Senior Software Engineer RedHat Engineering Virtualization R D Phone: +420 532 294 625 IRC: vfeenstr or evilissimo Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Asynchronous tasks for live merge
On 03/03/14 14:28 +, Dan Kenigsberg wrote: On Fri, Feb 28, 2014 at 09:30:16AM -0500, Adam Litke wrote: Hi all, As part of our plan to support live merging of VM disk snapshots it seems we will need a new form of asynchronous task in ovirt-engine. I am aware of AsyncTaskManager but it seems to be limited to managing SPM tasks. For live merge, we are going to need something called VmTasks since the async command can be run only on the host that currently runs the VM. The way I see this working from an engine perspective is: 1. RemoveSnapshotCommand in bll is invoked as usual but since the VM is found to be up, we activate an alternative live merge flow. 2. We submit a LiveMerge VDS Command for each impacted disk. This is an asynchronous command which we need to monitor for completion. 3. A VmJob is inserted into the DB so we'll remember to handle it. 4. The VDS Broker monitors the operation via an extension to the already collected VmStatistics data. Vdsm will report active Block Jobs only. Once the job stops (in error or success) it will cease to be reported by vdsm and engine will know to proceed. You describe a reasonable way for Vdsm to report whether an async operation has finished. However, may we instead use the oportunity to introduce generic hsm tasks? Sure, I am happy to have that conversation :) If I understand correctly, HSM tasks, while ideal, might be too complex to get right and would block the Live Merge feature for longer than we would like. Has anyone looked into what it would take to implement a HSM Tasks framework like this in vdsm? Are there any WIP implementations? If the scope of this is not too big, it can be completed relatively quickly, and the resulting implementation would cover all known use cases, then this could be worth it. It's important to support Live Merge soon. Regarding deprecation of the current tasks API: Could your suggested HSM Tasks framework be extended to cover SPM/SDM tasks as well? I would hope that a it could. In that case, we could look forward to a unified async task architecture in vdsm. I suggest to have something loosely modeled on posix fork/wait. - Engine asks Vdsm to start an API verb asynchronously and supplies a uuid. This is unlike fork(2), where the system chooses the pid, but that's required so that Engine could tell if the command has reached Vdsm in case of a network error. - Engine may monitor the task (a-la wait(WNOHANG)) Allon has communicated a desire to limit engine-side polling. Perhaps the active tasks could be added to the host stats? - When the task is finished, Engine may collect its result (a-la wait). Until that happens, Vdsm must report the task forever; restart or upgrade are no excuses. On reboot, though, all tasks are forgotten, so Engine may stop monitoring tasks on a fenced host. This could be a good comprimise. I hate the idea of requiring engine to play janitor and clean up stale vdsm data, but there is not much better of a way to do it. Allowing reboot to auto-clear tasks will at least provide some backstop to how long tasks could pile up if forgotten. This may be an over kill for your use case, but it would come useful for other cases. In particular, setupNetwork returns before it is completely done, since dhcp address acquisition may take too much time. Engine may poll getVdsCaps to see when it's done (or timeout), but it would be nicer to have a generic mechanism that can serve us all. If we were to consider this, I would want to vet the architecture against all known use cases for tasks to make sure we don't need to create a new framework in 3 months. Note that I'm suggesting a completely new task framwork, at least on Vdsm side, as the current one (with its broken persistence, arcane states and never-reliable rollback) is beyond redemption, imho. Are we okay with abandoning vdsm-side rollback entirely as we move forward? Won't that be a regression for at least some error flows (especially in the realm of SPM tasks)? 5. When the job has completed, VDS Broker raises an event up to bll. Maybe this could be done via VmJobDAO on the stored VmJob? 6. Bll receives the event and issues a series of VDS commands to complete the operation: a) Verify the new image chain matches our expectations (the snap is no longer present in the chain). b) Delete the snapshot volume c) Remove the VmJob from the DB Could you guys review this proposed flow for sanity? The main conceptual gaps I am left with concern #5 and #6. What is the appropriate way for VDSBroker to communicate with BLL? Is there an event mechanism I can explore or should I use the database? I am leaning toward the database because it is persistent and will ensure #6 gets completed even if engine is restarted somewhere in the middle. For #6, is there an existing polling / event loop in bll that I can plug into? Thanks in advance for taking the time to think about this flow and for providing
[Engine-devel] oVirt February 2014 Updates
1. Releases - oVirt 3.3.3 was released early in the month: http://www.ovirt.org/OVirt_3.3.3_release_notes - oVirt 3.3.4 about to release. http://www.ovirt.org/OVirt_3.3.4_release_notes - oVirt 3.4.0 about to release! 2. Events - Leonardo Vaz is organizing ovirt attendance at FISL15, the largest FOSS conference in LATAM which will happen from 7th to 10th of May in Porto Alegre, Brazil: http://softwarelivre.org/fisl15 - Allon Mureinik gave a presentation on DR with oVirt at devconf.cz http://www.slideshare.net/AllonMureinik/dev-conf-ovirt-dr - oVirt workshop in korea slides (korean) http://www.slideshare.net/rogan/20140208-ovirtkorea-01 https://www.facebook.com/groups/ovirt.korea - Rogan also presented oVirt integration with OpenStack in OpenStack day in Korea http://alturl.com/m3jnx - Pat Pierson posted on basic network setup http://izen.ghostpeppersrus.com/setting-up-networks/ - Fosdem 2014 sessions (slides and videos) are at: http://www.ovirt.org/FOSDEM_2014 - and some at Infrastructure.Next Ghent the week after forsdem. 3. oVirt Activity (software) - oVirt Jenkins plugin by Dustin Kut Moy Cheung to control VM slaves managed by ovirt/RHEV https://github.com/thescouser89/ovirt-slaves-plugin - Opaque oVirt/RHEV/Proxmox client and source code released https://play.google.com/store/apps/details?id=com.undatech.opaque - great to see the NUMA push from HP: http://www.ovirt.org/Features/NUMA_and_Virtual_NUMA http://www.ovirt.org/Features/Detailed_NUMA_and_Virtual_NUMA 4. oVirt Activity (blogs, preso's) - oVirt's has been accepted as a mentoring project for the Google Summer of Code 2014. - Oved Ourfali posted on Importing Glance images as oVirt templates http://alturl.com/h7xid - v2v had seen many active discussions. here's a post by Jon Archer on how to Import regular kvm image to oVirt or RHEV http://jonarcher.info/2014/02/import-regular-kvm-image-ovirt-rhev/ - great reviews on amazon.com for Getting Started with oVirt 3.3 http://alturl.com/5rk2p - oVirt Deep Dive 3.3 slides (Chinese) http://www.slideshare.net/mobile/johnwoolee/ovirt-deep-dive# - oVirt intro video (russian) http://alturl.com/it546 - how to install oVirt 3.3 on CentOS 6.5 http://www.youtube.com/watch?v=5i5ilSKsmbo 5. Related - NetApp GA'd their Virtual Storage Console for RHEV, which is implemented as an oVirt UI plugin (and then some) http://captainkvm.com/2014/02/vsc-for-rhev-is-ga-today/#more-660 ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Asynchronous tasks for live merge
On 03/03/14 16:36 +0200, Itamar Heim wrote: On 03/03/2014 04:28 PM, Dan Kenigsberg wrote: On Fri, Feb 28, 2014 at 09:30:16AM -0500, Adam Litke wrote: Hi all, As part of our plan to support live merging of VM disk snapshots it seems we will need a new form of asynchronous task in ovirt-engine. I am aware of AsyncTaskManager but it seems to be limited to managing SPM tasks. For live merge, we are going to need something called VmTasks since the async command can be run only on the host that currently runs the VM. The way I see this working from an engine perspective is: 1. RemoveSnapshotCommand in bll is invoked as usual but since the VM is found to be up, we activate an alternative live merge flow. 2. We submit a LiveMerge VDS Command for each impacted disk. This is an asynchronous command which we need to monitor for completion. 3. A VmJob is inserted into the DB so we'll remember to handle it. 4. The VDS Broker monitors the operation via an extension to the already collected VmStatistics data. Vdsm will report active Block Jobs only. Once the job stops (in error or success) it will cease to be reported by vdsm and engine will know to proceed. You describe a reasonable way for Vdsm to report whether an async operation has finished. However, may we instead use the oportunity to introduce generic hsm tasks? I suggest to have something loosely modeled on posix fork/wait. - Engine asks Vdsm to start an API verb asynchronously and supplies a uuid. This is unlike fork(2), where the system chooses the pid, but that's required so that Engine could tell if the command has reached Vdsm in case of a network error. - Engine may monitor the task (a-la wait(WNOHANG)) - When the task is finished, Engine may collect its result (a-la wait). Until that happens, Vdsm must report the task forever; restart or upgrade are no excuses. On reboot, though, all tasks are forgotten, so Engine may stop monitoring tasks on a fenced host. This may be an over kill for your use case, but it would come useful for other cases. In particular, setupNetwork returns before it is completely done, since dhcp address acquisition may take too much time. Engine may poll getVdsCaps to see when it's done (or timeout), but it would be nicer to have a generic mechanism that can serve us all. Note that I'm suggesting a completely new task framwork, at least on Vdsm side, as the current one (with its broken persistence, arcane states and never-reliable rollback) is beyond redemption, imho. 5. When the job has completed, VDS Broker raises an event up to bll. Maybe this could be done via VmJobDAO on the stored VmJob? 6. Bll receives the event and issues a series of VDS commands to complete the operation: a) Verify the new image chain matches our expectations (the snap is no longer present in the chain). b) Delete the snapshot volume c) Remove the VmJob from the DB Could you guys review this proposed flow for sanity? The main conceptual gaps I am left with concern #5 and #6. What is the appropriate way for VDSBroker to communicate with BLL? Is there an event mechanism I can explore or should I use the database? I am leaning toward the database because it is persistent and will ensure #6 gets completed even if engine is restarted somewhere in the middle. For #6, is there an existing polling / event loop in bll that I can plug into? Thanks in advance for taking the time to think about this flow and for providing your insights! ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel the way i read Adam's proposal, there is no task entity at vdsm side to monitor, rather the state of the object the operation is performed on (similar to CreateVM, where the engine monitors the state of the VM, rather than the CreateVM request). Yeah, we use the term job in order to avoid assumptions and implications (ie. rollback/cancel, persistence) that come with the word task. Job essentially means libvirt Block Job, but I am trying to allow for extension in the future. Vdsm would collect block job information for devices it expects to have active block jobs and report them all under a single structure in the VM statistics. There would be no persistence of information so when a libvirt block job goes poof, vdsm will stop reporting it. -- Adam Litke ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Asynchronous tasks for live merge
On Mon, Mar 03, 2014 at 09:56:56AM -0500, Adam Litke wrote: On 03/03/14 16:36 +0200, Itamar Heim wrote: On 03/03/2014 04:28 PM, Dan Kenigsberg wrote: On Fri, Feb 28, 2014 at 09:30:16AM -0500, Adam Litke wrote: Hi all, As part of our plan to support live merging of VM disk snapshots it seems we will need a new form of asynchronous task in ovirt-engine. I am aware of AsyncTaskManager but it seems to be limited to managing SPM tasks. For live merge, we are going to need something called VmTasks since the async command can be run only on the host that currently runs the VM. The way I see this working from an engine perspective is: 1. RemoveSnapshotCommand in bll is invoked as usual but since the VM is found to be up, we activate an alternative live merge flow. 2. We submit a LiveMerge VDS Command for each impacted disk. This is an asynchronous command which we need to monitor for completion. 3. A VmJob is inserted into the DB so we'll remember to handle it. 4. The VDS Broker monitors the operation via an extension to the already collected VmStatistics data. Vdsm will report active Block Jobs only. Once the job stops (in error or success) it will cease to be reported by vdsm and engine will know to proceed. You describe a reasonable way for Vdsm to report whether an async operation has finished. However, may we instead use the oportunity to introduce generic hsm tasks? I suggest to have something loosely modeled on posix fork/wait. - Engine asks Vdsm to start an API verb asynchronously and supplies a uuid. This is unlike fork(2), where the system chooses the pid, but that's required so that Engine could tell if the command has reached Vdsm in case of a network error. - Engine may monitor the task (a-la wait(WNOHANG)) - When the task is finished, Engine may collect its result (a-la wait). Until that happens, Vdsm must report the task forever; restart or upgrade are no excuses. On reboot, though, all tasks are forgotten, so Engine may stop monitoring tasks on a fenced host. This may be an over kill for your use case, but it would come useful for other cases. In particular, setupNetwork returns before it is completely done, since dhcp address acquisition may take too much time. Engine may poll getVdsCaps to see when it's done (or timeout), but it would be nicer to have a generic mechanism that can serve us all. Note that I'm suggesting a completely new task framwork, at least on Vdsm side, as the current one (with its broken persistence, arcane states and never-reliable rollback) is beyond redemption, imho. 5. When the job has completed, VDS Broker raises an event up to bll. Maybe this could be done via VmJobDAO on the stored VmJob? 6. Bll receives the event and issues a series of VDS commands to complete the operation: a) Verify the new image chain matches our expectations (the snap is no longer present in the chain). b) Delete the snapshot volume c) Remove the VmJob from the DB Could you guys review this proposed flow for sanity? The main conceptual gaps I am left with concern #5 and #6. What is the appropriate way for VDSBroker to communicate with BLL? Is there an event mechanism I can explore or should I use the database? I am leaning toward the database because it is persistent and will ensure #6 gets completed even if engine is restarted somewhere in the middle. For #6, is there an existing polling / event loop in bll that I can plug into? Thanks in advance for taking the time to think about this flow and for providing your insights! ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel the way i read Adam's proposal, there is no task entity at vdsm side to monitor, rather the state of the object the operation is performed on (similar to CreateVM, where the engine monitors the state of the VM, rather than the CreateVM request). Yeah, we use the term job in order to avoid assumptions and implications (ie. rollback/cancel, persistence) that come with the word task. Job essentially means libvirt Block Job, but I am trying to allow for extension in the future. Vdsm would collect block job information for devices it expects to have active block jobs and report them all under a single structure in the VM statistics. There would be no persistence of information so when a libvirt block job goes poof, vdsm will stop reporting it. I know, but since we need someothing quite similar for setupNetwork, I'm suggesting to have have something generic enough to fulfill both use cases. Instead of having one part of Engine poll for pending VmJobs, and another polling on whether a network finally got its address from the dhcp server. As Itamar said, a very similar logic exists for migration, and for starting up a new VM. I'm not suggesting rollback, cancelation or fancy
Re: [Engine-devel] Asynchronous tasks for live merge
On Mon, Mar 03, 2014 at 09:51:15AM -0500, Adam Litke wrote: On 03/03/14 14:28 +, Dan Kenigsberg wrote: On Fri, Feb 28, 2014 at 09:30:16AM -0500, Adam Litke wrote: Hi all, As part of our plan to support live merging of VM disk snapshots it seems we will need a new form of asynchronous task in ovirt-engine. I am aware of AsyncTaskManager but it seems to be limited to managing SPM tasks. For live merge, we are going to need something called VmTasks since the async command can be run only on the host that currently runs the VM. The way I see this working from an engine perspective is: 1. RemoveSnapshotCommand in bll is invoked as usual but since the VM is found to be up, we activate an alternative live merge flow. 2. We submit a LiveMerge VDS Command for each impacted disk. This is an asynchronous command which we need to monitor for completion. 3. A VmJob is inserted into the DB so we'll remember to handle it. 4. The VDS Broker monitors the operation via an extension to the already collected VmStatistics data. Vdsm will report active Block Jobs only. Once the job stops (in error or success) it will cease to be reported by vdsm and engine will know to proceed. You describe a reasonable way for Vdsm to report whether an async operation has finished. However, may we instead use the oportunity to introduce generic hsm tasks? Sure, I am happy to have that conversation :) If I understand correctly, HSM tasks, while ideal, might be too complex to get right and would block the Live Merge feature for longer than we would like. Has anyone looked into what it would take to implement a HSM Tasks framework like this in vdsm? Are there any WIP implementations? If the scope of this is not too big, it can be completed relatively quickly, and the resulting implementation would cover all known use cases, then this could be worth it. It's important to support Live Merge soon. Regarding deprecation of the current tasks API: Could your suggested HSM Tasks framework be extended to cover SPM/SDM tasks as well? I would hope that a it could. In that case, we could look forward to a unified async task architecture in vdsm. The current task framework in Vdsm is outrageously complex, yet unreliable. It meant to do all kinds of things, like having the new spm take over a task that was orphaned by the former spm. This has never worked properly. I'm looking for a much simpler infrastructure, where rollback is done by virtue of having an except clause, and spm-only verbs simply fail when the host loses spm status for some reason. I suggest to have something loosely modeled on posix fork/wait. - Engine asks Vdsm to start an API verb asynchronously and supplies a uuid. This is unlike fork(2), where the system chooses the pid, but that's required so that Engine could tell if the command has reached Vdsm in case of a network error. - Engine may monitor the task (a-la wait(WNOHANG)) Allon has communicated a desire to limit engine-side polling. Perhaps the active tasks could be added to the host stats? Engine is reluctant to add more polling, I understand that. I'd prefer a standalone new getAllTaskStats2() verb, but if lumping it into getVdsStats is going to convince everybody to have it, I'd put my aesthetic taste in the fridge. - When the task is finished, Engine may collect its result (a-la wait). Until that happens, Vdsm must report the task forever; restart or upgrade are no excuses. On reboot, though, all tasks are forgotten, so Engine may stop monitoring tasks on a fenced host. This could be a good comprimise. I hate the idea of requiring engine to play janitor and clean up stale vdsm data, but there is not much better of a way to do it. Allowing reboot to auto-clear tasks will at least provide some backstop to how long tasks could pile up if forgotten. This may be an over kill for your use case, but it would come useful for other cases. In particular, setupNetwork returns before it is completely done, since dhcp address acquisition may take too much time. Engine may poll getVdsCaps to see when it's done (or timeout), but it would be nicer to have a generic mechanism that can serve us all. If we were to consider this, I would want to vet the architecture against all known use cases for tasks to make sure we don't need to create a new framework in 3 months. I'm afraid that our time scale is a bit longer (for good and for worse), but for sure, we'd need to list all possible users of such an infrastructure. Note that I'm suggesting a completely new task framwork, at least on Vdsm side, as the current one (with its broken persistence, arcane states and never-reliable rollback) is beyond redemption, imho. Are we okay with abandoning vdsm-side rollback entirely as we move forward? Won't that be a regression for at least some error flows (especially in the realm of SPM tasks)? We would have to maintain the
[Engine-devel] Tweaking backup/restore of engine
Hi, currently all events are stored in the table audit_log which all gets saved when you use the engine-backup shell script. the event log is full of these login lines (engine 3.3.2): 25652 fdfc627c-d875-11e0-90f0-83df133b58ccadmin@internal ----\N \N \N \N \N 2014-01-20 06:39:17.222+01 USER_VDC_LOGIN 30 0 User admin@internal logged in. f \N \N ----\N \N \N \N ----\N oVirt -1 30 f \N this makes the log and db grow very large when you use the REST-API to query ovirt for various data. Is this necessary for a working restore? It would be cool if we could tweak the engine-backup tool to just dump necessary tables so you don't have to restore events from the past no one is interested in. How does ovirt react, if I do not restore the content of the audit_log table? If this works (restore without audit_log) I would prefer to have this code upstream in ovirt git so I don't have to maintain my own backupscript. Would it be possible to extend the existing backupscript with a switch to not backup logs? Currently it's just all or just db. I also recall that there shouldn't occur multiple login events any more since ovirt 3.3. but it still seems to be the case. I also do not understand how you would manage a stored authentication via REST as REST is stateless. I would appreciate any feedback or thoughts on this topic. -- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH Co. KG Königsberger Straße 6 32339 Espelkamp T: +49-5772-293-100 F: +49-5772-293-333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Tweaking backup/restore of engine and the table 'audit_log'
- Original Message - From: Sven Kieske s.kie...@mittwald.de To: engine-devel@ovirt.org Sent: Monday, March 3, 2014 6:25:39 PM Subject: [Engine-devel] Tweaking backup/restore of engine Hi, currently all events are stored in the table audit_log which all gets saved when you use the engine-backup shell script. the event log is full of these login lines (engine 3.3.2): 25652 fdfc627c-d875-11e0-90f0-83df133b58ccadmin@internal ---- \N \N \N \N \N 2014-01-20 06:39:17.222+01 USER_VDC_LOGIN 30 0 User admin@internal logged in.f \N \N ----\N \N \N \N----\N oVirt -1 30 f \N this makes the log and db grow very large when you use the REST-API to query ovirt for various data. Is this necessary for a working restore? I have no idea - I guess the data is not necessary. I also guess that the schema is. It would be cool if we could tweak the engine-backup tool to just dump necessary tables so you don't have to restore events from the past no one is interested in. How does ovirt react, if I do not restore the content of the audit_log table? If this works (restore without audit_log) I would prefer to have this code upstream in ovirt git so I don't have to maintain my own backupscript. Would it be possible to extend the existing backupscript with a switch to not backup logs? Currently it's just all or just db. It would be easy to let you pass an extra options argument for pg_dump. This will allow adding '-T audit_log'. As I said, I am pretty certain that you do need the table itself, so this will not help you much. I personally think that this isn't the right way to go. If you do not need the data, create a cron job that will periodically truncate it - say, keep the last X days and delete the rest. Perhaps also archive before deleting if you want. If you want, open a bug to provide a script to do that for you. Or make the engine itself do that, etc. Of course, after verifying that this does not have a significant impact on the engine :-) I also recall that there shouldn't occur multiple login events any more since ovirt 3.3. but it still seems to be the case. I also do not understand how you would manage a stored authentication via REST as REST is stateless. I would appreciate any feedback or thoughts on this topic. Best regards, -- Didi ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Schema upgrade failure on master
- Original Message - From: Adam Litke ali...@redhat.com To: engine-devel@ovirt.org Sent: Tuesday, March 4, 2014 12:26:28 AM Subject: [Engine-devel] Schema upgrade failure on master Hi, I've recently rebased to master and it looks like the 03_05_0050_event_notification_methods.sql script is failing on schema upgrade. Is this a bug or am I doing something wrong? To upgrade I did the normal proceedure with my development installation: make install-dev ... ~/ovirt/bin/engine-setup Got this result in the log file: psql:/home/alitke/ovirt-**FILTERED**/share/ovirt-**FILTERED**/dbscripts/upgrade/03_05_0050_event_notification_methods.sql:10: ERROR: column notification_method contains null values FATAL: Cannot execute sql command: --file=/home/alitke/ovirt-**FILTERED**/share/ovirt-**FILTERED**/dbscripts/upgrade/03_05_0050_event_notification_methods.sql 2014-03-03 17:20:34 DEBUG otopi.context context._executeMethod:152 method exception Traceback (most recent call last): File /usr/lib/python2.7/site-packages/otopi/context.py, line 142, in _executeMethod method['method']() File /home/alitke/ovirt-**FILTERED**/share/ovirt-**FILTERED**/setup/bin/../plugins/ovirt-**FILTERED**-setup/ovirt-**FILTERED**/db/schema.py, line 280, in _misc osetupcons.DBEnv.PGPASS_FILE File /usr/lib/python2.7/site-packages/otopi/plugin.py, line 451, in execute command=args[0], RuntimeError: Command '/home/alitke/ovirt-**FILTERED**/share/ovirt-**FILTERED**/dbscripts/schema.sh' failed to execute -- Adam Litke ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] Schema upgrade failure on master
Hi, I've recently rebased to master and it looks like the 03_05_0050_event_notification_methods.sql script is failing on schema upgrade. Is this a bug or am I doing something wrong? To upgrade I did the normal proceedure with my development installation: make install-dev ... ~/ovirt/bin/engine-setup Got this result in the log file: psql:/home/alitke/ovirt-**FILTERED**/share/ovirt-**FILTERED**/dbscripts/upgrade/03_05_0050_event_notification_methods.sql:10: ERROR: column notification_method contains null values FATAL: Cannot execute sql command: --file=/home/alitke/ovirt-**FILTERED**/share/ovirt-**FILTERED**/dbscripts/upgrade/03_05_0050_event_notification_methods.sql 2014-03-03 17:20:34 DEBUG otopi.context context._executeMethod:152 method exception Traceback (most recent call last): File /usr/lib/python2.7/site-packages/otopi/context.py, line 142, in _executeMethod method['method']() File /home/alitke/ovirt-**FILTERED**/share/ovirt-**FILTERED**/setup/bin/../plugins/ovirt-**FILTERED**-setup/ovirt-**FILTERED**/db/schema.py, line 280, in _misc osetupcons.DBEnv.PGPASS_FILE File /usr/lib/python2.7/site-packages/otopi/plugin.py, line 451, in execute command=args[0], RuntimeError: Command '/home/alitke/ovirt-**FILTERED**/share/ovirt-**FILTERED**/dbscripts/schema.sh' failed to execute -- Adam Litke ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Schema upgrade failure on master
Hi Adam, I didn't notice any problem with this script. Was the database you tried to upgrade empty? If not could you please send me contents of your event_subscriber table? Thanks Martin Perina - Original Message - From: Adam Litke ali...@redhat.com To: engine-devel@ovirt.org Sent: Monday, March 3, 2014 11:26:28 PM Subject: [Engine-devel] Schema upgrade failure on master Hi, I've recently rebased to master and it looks like the 03_05_0050_event_notification_methods.sql script is failing on schema upgrade. Is this a bug or am I doing something wrong? To upgrade I did the normal proceedure with my development installation: make install-dev ... ~/ovirt/bin/engine-setup Got this result in the log file: psql:/home/alitke/ovirt-**FILTERED**/share/ovirt-**FILTERED**/dbscripts/upgrade/03_05_0050_event_notification_methods.sql:10: ERROR: column notification_method contains null values FATAL: Cannot execute sql command: --file=/home/alitke/ovirt-**FILTERED**/share/ovirt-**FILTERED**/dbscripts/upgrade/03_05_0050_event_notification_methods.sql 2014-03-03 17:20:34 DEBUG otopi.context context._executeMethod:152 method exception Traceback (most recent call last): File /usr/lib/python2.7/site-packages/otopi/context.py, line 142, in _executeMethod method['method']() File /home/alitke/ovirt-**FILTERED**/share/ovirt-**FILTERED**/setup/bin/../plugins/ovirt-**FILTERED**-setup/ovirt-**FILTERED**/db/schema.py, line 280, in _misc osetupcons.DBEnv.PGPASS_FILE File /usr/lib/python2.7/site-packages/otopi/plugin.py, line 451, in execute command=args[0], RuntimeError: Command '/home/alitke/ovirt-**FILTERED**/share/ovirt-**FILTERED**/dbscripts/schema.sh' failed to execute -- Adam Litke ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Tweaking backup/restore of engine
- Original Message - From: Sven Kieske s.kie...@mittwald.de To: engine-devel@ovirt.org Sent: Monday, March 3, 2014 6:25:39 PM Subject: [Engine-devel] Tweaking backup/restore of engine Hi, currently all events are stored in the table audit_log which all gets saved when you use the engine-backup shell script. the event log is full of these login lines (engine 3.3.2): 25652 fdfc627c-d875-11e0-90f0-83df133b58ccadmin@internal ---- \N \N \N \N \N 2014-01-20 06:39:17.222+01 USER_VDC_LOGIN 30 0 User admin@internal logged in.f \N \N ----\N \N \N \N----\N oVirt -1 30 f \N this makes the log and db grow very large when you use the REST-API to query ovirt for various data. Is this necessary for a working restore? It would be cool if we could tweak the engine-backup tool to just dump necessary tables so you don't have to restore events from the past no one is interested in. How does ovirt react, if I do not restore the content of the audit_log table? If this works (restore without audit_log) I would prefer to have this code upstream in ovirt git so I don't have to maintain my own backupscript. Would it be possible to extend the existing backupscript with a switch to not backup logs? Currently it's just all or just db. I also recall that there shouldn't occur multiple login events any more since ovirt 3.3. but it still seems to be the case. Hi Sven, This is not entirely accurate - The solution introduced at commit hash cb56de8808cec33b7599828ead890f52e32bcaea solves the problem for a specific case in which we have a multiple login in a very short interval of time - mainly due to attempt to login from webadmin, while UI plugin tries to login as well. We have an anti flood mechanism for events, allowing us to define an interval, in which an event will not be logged twice. In the case of the login event this is set to 5 seconds, which is enough to solve the above described scenario. I also do not understand how you would manage a stored authentication via REST as REST is stateless. I would appreciate any feedback or thoughts on this topic. -- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH Co. KG Königsberger Straße 6 32339 Espelkamp T: +49-5772-293-100 F: +49-5772-293-333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
