Re: [Engine-devel] Proposal to add Shireesh Anjal as maintainer to engine-gluster
- Original Message - From: Itamar Heim ih...@redhat.com To: engine-devel@ovirt.org Sent: Friday, November 30, 2012 1:28:51 PM Subject: [Engine-devel] Proposal to add Shireesh Anjal as maintainer to engine-gluster Shireesh has been working on the engine gluster support for the past year, covering patches and leading reviews of the various engine gluster patches. I'd like to propose a new sub-component of the engine for the gluster module, adding shireesh as its maintainer. +1 Thanks, Itamar ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Proposal to add Shireesh Anjal as maintainer to engine-gluster
- Original Message - From: Omer Frenkel ofren...@redhat.com To: Itamar Heim ih...@redhat.com Cc: engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 10:29:34 AM Subject: Re: [Engine-devel] Proposal to add Shireesh Anjal as maintainer to engine-gluster - Original Message - From: Itamar Heim ih...@redhat.com To: engine-devel@ovirt.org Sent: Friday, November 30, 2012 1:28:51 PM Subject: [Engine-devel] Proposal to add Shireesh Anjal as maintainer to engine-gluster Shireesh has been working on the engine gluster support for the past year, covering patches and leading reviews of the various engine gluster patches. I'd like to propose a new sub-component of the engine for the gluster module, adding shireesh as its maintainer. +1 +1 Thanks, Itamar ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
[Engine-devel] host cpu feature
Hi, CPU-Host support allows the virtual machines to see and utilize the host's CPU flags, this enables better performance in VM's, at the price of worse portablity. http://www.ovirt.org/Features/Cpu-host_Support Your feedback is welcome! Thank you, Laszlo ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] [Users] RHEVM 3.1 DB Schema diagram
- Original Message - From: Allon Mureinik amure...@redhat.com To: Eli Mesika emes...@redhat.com Cc: users us...@ovirt.org, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 9:21:34 AM Subject: Re: [Users] RHEVM 3.1 DB Schema diagram Nice! How are the names of the blocks generated? Some names strike me a bit there (e.g., images is in the storage_domains_static block). Auto-generated ... (nice work IMHO) - Original Message - From: Eli Mesika emes...@redhat.com To: users us...@ovirt.org, engine-devel engine-devel@ovirt.org Sent: Tuesday, December 4, 2012 7:12:28 PM Subject: [Users] RHEVM 3.1 DB Schema diagram Hi There were some requests on the users list to get our DB schema diagram. After looking a while for a tool for generating that , I have found the DBSchema tool. Please find attached the RHEVM 3.1 DB Schema diagram. I intend to get it into the oVirt site but currently it's too large and does not fit well. Regards Eli Mesika ___ Users mailing list us...@ovirt.org http://lists.ovirt.org/mailman/listinfo/users ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] host cpu feature
On 12/05/2012 01:46 PM, Laszlo Hornyak wrote: - Original Message - From: Yaniv Kaul yk...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 12:23:47 PM Subject: Re: [Engine-devel] host cpu feature On 12/05/2012 12:32 PM, Laszlo Hornyak wrote: Hi, CPU-Host support allows the virtual machines to see and utilize the host's CPU flags, this enables better performance in VM's, at the price of worse portablity. http://www.ovirt.org/Features/Cpu-host_Support Your feedback is welcome! Thank you, Laszlo ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel - I assume that when you allow migration, you'd use host-model? This is not clear from the design. It seems like we VDSM developers can choose to use either this or passthrough, while in practice we should support both. If AllowMigrateCPUHost is set to true (in case you have the same cpu model everywhere in your DC) migration of such hsots will be enabled. Otherwise it will not be enabled. It's not going to help just enabling it - you need to use 'host-model' and not 'host-passthrough', so there'll be a reasonable chance to succeed the migration, AFAIK. - I'm still convinced and commented on both relevat oVirt and libvirt BZs that we need to add x2apic support to the CPU, regardless of what the host CPU exposes. AFAIK, the KVM developers agree with me. Not quite sure how is this related... could you send some URL's for the bugreports? It's related because x2apic improves performance, and so does '-cpu host', but that doesn't enable x2apic implicitly. https://bugzilla.redhat.com/show_bug.cgi?id=838469#c7 https://bugzilla.redhat.com/show_bug.cgi?id=700272#c28 Y. Y. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] host cpu feature
On 12/05/2012 03:39 PM, Laszlo Hornyak wrote: - Original Message - From: Dan Kenigsberg dan...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: Yaniv Kaul yk...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 1:55:19 PM Subject: Re: [Engine-devel] host cpu feature On Wed, Dec 05, 2012 at 06:46:09AM -0500, Laszlo Hornyak wrote: - Original Message - From: Yaniv Kaul yk...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 12:23:47 PM Subject: Re: [Engine-devel] host cpu feature On 12/05/2012 12:32 PM, Laszlo Hornyak wrote: Hi, CPU-Host support allows the virtual machines to see and utilize the host's CPU flags, this enables better performance in VM's, at the price of worse portablity. http://www.ovirt.org/Features/Cpu-host_Support Your feedback is welcome! Thank you, Laszlo ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel - I assume that when you allow migration, you'd use host-model? This is not clear from the design. It seems like we VDSM developers can choose to use either this or passthrough, while in practice we should support both. I join Kaul's question: it is an ovirt-level question whether hostPassthrough or hostModel or both should be supported. It should not be a unilateral vdsm decision. Ah, possibly misunderstanding, I did not mean that VDSM should decide whether to use host-passthrough or host-model. The engine should decide. I meant _you_ should decide which version of vdsm api modification do you want :) If AllowMigrateCPUHost is set to true (in case you have the same cpu model everywhere in your DC) migration of such hsots will be enabled. Otherwise it will not be enabled. What is the breadth of AllowMigrateCPUHost? Engine wide? Per DC? Per cluster? I thought of eninge-wide. The of course you can have different models in two different DC, but they should be unique in one. We can add this to DC or cluster level, imho it would be just another checkbox on the UI that most users would not use. I favor the latter; a user may have a cluster of exact-same hosts, where hostcpu migration is allowed, and other cluster where it is forbiden. The nice thing about hostModel (unlike hostPassthrough) is that once we created the VM we can migrate it to stronger hosts, and back to the original host. I suppose that it complicates the scheduler. Yes with host-model you get the features that libvirt handles. In such cases the engine could decide, if you want this functionality. Well the scheduler architecture is just being reinvented. For the host-passthrough, I think the AllowMigrateCPUHost configuration option would be a simple decision for the administrator: set it to true if all hosts are uniform. If it is not set to true, then we will not allow migration of such VMs. That's not what I understood from libvirt's documentation. I understood that if you want host+migration, you need to use host-model. Otherwise - host-passthrough. Y. - I'm still convinced and commented on both relevat oVirt and libvirt BZs that we need to add x2apic support to the CPU, regardless of what the host CPU exposes. AFAIK, the KVM developers agree with me. Not quite sure how is this related... could you send some URL's for the bugreports? ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] host cpu feature
- Original Message - From: Yaniv Kaul yk...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: Dan Kenigsberg dan...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 2:45:46 PM Subject: Re: [Engine-devel] host cpu feature On 12/05/2012 03:39 PM, Laszlo Hornyak wrote: - Original Message - From: Dan Kenigsberg dan...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: Yaniv Kaul yk...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 1:55:19 PM Subject: Re: [Engine-devel] host cpu feature On Wed, Dec 05, 2012 at 06:46:09AM -0500, Laszlo Hornyak wrote: - Original Message - From: Yaniv Kaul yk...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 12:23:47 PM Subject: Re: [Engine-devel] host cpu feature On 12/05/2012 12:32 PM, Laszlo Hornyak wrote: Hi, CPU-Host support allows the virtual machines to see and utilize the host's CPU flags, this enables better performance in VM's, at the price of worse portablity. http://www.ovirt.org/Features/Cpu-host_Support Your feedback is welcome! Thank you, Laszlo ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel - I assume that when you allow migration, you'd use host-model? This is not clear from the design. It seems like we VDSM developers can choose to use either this or passthrough, while in practice we should support both. I join Kaul's question: it is an ovirt-level question whether hostPassthrough or hostModel or both should be supported. It should not be a unilateral vdsm decision. Ah, possibly misunderstanding, I did not mean that VDSM should decide whether to use host-passthrough or host-model. The engine should decide. I meant _you_ should decide which version of vdsm api modification do you want :) If AllowMigrateCPUHost is set to true (in case you have the same cpu model everywhere in your DC) migration of such hsots will be enabled. Otherwise it will not be enabled. What is the breadth of AllowMigrateCPUHost? Engine wide? Per DC? Per cluster? I thought of eninge-wide. The of course you can have different models in two different DC, but they should be unique in one. We can add this to DC or cluster level, imho it would be just another checkbox on the UI that most users would not use. I favor the latter; a user may have a cluster of exact-same hosts, where hostcpu migration is allowed, and other cluster where it is forbiden. The nice thing about hostModel (unlike hostPassthrough) is that once we created the VM we can migrate it to stronger hosts, and back to the original host. I suppose that it complicates the scheduler. Yes with host-model you get the features that libvirt handles. In such cases the engine could decide, if you want this functionality. Well the scheduler architecture is just being reinvented. For the host-passthrough, I think the AllowMigrateCPUHost configuration option would be a simple decision for the administrator: set it to true if all hosts are uniform. If it is not set to true, then we will not allow migration of such VMs. That's not what I understood from libvirt's documentation. I You may be right, could you send an URL to that point of the documentation or copy-paste? understood that if you want host+migration, you need to use host-model. Otherwise - host-passthrough. Y. - I'm still convinced and commented on both relevat oVirt and libvirt BZs that we need to add x2apic support to the CPU, regardless of what the host CPU exposes. AFAIK, the KVM developers agree with me. Not quite sure how is this related... could you send some URL's for the bugreports? ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] host cpu feature
On Wed, Dec 05, 2012 at 04:05:00PM +0200, Yaniv Kaul wrote: On 12/05/2012 03:55 PM, Laszlo Hornyak wrote: The nice thing about hostModel (unlike hostPassthrough) is that once we created the VM we can migrate it to stronger hosts, and back to the original host. I suppose that it complicates the scheduler. Yes with host-model you get the features that libvirt handles. In such cases the engine could decide, if you want this functionality. Well the scheduler architecture is just being reinvented. For the host-passthrough, I think the AllowMigrateCPUHost configuration option would be a simple decision for the administrator: set it to true if all hosts are uniform. If it is THAT simple, Engine could take this decision without human intervension. If it is not set to true, then we will not allow migration of such VMs. That's not what I understood from libvirt's documentation. I You may be right, could you send an URL to that point of the documentation or copy-paste? The link I followed from your feature page: http://libvirt.org/formatdomain.html#elementsCPU : host-model The host-model mode is essentially a shortcut to copying host CPU definition from capabilities XML into domain XML. Since the CPU definition is copied just before starting a domain, exactly the same XML can be used on different hosts while still providing the best guest CPU each host supports. Neither match attribute nor any feature elements can be used in this mode. Specifying CPU model is not supported either, but model's fallback attribute may still be used. Libvirt does not model every aspect of each CPU so the guest CPU will not match the host CPU exactly. On the other hand, the ABI provided to the guest is reproducible. During migration, complete CPU model definition is transferred to the destination host so the migrated guest will see exactly the same CPU model even if the destination host contains more capable CPUs for the running instance of the guest; but shutting down and restarting the guest may present different hardware to the guest according to the capabilities of the new host. host-passthrough With this mode, the CPU visible to the guest should be exactly the same as the host CPU even in the aspects that libvirt does not understand. Though the downside of this mode is that the guest environment cannot be reproduced on different hardware. Thus, if you hit any bugs, you are on your own. That's exactly where AllowMigrateCPUHost fits well: when a user ticks this for a cluster he says yeah, I like to be on my own. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] host cpu feature
- Original Message - From: Yaniv Kaul yk...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: Dan Kenigsberg dan...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 3:05:00 PM Subject: Re: [Engine-devel] host cpu feature On 12/05/2012 03:55 PM, Laszlo Hornyak wrote: - Original Message - From: Yaniv Kaul yk...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: Dan Kenigsberg dan...@redhat.com , engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 2:45:46 PM Subject: Re: [Engine-devel] host cpu feature On 12/05/2012 03:39 PM, Laszlo Hornyak wrote: - Original Message - From: Dan Kenigsberg dan...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: Yaniv Kaul yk...@redhat.com , engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 1:55:19 PM Subject: Re: [Engine-devel] host cpu feature On Wed, Dec 05, 2012 at 06:46:09AM -0500, Laszlo Hornyak wrote: - Original Message - From: Yaniv Kaul yk...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 12:23:47 PM Subject: Re: [Engine-devel] host cpu feature On 12/05/2012 12:32 PM, Laszlo Hornyak wrote: Hi, CPU-Host support allows the virtual machines to see and utilize the host's CPU flags, this enables better performance in VM's, at the price of worse portablity. http://www.ovirt.org/Features/Cpu-host_Support Your feedback is welcome! Thank you, Laszlo ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel - I assume that when you allow migration, you'd use host-model? This is not clear from the design. It seems like we VDSM developers can choose to use either this or passthrough, while in practice we should support both. I join Kaul's question: it is an ovirt-level question whether hostPassthrough or hostModel or both should be supported. It should not be a unilateral vdsm decision. Ah, possibly misunderstanding, I did not mean that VDSM should decide whether to use host-passthrough or host-model. The engine should decide. I meant _you_ should decide which version of vdsm api modification do you want :) If AllowMigrateCPUHost is set to true (in case you have the same cpu model everywhere in your DC) migration of such hsots will be enabled. Otherwise it will not be enabled. What is the breadth of AllowMigrateCPUHost? Engine wide? Per DC? Per cluster? I thought of eninge-wide. The of course you can have different models in two different DC, but they should be unique in one. We can add this to DC or cluster level, imho it would be just another checkbox on the UI that most users would not use. I favor the latter; a user may have a cluster of exact-same hosts, where hostcpu migration is allowed, and other cluster where it is forbiden. The nice thing about hostModel (unlike hostPassthrough) is that once we created the VM we can migrate it to stronger hosts, and back to the original host. I suppose that it complicates the scheduler. Yes with host-model you get the features that libvirt handles. In such cases the engine could decide, if you want this functionality. Well the scheduler architecture is just being reinvented. For the host-passthrough, I think the AllowMigrateCPUHost configuration option would be a simple decision for the administrator: set it to true if all hosts are uniform. If it is not set to true, then we will not allow migration of such VMs. That's not what I understood from libvirt's documentation. I You may be right, could you send an URL to that point of the documentation or copy-paste? The link I followed from your feature page: http://libvirt.org/formatdomain.html#elementsCPU : host-model The host-model mode is essentially a shortcut to copying host CPU definition from capabilities XML into domain XML. Since the CPU definition is copied just before starting a domain, exactly the same XML can be used on different hosts while still providing the best guest CPU each host supports. Neither match attribute nor any feature elements can be used in this mode. Specifying CPU model is not supported either, but model's fallback attribute may still be used. Libvirt does not model every aspect of each CPU so the guest CPU will not match the host CPU exactly. On the other hand, the ABI provided to the guest is reproducible. During migration, complete CPU model definition is transferred to the destination host so the migrated guest will see exactly the same CPU model even if the destination host contains more capable CPUs for the running instance of the guest; but shutting down and restarting the guest may present different hardware to the guest according to the capabilities of the new host. host-passthrough With this mode, the CPU visible to
Re: [Engine-devel] [vdsm] VDSM tasks, the future
I'm sorry but your email client messed up the formatting and I can't figure out what are you comments. Could you please use text only emails. - Original Message - From: ybronhei ybron...@redhat.com To: Saggi Mizrahi smizr...@redhat.com Cc: Adam Litke a...@us.ibm.com, engine-devel engine-devel@ovirt.org, VDSM Project Development vdsm-de...@lists.fedorahosted.org Sent: Wednesday, December 5, 2012 8:37:23 AM Subject: Re: [vdsm] VDSM tasks, the future On 12/05/2012 12:20 AM, Saggi Mizrahi wrote: As the only subsystem to use asynchronous tasks until now is the storage subsystem I suggest going over how I suggest tackling task creation, task stop, task remove and task recovery. Other subsystem can create similar mechanisms depending on their needs. There is no way of avoiding it, different types of tasks need different ways of tracking\recovering from them. network should always auto-recover because it can't get a please fix command if the network is down. Storage on the other hand should never start operations on it's own because it might take up valuable resources from the host. Tasks that need to be tracked on a single host, 2 hosts, or the entire cluster need to have their own APIs. VM configuration never persist across reboots, networking sometimes persists and storage always persists. This means that recovery procedures (from the managers point of view) need to be vastly different. Add policy, resource allocation, and error flows you see that VDSM doesn't have nearly as much information to deal with the tasks. - Original Message - From: Adam Litke a...@us.ibm.com To: Saggi Mizrahi smizr...@redhat.com Cc: VDSM Project Development vdsm-de...@lists.fedorahosted.org , engine-devel engine-devel@ovirt.org , Ayal Baron aba...@redhat.com , Barak Azulay bazu...@redhat.com , Shireesh Anjal san...@redhat.com Sent: Tuesday, December 4, 2012 3:50:28 PM Subject: Re: VDSM tasks, the future On Tue, Dec 04, 2012 at 10:35:01AM -0500, Saggi Mizrahi wrote: Because I started hinting about how VDSM tasks are going to look going forward I thought it's better I'll just write everything in an email so we can talk about it in context. This is not set in stone and I'm still debating things myself but it's very close to being done. Don't debate them yourself, debate them here! Even better, propose your idea in schema form to show how a command might work exactly. I don't like throwing ideas in the air It can be much easier to understand the flow of a task in vdsm and outside vdsm by a small schema, mainly for the each task's states. To define the flow of a task you can separate between type of tasks (network, storage, vms, or else), we should have task's states that clarify if the task can be recovered or not, can be canceled or not and inc.. Canceling\Aborting\Reverting states should be more clarified and not every state can lead to all types of states. I tries to figure how task flow works today in vdsm, and this is what I've got - http://wiki.ovirt.org/Vdsm_tasks - Everything is asynchronous. The nature of message based communication is that you can't have synchronous operations. This is not really debatable because it's just how TCP\AMQP\messaging works. Can you show how a traditionally synchronous command might work? Let's take Host.getVmList as an example. The same as it works today, it's all a matter of how you wrap the transport layer. You will send a json-rpc request and wait for a response with the same id. As for the bindings, there are a lot of way we can tackle that. Always wait for the response and simulate synchronous behavior. Make every method return an object to track the task. task = host.getVmList() if not task.wait(1): task.cancel() else: res = task.result() It looks like traditional timeout.. why not to split blocking actions and non-blocking actions, non-blocking action will supply callback function to return to if the task fails or success. for example: createAsyncTask(host.getVmList, params, timeout=30, callbackGetVmList) Instead of using the dispatcher? Do you want to keep the dispatcher concept? Have it both ways (it's auto generated anyway) and have list = host.getVmList() task = host.getVmList_async() Have a high level and low level interfaces. host = host() host.connect(tcp://host:3233) req = host.sendRequest(123213, getVmList, []) if not req.wait(1): shost = SynchHost(host) shost.getVmList() # Actually wraps a request object ahost = AsyncHost(host) task = getVmList() # Actually wraps a request object - Task IDs will be decided by the caller. This is how json-rpc works and also makes sense because no the engine can track the task without needing to have a stage where we give it the task ID back. IDs are reusable as long as no one else is using them at the time so they can be used for
Re: [Engine-devel] [vdsm] VDSM tasks, the future
On 12/05/2012 12:20 AM, Saggi Mizrahi wrote: As the only subsystem to use asynchronous tasks until now is the storage subsystem I suggest going over how I suggest tackling task creation, task stop, task remove and task recovery. Other subsystem can create similar mechanisms depending on their needs. There is no way of avoiding it, different types of tasks need different ways of tracking\recovering from them. network should always auto-recover because it can't get a please fix command if the network is down. Storage on the other hand should never start operations on it's own because it might take up valuable resources from the host. Tasks that need to be tracked on a single host, 2 hosts, or the entire cluster need to have their own APIs. VM configuration never persist across reboots, networking sometimes persists and storage always persists. This means that recovery procedures (from the managers point of view) need to be vastly different. Add policy, resource allocation, and error flows you see that VDSM doesn't have nearly as much information to deal with the tasks. - Original Message - From: Adam Litke a...@us.ibm.com To: Saggi Mizrahi smizr...@redhat.com Cc: VDSM Project Development vdsm-de...@lists.fedorahosted.org, engine-devel engine-devel@ovirt.org, Ayal Baron aba...@redhat.com, Barak Azulay bazu...@redhat.com, Shireesh Anjal san...@redhat.com Sent: Tuesday, December 4, 2012 3:50:28 PM Subject: Re: VDSM tasks, the future On Tue, Dec 04, 2012 at 10:35:01AM -0500, Saggi Mizrahi wrote: Because I started hinting about how VDSM tasks are going to look going forward I thought it's better I'll just write everything in an email so we can talk about it in context. This is not set in stone and I'm still debating things myself but it's very close to being done. Don't debate them yourself, debate them here! Even better, propose your idea in schema form to show how a command might work exactly. I don't like throwing ideas in the air It can be much easier to understand the flow of a task in vdsm and outside vdsm by a small schema, mainly for the each task's states. To define the flow of a task you can separate between type of tasks (network, storage, vms, or else), we should have task's states that clarify if the task can be recovered or not, can be canceled or not and inc.. Canceling\Aborting\Reverting states should be more clarified and not every state can lead to all types of states. I tries to figure how task flow works today in vdsm, and this is what I've got - http://wiki.ovirt.org/Vdsm_tasks - Everything is asynchronous. The nature of message based communication is that you can't have synchronous operations. This is not really debatable because it's just how TCP\AMQP\messaging works. Can you show how a traditionally synchronous command might work? Let's take Host.getVmList as an example. The same as it works today, it's all a matter of how you wrap the transport layer. You will send a json-rpc request and wait for a response with the same id. As for the bindings, there are a lot of way we can tackle that. Always wait for the response and simulate synchronous behavior. Make every method return an object to track the task. task = host.getVmList() if not task.wait(1): task.cancel() else: res = task.result() It looks like traditional timeout.. why not to split blocking actions and non-blocking actions, non-blocking action will supply callback function to return to if the task fails or success. for example: createAsyncTask(host.getVmList, params, timeout=30, callbackGetVmList) Instead of using the dispatcher? Do you want to keep the dispatcher concept? Have it both ways (it's auto generated anyway) and have list = host.getVmList() task = host.getVmList_async() Have a high level and low level interfaces. host = host() host.connect(tcp://host:3233) req = host.sendRequest(123213, getVmList, []) if not req.wait(1): shost = SynchHost(host) shost.getVmList() # Actually wraps a request object ahost = AsyncHost(host) task = getVmList() # Actually wraps a request object - Task IDs will be decided by the caller. This is how json-rpc works and also makes sense because no the engine can track the task without needing to have a stage where we give it the task ID back. IDs are reusable as long as no one else is using them at the time so they can be used for synchronizing operations between clients (making sure a command is only executed once on a specific host without locking). - Tasks are transient If VDSM restarts it forgets all the task information. There are 2 ways to have persistent tasks: 1. The task creates an object that you can continue work on in VDSM. The new storage does that by the fact that copyImage() returns one the target volume has been created but before the data has been fully copied. From that moment on the stat of the copy can be queried from any host using getImageStatus() and the specific copy operation can be queried
Re: [Engine-devel] host cpu feature
- Original Message - From: Dan Kenigsberg dan...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: Laszlo Hornyak lhorn...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 3:22:18 PM Subject: Re: [Engine-devel] host cpu feature On Wed, Dec 05, 2012 at 04:05:00PM +0200, Yaniv Kaul wrote: On 12/05/2012 03:55 PM, Laszlo Hornyak wrote: The nice thing about hostModel (unlike hostPassthrough) is that once we created the VM we can migrate it to stronger hosts, and back to the original host. I suppose that it complicates the scheduler. Yes with host-model you get the features that libvirt handles. In such cases the engine could decide, if you want this functionality. Well the scheduler architecture is just being reinvented. For the host-passthrough, I think the AllowMigrateCPUHost configuration option would be a simple decision for the administrator: set it to true if all hosts are uniform. If it is THAT simple, Engine could take this decision without human intervension. Is there a way engine can figure out if the cpu-models in all the hosts are the same? I mean even if some host flags are not handled by libvirt and therefore vdsm and engine... So I would really need that permission from the user. If it is not set to true, then we will not allow migration of such VMs. That's not what I understood from libvirt's documentation. I You may be right, could you send an URL to that point of the documentation or copy-paste? The link I followed from your feature page: http://libvirt.org/formatdomain.html#elementsCPU : host-model The host-model mode is essentially a shortcut to copying host CPU definition from capabilities XML into domain XML. Since the CPU definition is copied just before starting a domain, exactly the same XML can be used on different hosts while still providing the best guest CPU each host supports. Neither match attribute nor any feature elements can be used in this mode. Specifying CPU model is not supported either, but model's fallback attribute may still be used. Libvirt does not model every aspect of each CPU so the guest CPU will not match the host CPU exactly. On the other hand, the ABI provided to the guest is reproducible. During migration, complete CPU model definition is transferred to the destination host so the migrated guest will see exactly the same CPU model even if the destination host contains more capable CPUs for the running instance of the guest; but shutting down and restarting the guest may present different hardware to the guest according to the capabilities of the new host. host-passthrough With this mode, the CPU visible to the guest should be exactly the same as the host CPU even in the aspects that libvirt does not understand. Though the downside of this mode is that the guest environment cannot be reproduced on different hardware. Thus, if you hit any bugs, you are on your own. That's exactly where AllowMigrateCPUHost fits well: when a user ticks this for a cluster he says yeah, I like to be on my own. cpu mode=host-passthrough migration: I talked to the libvirt guys and they said it is OK if the hardware and the software are the same, and it will work, but they would not recommend. So if they do not recommend it, I would drop this from the feature spec. Anyone against it? Laszlo ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] host cpu feature
On 12/05/2012 03:55 PM, Laszlo Hornyak wrote: - Original Message - From: Yaniv Kaul yk...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: Dan Kenigsberg dan...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 2:45:46 PM Subject: Re: [Engine-devel] host cpu feature On 12/05/2012 03:39 PM, Laszlo Hornyak wrote: - Original Message - From: Dan Kenigsberg dan...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: Yaniv Kaul yk...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 1:55:19 PM Subject: Re: [Engine-devel] host cpu feature On Wed, Dec 05, 2012 at 06:46:09AM -0500, Laszlo Hornyak wrote: - Original Message - From: Yaniv Kaul yk...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 12:23:47 PM Subject: Re: [Engine-devel] host cpu feature On 12/05/2012 12:32 PM, Laszlo Hornyak wrote: Hi, CPU-Host support allows the virtual machines to see and utilize the host's CPU flags, this enables better performance in VM's, at the price of worse portablity. http://www.ovirt.org/Features/Cpu-host_Support Your feedback is welcome! Thank you, Laszlo ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel - I assume that when you allow migration, you'd use host-model? This is not clear from the design. It seems like we VDSM developers can choose to use either this or passthrough, while in practice we should support both. I join Kaul's question: it is an ovirt-level question whether hostPassthrough or hostModel or both should be supported. It should not be a unilateral vdsm decision. Ah, possibly misunderstanding, I did not mean that VDSM should decide whether to use host-passthrough or host-model. The engine should decide. I meant _you_ should decide which version of vdsm api modification do you want :) If AllowMigrateCPUHost is set to true (in case you have the same cpu model everywhere in your DC) migration of such hsots will be enabled. Otherwise it will not be enabled. What is the breadth of AllowMigrateCPUHost? Engine wide? Per DC? Per cluster? I thought of eninge-wide. The of course you can have different models in two different DC, but they should be unique in one. We can add this to DC or cluster level, imho it would be just another checkbox on the UI that most users would not use. I favor the latter; a user may have a cluster of exact-same hosts, where hostcpu migration is allowed, and other cluster where it is forbiden. The nice thing about hostModel (unlike hostPassthrough) is that once we created the VM we can migrate it to stronger hosts, and back to the original host. I suppose that it complicates the scheduler. Yes with host-model you get the features that libvirt handles. In such cases the engine could decide, if you want this functionality. Well the scheduler architecture is just being reinvented. For the host-passthrough, I think the AllowMigrateCPUHost configuration option would be a simple decision for the administrator: set it to true if all hosts are uniform. If it is not set to true, then we will not allow migration of such VMs. That's not what I understood from libvirt's documentation. I You may be right, could you send an URL to that point of the documentation or copy-paste? The link I followed from your feature page: http://libvirt.org/formatdomain.html#elementsCPU : host-model The host-model mode is essentially a shortcut to copying host CPU definition from capabilities XML into domain XML. Since the CPU definition is copied just before starting a domain, exactly the same XML can be used on different hosts while still providing the best guest CPU each host supports. Neither match attribute nor any feature elements can be used in this mode. Specifying CPU model is not supported either, but model's fallback attribute may still be used. Libvirt does not model every aspect of each CPU so the guest CPU will not match the host CPU exactly. On the other hand, the ABI provided to the guest is reproducible. During migration, complete CPU model definition is transferred to the destination host so the migrated guest will see exactly the same CPU model even if the destination host contains more capable CPUs for the running instance of the guest; but shutting down and restarting the guest may present different hardware to the guest according to the capabilities of the new host. host-passthrough With this mode, the CPU visible to the guest should be exactly the same as the host CPU even in the aspects that libvirt does not understand. Though the downside of this mode is that the guest environment cannot be reproduced on different hardware. Thus, if you hit any bugs, you are on your own. Neither model nor feature elements are allowed in this mode. Y. understood that if you want
Re: [Engine-devel] host cpu feature
On 12/05/2012 04:59 PM, Laszlo Hornyak wrote: - Original Message - From: Dan Kenigsberg dan...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: Laszlo Hornyak lhorn...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 3:22:18 PM Subject: Re: [Engine-devel] host cpu feature On Wed, Dec 05, 2012 at 04:05:00PM +0200, Yaniv Kaul wrote: On 12/05/2012 03:55 PM, Laszlo Hornyak wrote: The nice thing about hostModel (unlike hostPassthrough) is that once we created the VM we can migrate it to stronger hosts, and back to the original host. I suppose that it complicates the scheduler. Yes with host-model you get the features that libvirt handles. In such cases the engine could decide, if you want this functionality. Well the scheduler architecture is just being reinvented. For the host-passthrough, I think the AllowMigrateCPUHost configuration option would be a simple decision for the administrator: set it to true if all hosts are uniform. If it is THAT simple, Engine could take this decision without human intervension. Is there a way engine can figure out if the cpu-models in all the hosts are the same? I mean even if some host flags are not handled by libvirt and therefore vdsm and engine... So I would really need that permission from the user. If it is not set to true, then we will not allow migration of such VMs. That's not what I understood from libvirt's documentation. I You may be right, could you send an URL to that point of the documentation or copy-paste? The link I followed from your feature page: http://libvirt.org/formatdomain.html#elementsCPU : host-model The host-model mode is essentially a shortcut to copying host CPU definition from capabilities XML into domain XML. Since the CPU definition is copied just before starting a domain, exactly the same XML can be used on different hosts while still providing the best guest CPU each host supports. Neither match attribute nor any feature elements can be used in this mode. Specifying CPU model is not supported either, but model's fallback attribute may still be used. Libvirt does not model every aspect of each CPU so the guest CPU will not match the host CPU exactly. On the other hand, the ABI provided to the guest is reproducible. During migration, complete CPU model definition is transferred to the destination host so the migrated guest will see exactly the same CPU model even if the destination host contains more capable CPUs for the running instance of the guest; but shutting down and restarting the guest may present different hardware to the guest according to the capabilities of the new host. host-passthrough With this mode, the CPU visible to the guest should be exactly the same as the host CPU even in the aspects that libvirt does not understand. Though the downside of this mode is that the guest environment cannot be reproduced on different hardware. Thus, if you hit any bugs, you are on your own. That's exactly where AllowMigrateCPUHost fits well: when a user ticks this for a cluster he says yeah, I like to be on my own. cpu mode=host-passthrough migration: I talked to the libvirt guys and they said it is OK if the hardware and the software are the same, and it will work, but they would not recommend. So if they do not recommend it, I would drop this from the feature spec. Anyone against it? Laszlo I'm a bit against it. I don't see why it's that complicated: Allow migration - use 'host-model' Do not allow - use 'host-passthrough'. The reason of why we need host-passthrough is that otherwise I suspect we depend on libvirt for newer features to be somehow exposed to the guest (not sure about it). Y. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] host cpu feature
- Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 5:24:59 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Yaniv Kaul yk...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: Dan Kenigsberg dan...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 4:10:13 PM Subject: Re: [Engine-devel] host cpu feature On 12/05/2012 04:59 PM, Laszlo Hornyak wrote: - Original Message - From: Dan Kenigsberg dan...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: Laszlo Hornyak lhorn...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 3:22:18 PM Subject: Re: [Engine-devel] host cpu feature On Wed, Dec 05, 2012 at 04:05:00PM +0200, Yaniv Kaul wrote: On 12/05/2012 03:55 PM, Laszlo Hornyak wrote: The nice thing about hostModel (unlike hostPassthrough) is that once we created the VM we can migrate it to stronger hosts, and back to the original host. I suppose that it complicates the scheduler. Yes with host-model you get the features that libvirt handles. In such cases the engine could decide, if you want this functionality. Well the scheduler architecture is just being reinvented. For the host-passthrough, I think the AllowMigrateCPUHost configuration option would be a simple decision for the administrator: set it to true if all hosts are uniform. If it is THAT simple, Engine could take this decision without human intervension. Is there a way engine can figure out if the cpu-models in all the hosts are the same? I mean even if some host flags are not handled by libvirt and therefore vdsm and engine... So I would really need that permission from the user. If it is not set to true, then we will not allow migration of such VMs. That's not what I understood from libvirt's documentation. I You may be right, could you send an URL to that point of the documentation or copy-paste? The link I followed from your feature page: http://libvirt.org/formatdomain.html#elementsCPU : host-model The host-model mode is essentially a shortcut to copying host CPU definition from capabilities XML into domain XML. Since the CPU definition is copied just before starting a domain, exactly the same XML can be used on different hosts while still providing the best guest CPU each host supports. Neither match attribute nor any feature elements can be used in this mode. Specifying CPU model is not supported either, but model's fallback attribute may still be used. Libvirt does not model every aspect of each CPU so the guest CPU will not match the host CPU exactly. On the other hand, the ABI provided to the guest is reproducible. During migration, complete CPU model definition is transferred to the destination host so the migrated guest will see exactly the same CPU model even if the destination host contains more capable CPUs for the running instance of the guest; but shutting down and restarting the guest may present different hardware to the guest according to the capabilities of the new host. host-passthrough With this mode, the CPU visible to the guest should be exactly the same as the host CPU even in the aspects that libvirt does not understand. Though the downside of this mode is that the guest environment cannot be reproduced on different hardware. Thus, if you hit any bugs, you are on your own. That's exactly where AllowMigrateCPUHost fits well: when a user ticks this for a cluster he says yeah, I like to be on my own. cpu mode=host-passthrough migration: I talked to the libvirt guys and they said it is OK if the hardware and the software are the same, and it will work, but they would not recommend. So if they do not recommend it, I would drop this from the feature spec. Anyone against it? Laszlo I'm a bit against it. I don't see why it's that complicated: Allow migration - use 'host-model' Do not allow - use 'host-passthrough'. So the actual cpu capabilities would depend not only on the 'host cpu' checkbox, but also on the 'pinned to host' checkbox. I think this logical trick would be both funny and confusing from the user's perspective. The reason of why we need host-passthrough is that otherwise I suspect we depend on libvirt for newer features to be somehow exposed to the guest (not sure about it). Yes, with other words: this is a tuning feature. Let's keep it simple. 1. Please remove AllowMigrateCPUHost. No reason for us to do what libvirt is asking to void. 2. host-passthrough should be available only for non migratable VMs. Y.
Re: [Engine-devel] host cpu feature
- Original Message - From: Doron Fediuck dfedi...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: engine-devel engine-devel@ovirt.org, Yaniv Kaul yk...@redhat.com Sent: Wednesday, December 5, 2012 5:33:35 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 5:24:59 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Yaniv Kaul yk...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: Dan Kenigsberg dan...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 4:10:13 PM Subject: Re: [Engine-devel] host cpu feature On 12/05/2012 04:59 PM, Laszlo Hornyak wrote: - Original Message - From: Dan Kenigsberg dan...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: Laszlo Hornyak lhorn...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 3:22:18 PM Subject: Re: [Engine-devel] host cpu feature On Wed, Dec 05, 2012 at 04:05:00PM +0200, Yaniv Kaul wrote: On 12/05/2012 03:55 PM, Laszlo Hornyak wrote: The nice thing about hostModel (unlike hostPassthrough) is that once we created the VM we can migrate it to stronger hosts, and back to the original host. I suppose that it complicates the scheduler. Yes with host-model you get the features that libvirt handles. In such cases the engine could decide, if you want this functionality. Well the scheduler architecture is just being reinvented. For the host-passthrough, I think the AllowMigrateCPUHost configuration option would be a simple decision for the administrator: set it to true if all hosts are uniform. If it is THAT simple, Engine could take this decision without human intervension. Is there a way engine can figure out if the cpu-models in all the hosts are the same? I mean even if some host flags are not handled by libvirt and therefore vdsm and engine... So I would really need that permission from the user. If it is not set to true, then we will not allow migration of such VMs. That's not what I understood from libvirt's documentation. I You may be right, could you send an URL to that point of the documentation or copy-paste? The link I followed from your feature page: http://libvirt.org/formatdomain.html#elementsCPU : host-model The host-model mode is essentially a shortcut to copying host CPU definition from capabilities XML into domain XML. Since the CPU definition is copied just before starting a domain, exactly the same XML can be used on different hosts while still providing the best guest CPU each host supports. Neither match attribute nor any feature elements can be used in this mode. Specifying CPU model is not supported either, but model's fallback attribute may still be used. Libvirt does not model every aspect of each CPU so the guest CPU will not match the host CPU exactly. On the other hand, the ABI provided to the guest is reproducible. During migration, complete CPU model definition is transferred to the destination host so the migrated guest will see exactly the same CPU model even if the destination host contains more capable CPUs for the running instance of the guest; but shutting down and restarting the guest may present different hardware to the guest according to the capabilities of the new host. host-passthrough With this mode, the CPU visible to the guest should be exactly the same as the host CPU even in the aspects that libvirt does not understand. Though the downside of this mode is that the guest environment cannot be reproduced on different hardware. Thus, if you hit any bugs, you are on your own. That's exactly where AllowMigrateCPUHost fits well: when a user ticks this for a cluster he says yeah, I like to be on my own. cpu mode=host-passthrough migration: I talked to the libvirt guys and they said it is OK if the hardware and the software are the same, and it will work, but they would not recommend. So if they do not recommend it, I would drop this from the feature spec. Anyone against it? Laszlo I'm a bit against it. I don't see why it's that complicated: Allow migration - use 'host-model' Do not allow - use 'host-passthrough'. So the actual cpu capabilities would depend not only on the 'host cpu' checkbox, but also on the 'pinned to host' checkbox. I think this logical trick would be both funny and confusing from the
Re: [Engine-devel] host cpu feature
- Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org, Yaniv Kaul yk...@redhat.com Sent: Wednesday, December 5, 2012 6:41:24 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: engine-devel engine-devel@ovirt.org, Yaniv Kaul yk...@redhat.com Sent: Wednesday, December 5, 2012 5:33:35 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 5:24:59 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Yaniv Kaul yk...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: Dan Kenigsberg dan...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 4:10:13 PM Subject: Re: [Engine-devel] host cpu feature On 12/05/2012 04:59 PM, Laszlo Hornyak wrote: - Original Message - From: Dan Kenigsberg dan...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: Laszlo Hornyak lhorn...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 3:22:18 PM Subject: Re: [Engine-devel] host cpu feature On Wed, Dec 05, 2012 at 04:05:00PM +0200, Yaniv Kaul wrote: On 12/05/2012 03:55 PM, Laszlo Hornyak wrote: The nice thing about hostModel (unlike hostPassthrough) is that once we created the VM we can migrate it to stronger hosts, and back to the original host. I suppose that it complicates the scheduler. Yes with host-model you get the features that libvirt handles. In such cases the engine could decide, if you want this functionality. Well the scheduler architecture is just being reinvented. For the host-passthrough, I think the AllowMigrateCPUHost configuration option would be a simple decision for the administrator: set it to true if all hosts are uniform. If it is THAT simple, Engine could take this decision without human intervension. Is there a way engine can figure out if the cpu-models in all the hosts are the same? I mean even if some host flags are not handled by libvirt and therefore vdsm and engine... So I would really need that permission from the user. If it is not set to true, then we will not allow migration of such VMs. That's not what I understood from libvirt's documentation. I You may be right, could you send an URL to that point of the documentation or copy-paste? The link I followed from your feature page: http://libvirt.org/formatdomain.html#elementsCPU : host-model The host-model mode is essentially a shortcut to copying host CPU definition from capabilities XML into domain XML. Since the CPU definition is copied just before starting a domain, exactly the same XML can be used on different hosts while still providing the best guest CPU each host supports. Neither match attribute nor any feature elements can be used in this mode. Specifying CPU model is not supported either, but model's fallback attribute may still be used. Libvirt does not model every aspect of each CPU so the guest CPU will not match the host CPU exactly. On the other hand, the ABI provided to the guest is reproducible. During migration, complete CPU model definition is transferred to the destination host so the migrated guest will see exactly the same CPU model even if the destination host contains more capable CPUs for the running instance of the guest; but shutting down and restarting the guest may present different hardware to the guest according to the capabilities of the new host. host-passthrough With this mode, the CPU visible to the guest should be exactly the same as the host CPU even in the aspects that libvirt does not understand. Though the downside of this mode is that the guest environment cannot be reproduced on different hardware. Thus, if you hit any bugs, you are on your own. That's exactly where AllowMigrateCPUHost fits well: when a user ticks this for a cluster he says yeah, I like to be on my own. cpu mode=host-passthrough migration: I talked to the libvirt guys and they said it is OK if the hardware and the software are the same, and it will work, but they would not
Re: [Engine-devel] host cpu feature
On 12/05/2012 06:46 PM, Doron Fediuck wrote: - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org, Yaniv Kaul yk...@redhat.com Sent: Wednesday, December 5, 2012 6:41:24 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: engine-devel engine-devel@ovirt.org, Yaniv Kaul yk...@redhat.com Sent: Wednesday, December 5, 2012 5:33:35 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 5:24:59 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Yaniv Kaul yk...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: Dan Kenigsberg dan...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 4:10:13 PM Subject: Re: [Engine-devel] host cpu feature On 12/05/2012 04:59 PM, Laszlo Hornyak wrote: - Original Message - From: Dan Kenigsberg dan...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: Laszlo Hornyak lhorn...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 3:22:18 PM Subject: Re: [Engine-devel] host cpu feature On Wed, Dec 05, 2012 at 04:05:00PM +0200, Yaniv Kaul wrote: On 12/05/2012 03:55 PM, Laszlo Hornyak wrote: The nice thing about hostModel (unlike hostPassthrough) is that once we created the VM we can migrate it to stronger hosts, and back to the original host. I suppose that it complicates the scheduler. Yes with host-model you get the features that libvirt handles. In such cases the engine could decide, if you want this functionality. Well the scheduler architecture is just being reinvented. For the host-passthrough, I think the AllowMigrateCPUHost configuration option would be a simple decision for the administrator: set it to true if all hosts are uniform. If it is THAT simple, Engine could take this decision without human intervension. Is there a way engine can figure out if the cpu-models in all the hosts are the same? I mean even if some host flags are not handled by libvirt and therefore vdsm and engine... So I would really need that permission from the user. If it is not set to true, then we will not allow migration of such VMs. That's not what I understood from libvirt's documentation. I You may be right, could you send an URL to that point of the documentation or copy-paste? The link I followed from your feature page: http://libvirt.org/formatdomain.html#elementsCPU : host-model The host-model mode is essentially a shortcut to copying host CPU definition from capabilities XML into domain XML. Since the CPU definition is copied just before starting a domain, exactly the same XML can be used on different hosts while still providing the best guest CPU each host supports. Neither match attribute nor any feature elements can be used in this mode. Specifying CPU model is not supported either, but model's fallback attribute may still be used. Libvirt does not model every aspect of each CPU so the guest CPU will not match the host CPU exactly. On the other hand, the ABI provided to the guest is reproducible. During migration, complete CPU model definition is transferred to the destination host so the migrated guest will see exactly the same CPU model even if the destination host contains more capable CPUs for the running instance of the guest; but shutting down and restarting the guest may present different hardware to the guest according to the capabilities of the new host. host-passthrough With this mode, the CPU visible to the guest should be exactly the same as the host CPU even in the aspects that libvirt does not understand. Though the downside of this mode is that the guest environment cannot be reproduced on different hardware. Thus, if you hit any bugs, you are on your own. That's exactly where AllowMigrateCPUHost fits well: when a user ticks this for a cluster he says yeah, I like to be on my own. cpu mode=host-passthrough migration: I talked to the libvirt guys and they said it is OK if the hardware and the software are the same, and it will work, but they would not recommend. So if they do not recommend it, I would drop this from the feature spec. Anyone against it? Laszlo I'm a bit against it. I don't see why it's that complicated: Allow migration - use 'host-model' Do not allow - use 'host-passthrough'. So the actual cpu capabilities would depend not only on the 'host cpu' checkbox, but also on the 'pinned to host' checkbox. I think this logical trick would be both funny and confusing from the user's perspective. The reason of why we need host-passthrough is that otherwise I suspect we depend on libvirt for newer features to be somehow exposed to the guest (not sure about it). Yes, with
Re: [Engine-devel] host cpu feature
- Original Message - From: Yaniv Kaul yk...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: Laszlo Hornyak lhorn...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 6:48:33 PM Subject: Re: [Engine-devel] host cpu feature On 12/05/2012 06:46 PM, Doron Fediuck wrote: - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org, Yaniv Kaul yk...@redhat.com Sent: Wednesday, December 5, 2012 6:41:24 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: engine-devel engine-devel@ovirt.org, Yaniv Kaul yk...@redhat.com Sent: Wednesday, December 5, 2012 5:33:35 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 5:24:59 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Yaniv Kaul yk...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: Dan Kenigsberg dan...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 4:10:13 PM Subject: Re: [Engine-devel] host cpu feature On 12/05/2012 04:59 PM, Laszlo Hornyak wrote: - Original Message - From: Dan Kenigsberg dan...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: Laszlo Hornyak lhorn...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 3:22:18 PM Subject: Re: [Engine-devel] host cpu feature On Wed, Dec 05, 2012 at 04:05:00PM +0200, Yaniv Kaul wrote: On 12/05/2012 03:55 PM, Laszlo Hornyak wrote: The nice thing about hostModel (unlike hostPassthrough) is that once we created the VM we can migrate it to stronger hosts, and back to the original host. I suppose that it complicates the scheduler. Yes with host-model you get the features that libvirt handles. In such cases the engine could decide, if you want this functionality. Well the scheduler architecture is just being reinvented. For the host-passthrough, I think the AllowMigrateCPUHost configuration option would be a simple decision for the administrator: set it to true if all hosts are uniform. If it is THAT simple, Engine could take this decision without human intervension. Is there a way engine can figure out if the cpu-models in all the hosts are the same? I mean even if some host flags are not handled by libvirt and therefore vdsm and engine... So I would really need that permission from the user. If it is not set to true, then we will not allow migration of such VMs. That's not what I understood from libvirt's documentation. I You may be right, could you send an URL to that point of the documentation or copy-paste? The link I followed from your feature page: http://libvirt.org/formatdomain.html#elementsCPU : host-model The host-model mode is essentially a shortcut to copying host CPU definition from capabilities XML into domain XML. Since the CPU definition is copied just before starting a domain, exactly the same XML can be used on different hosts while still providing the best guest CPU each host supports. Neither match attribute nor any feature elements can be used in this mode. Specifying CPU model is not supported either, but model's fallback attribute may still be used. Libvirt does not model every aspect of each CPU so the guest CPU will not match the host CPU exactly. On the other hand, the ABI provided to the guest is reproducible. During migration, complete CPU model definition is transferred to the destination host so the migrated guest will see exactly the same CPU model even if the destination host contains more capable CPUs for the running instance of the guest; but shutting down and restarting the guest may present different hardware to the guest according to the capabilities of the new host. host-passthrough With this mode, the CPU visible to the guest should be exactly the same as the host CPU even in the aspects that libvirt does not understand. Though the downside of this mode is that the guest environment cannot be reproduced on different hardware. Thus, if you hit any bugs, you are on your own. That's exactly where AllowMigrateCPUHost fits well: when a user ticks this for a cluster he says yeah, I like to be on my own. cpu mode=host-passthrough migration: I talked to the libvirt guys and they said it is OK if the hardware and the software are the same, and it will work, but they would not recommend. So if they do not recommend it, I would drop this from the
Re: [Engine-devel] host cpu feature
On 12/05/2012 07:10 PM, Doron Fediuck wrote: - Original Message - From: Yaniv Kaul yk...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: Laszlo Hornyak lhorn...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 6:48:33 PM Subject: Re: [Engine-devel] host cpu feature On 12/05/2012 06:46 PM, Doron Fediuck wrote: - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org, Yaniv Kaul yk...@redhat.com Sent: Wednesday, December 5, 2012 6:41:24 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: engine-devel engine-devel@ovirt.org, Yaniv Kaul yk...@redhat.com Sent: Wednesday, December 5, 2012 5:33:35 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 5:24:59 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Yaniv Kaul yk...@redhat.com To: Laszlo Hornyak lhorn...@redhat.com Cc: Dan Kenigsberg dan...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 4:10:13 PM Subject: Re: [Engine-devel] host cpu feature On 12/05/2012 04:59 PM, Laszlo Hornyak wrote: - Original Message - From: Dan Kenigsberg dan...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: Laszlo Hornyak lhorn...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 3:22:18 PM Subject: Re: [Engine-devel] host cpu feature On Wed, Dec 05, 2012 at 04:05:00PM +0200, Yaniv Kaul wrote: On 12/05/2012 03:55 PM, Laszlo Hornyak wrote: The nice thing about hostModel (unlike hostPassthrough) is that once we created the VM we can migrate it to stronger hosts, and back to the original host. I suppose that it complicates the scheduler. Yes with host-model you get the features that libvirt handles. In such cases the engine could decide, if you want this functionality. Well the scheduler architecture is just being reinvented. For the host-passthrough, I think the AllowMigrateCPUHost configuration option would be a simple decision for the administrator: set it to true if all hosts are uniform. If it is THAT simple, Engine could take this decision without human intervension. Is there a way engine can figure out if the cpu-models in all the hosts are the same? I mean even if some host flags are not handled by libvirt and therefore vdsm and engine... So I would really need that permission from the user. If it is not set to true, then we will not allow migration of such VMs. That's not what I understood from libvirt's documentation. I You may be right, could you send an URL to that point of the documentation or copy-paste? The link I followed from your feature page: http://libvirt.org/formatdomain.html#elementsCPU : host-model The host-model mode is essentially a shortcut to copying host CPU definition from capabilities XML into domain XML. Since the CPU definition is copied just before starting a domain, exactly the same XML can be used on different hosts while still providing the best guest CPU each host supports. Neither match attribute nor any feature elements can be used in this mode. Specifying CPU model is not supported either, but model's fallback attribute may still be used. Libvirt does not model every aspect of each CPU so the guest CPU will not match the host CPU exactly. On the other hand, the ABI provided to the guest is reproducible. During migration, complete CPU model definition is transferred to the destination host so the migrated guest will see exactly the same CPU model even if the destination host contains more capable CPUs for the running instance of the guest; but shutting down and restarting the guest may present different hardware to the guest according to the capabilities of the new host. host-passthrough With this mode, the CPU visible to the guest should be exactly the same as the host CPU even in the aspects that libvirt does not understand. Though the downside of this mode is that the guest environment cannot be reproduced on different hardware. Thus, if you hit any bugs, you are on your own. That's exactly where AllowMigrateCPUHost fits well: when a user ticks this for a cluster he says yeah, I like to be on my own. cpu mode=host-passthrough migration: I talked to the libvirt guys and they said it is OK if the hardware and the software are the same, and it will work, but they would not recommend. So if they do not recommend it, I would drop this from the feature spec. Anyone against it? Laszlo I'm a bit against it. I don't see why it's that complicated: Allow migration - use 'host-model' Do not allow - use 'host-passthrough'. So the actual cpu capabilities would depend not only on the 'host cpu'
Re: [Engine-devel] host cpu feature
- Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 7:14:46 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: Laszlo Hornyak lhorn...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 6:10:55 PM Subject: Re: [Engine-devel] host cpu feature Alternative idea, inspired by Thus, if you hit any bugs, you are on your own (http://libvirt.org/formatdomain.html#elementsCPU wrt 'host-passthrough'): A config option to determine if we use host-model or host-passthrough. Y. I do not think the engine should go to this level. ie- it can ask for passthrough as a feature, and the actual implementation is handled by vdsm. If vdsm decides over host-passthrough or host-model, then how will the engine user know what exactly his VM gets. I think vdsm must be told exactly what to do. VDSM maintains some level of independence. This is why it the engine should be able to ask for passthrough as a feature. Otherwize vdsm will handle it. So for now I suggest we stick to passthrough only, and if we get an RFE for advanced mode we'll support the host model. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] host cpu feature
Quoting Doron Fediuck dfedi...@redhat.com: - Original Message - From: Laszlo Hornyak lhorn...@redhat.com To: Doron Fediuck dfedi...@redhat.com Cc: engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 7:14:46 PM Subject: Re: [Engine-devel] host cpu feature - Original Message - From: Doron Fediuck dfedi...@redhat.com To: Yaniv Kaul yk...@redhat.com Cc: Laszlo Hornyak lhorn...@redhat.com, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 6:10:55 PM Subject: Re: [Engine-devel] host cpu feature Alternative idea, inspired by Thus, if you hit any bugs, you are on your own (http://libvirt.org/formatdomain.html#elementsCPU wrt 'host-passthrough'): A config option to determine if we use host-model or host-passthrough. Y. I do not think the engine should go to this level. ie- it can ask for passthrough as a feature, and the actual implementation is handled by vdsm. If vdsm decides over host-passthrough or host-model, then how will the engine user know what exactly his VM gets. I think vdsm must be told exactly what to do. VDSM maintains some level of independence. This is why it the engine should be able to ask for passthrough as a feature. Otherwize vdsm will handle it. So for now I suggest we stick to passthrough only, and if we get an RFE for advanced mode we'll support the host model. What are we gaining by using passthrough over host-model? Looking at libvirt documentation, it seems that both modes give host CPU capabilities to guest VM. Whereas the downside of passthrough is that it limits migration. Whereas host-model will migrate to other hardware and if the destination hardware is better than source then the guest VM performance can be improved by rebooting guest. As a stretch goal, ovirt can keep track of host capabilities and inform the user after migrating to a better host, that a reboot may improve guest performance. Regards Sharad Mishra ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] [Users] RHEVM 3.1 DB Schema diagram
- Original Message - From: Eli Mesika emes...@redhat.com To: Allon Mureinik amure...@redhat.com Cc: users us...@ovirt.org, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 12:51:20 PM Subject: Re: [Users] RHEVM 3.1 DB Schema diagram - Original Message - From: Allon Mureinik amure...@redhat.com To: Eli Mesika emes...@redhat.com Cc: users us...@ovirt.org, engine-devel engine-devel@ovirt.org Sent: Wednesday, December 5, 2012 9:21:34 AM Subject: Re: [Users] RHEVM 3.1 DB Schema diagram Nice! How are the names of the blocks generated? Some names strike me a bit there (e.g., images is in the storage_domains_static block). Auto-generated ... (nice work IMHO) That I get `-) I was asking if you understand where the names of the blocks are taken from, because I couldn't. - Original Message - From: Eli Mesika emes...@redhat.com To: users us...@ovirt.org, engine-devel engine-devel@ovirt.org Sent: Tuesday, December 4, 2012 7:12:28 PM Subject: [Users] RHEVM 3.1 DB Schema diagram Hi There were some requests on the users list to get our DB schema diagram. After looking a while for a tool for generating that , I have found the DBSchema tool. Please find attached the RHEVM 3.1 DB Schema diagram. I intend to get it into the oVirt site but currently it's too large and does not fit well. Regards Eli Mesika ___ Users mailing list us...@ovirt.org http://lists.ovirt.org/mailman/listinfo/users ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
