Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Hi again, Divakar, sorry for the delayed response! On Wed, 2014-04-09 at 14:52 +, Nandavar, Divakar Padiyar wrote: Hi Jay, Managing multiple clusters using the Compute Proxy is not new right? Prior to this nova baremetal driver has used this model already. Yes, unfortunately. However, nova-baremetal has moved to the Ironic project, which has its own scheduler that is built to deal with the problem of having a single compute manager responsible for 1 set of compute resources. Also this Proxy Compute model gives flexibility to deploy as many computes required based on the requirement. For example, one can setup one proxy compute node to manage a set of clusters and another proxy compute to manage a separate set of clusters or launch compute node for each of the clusters. It actually reduces the flexibility of Nova in two main ways: 1) It breaks the horizontal scale-out model that exists when a single nova-compute is responsible for a wholly-separate group of compute resources. Under the proxy compute model, if the proxy compute goes down, the whole cluster goes down. 2) It surrenders control of the VMs to an external system and in doing so, must constantly poll this external system in order to know the state of the externally-controlled resources. We do this already at the hypervisor driver level. The source of truth for VM state is the hypervisor, and nova-compute must periodically query the hypervisor for live data about the VMs running on the hypervisor. The Nova scheduler queries nova-compute workers for information about the VMs running on the host it controls. For the proxy-compute model, the Nova scheduler queries proxy-computes, which then must query some externally-controlled management platform, and then aggregate this data back up to the Nova scheduler (a job that the Nova scheduler is designed to do, not the nova-compute worker). This additional layer of complexity does not allow greater flexibility -- it instead limits the flexibility of Nova, since it changes the definition and responsibility of two of its major components: nova-compute and nova-scheduler. Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
It's z/VM which you might got more information here :) http://www.vm.ibm.com/ Best Regards! Kevin (Chen) Ji 纪 晨 Engineer, zVM Development, CSTL Notes: Chen CH Ji/China/IBM@IBMCN Internet: jiche...@cn.ibm.com Phone: +86-10-82454158 Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC From: Matthew Booth mbo...@redhat.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, Date: 04/09/2014 07:02 PM Subject:Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute On 09/04/14 07:07, Chen CH Ji wrote: we used to have one compute service corresponding to multiple hypervisors (like host and nodes concept ) our major issue on our platform is we can't run nova-compute service on the hypervisor and we need to find another place to run the nova-compute in order to talk to hypervisor management API through REST API It may not be directly relevant to this discussion, but I'm interested to know what constraint prevents you running nova-compute on the hypervisor. Matt -- Matthew Booth, RHCA, RHCSS Red Hat Engineering, Virtualisation Team GPG ID: D33C3490 GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev inline: graycol.gif___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
we used to have one compute service corresponding to multiple hypervisors (like host and nodes concept ) our major issue on our platform is we can't run nova-compute service on the hypervisor and we need to find another place to run the nova-compute in order to talk to hypervisor management API through REST API which means we have to run multiple compute service out side of our hypervisors and it's hard for us to control the compute services at that time, but we have no choice since nova migration only can be migrated to another host instead of node ,so we implement according to it if we can support host + node, then it might be helpful for the hypervisors with different arch The point is whether we are able to expose the internal (say, not only the host concept but also the node concept ) to outside guess live-migration is admin only feature, can we expose those node concept to admin and let admin decide it? Best Regards! Kevin (Chen) Ji 纪 晨 Engineer, zVM Development, CSTL Notes: Chen CH Ji/China/IBM@IBMCN Internet: jiche...@cn.ibm.com Phone: +86-10-82454158 Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC From: Jay Lau jay.lau@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, Date: 04/06/2014 07:02 PM Subject:Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute Hi Divakar, Can I say that the bare metal provisioning is now using kind of Parent - Child compute mode? I was also thinking that we can use host:node to identify a kind of Parent-Child or Hierarchy Compute. So can you please show some difference for your Parent - Child Compute Node and bare metal provisioning? Thanks! 2014-04-06 14:59 GMT+08:00 Nandavar, Divakar Padiyar divakar.padiyar-nanda...@hp.com: Well, it seems to me that the problem is the above blueprint and the code it introduced. This is an anti-feature IMO, and probably the best solution would be to remove the above code and go back to having a single nova-compute managing a single vCenter cluster, not multiple ones. Problem is not introduced by managing multiple clusters from single nova-compute proxy node. Internally this proxy driver is still presenting the compute-node for each of the cluster its managing. What we need to think about is applicability of the live migration use case when a cluster is modelled as a compute. Since the cluster is modelled as a compute, it is assumed that a typical use case of live-move is taken care by the underlying cluster itself. With this there are other use cases which are no-op today like host maintenance mode, live move, setting instance affinity etc., In order to resolve this I was thinking of A way to expose operations on individual ESX Hosts like Putting host in maintenance mode, live move, instance affinity etc., by introducing Parent - Child compute node concept. Scheduling can be restricted to Parent compute node and Child compute node can be used for providing more drill down on compute and also enable additional compute operations. Any thoughts on this? Thanks, Divakar -Original Message- From: Jay Pipes [mailto:jaypi...@gmail.com] Sent: Sunday, April 06, 2014 2:02 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute Importance: High On Fri, 2014-04-04 at 13:30 +0800, Jay Lau wrote: 2014-04-04 12:46 GMT+08:00 Jay Pipes jaypi...@gmail.com: On Fri, 2014-04-04 at 11:08 +0800, Jay Lau wrote: Thanks Jay and Chris for the comments! @Jay Pipes, I think that we still need to enable one nova compute live migration as one nova compute can manage multiple clusters and VMs can be migrated between those clusters managed by one nova compute. Why, though? That is what I am asking... seems to me like this is an anti-feature. What benefit does the user get from moving an instance from one VCenter cluster to another VCenter cluster if the two clusters are on the same physical machine? @Jay Pipes, for VMWare, one physical machine (ESX server) can only belong to one VCenter cluster, so we may have following scenarios. DC | |---Cluster1 | | | |---host1 | |---Cluser2 | |---host2 Then when using VCDriver, I can use one nova compute manage both Cluster1 and Cluster2, this caused me cannot migrate VM from host2 to host1 ;-( The bp was introduced by https://blueprints.launchpad.net/nova/+spec/multiple-clusters-managed- by-one-service Well, it seems to me
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Hi Kevin, Thanks for the contribution. Shawn from VMware already filed a bp to export those resources https://blueprints.launchpad.net/nova/+spec/vmware-auto-inventory , but this bp might some redesign as we need to decide how we will handle configuration and convention when it comes to vSphere inventory. Yes, I think that we need support host-node mode for live migration. Divakar give some idea of Parent - Child compute mode, I'm just wondering what is the difference of this mode with bare-metal host-node mode. 2014-04-09 14:07 GMT+08:00 Chen CH Ji jiche...@cn.ibm.com: we used to have one compute service corresponding to multiple hypervisors (like host and nodes concept ) our major issue on our platform is we can't run nova-compute service on the hypervisor and we need to find another place to run the nova-compute in order to talk to hypervisor management API through REST API which means we have to run multiple compute service out side of our hypervisors and it's hard for us to control the compute services at that time, but we have no choice since nova migration only can be migrated to another host instead of node ,so we implement according to it if we can support host + node, then it might be helpful for the hypervisors with different arch The point is whether we are able to expose the internal (say, not only the host concept but also the node concept ) to outside guess live-migration is admin only feature, can we expose those node concept to admin and let admin decide it? Best Regards! Kevin (Chen) Ji 纪 晨 Engineer, zVM Development, CSTL Notes: Chen CH Ji/China/IBM@IBMCN Internet: jiche...@cn.ibm.com Phone: +86-10-82454158 Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC [image: Inactive hide details for Jay Lau ---04/06/2014 07:02:15 PM---Hi Divakar, Can I say that the bare metal provisioning is now usi]Jay Lau ---04/06/2014 07:02:15 PM---Hi Divakar, Can I say that the bare metal provisioning is now using kind of Parent - From: Jay Lau jay.lau@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, Date: 04/06/2014 07:02 PM Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute -- Hi Divakar, Can I say that the bare metal provisioning is now using kind of Parent - Child compute mode? I was also thinking that we can use host:node to identify a kind of Parent-Child or Hierarchy Compute. So can you please show some difference for your Parent - Child Compute Node and bare metal provisioning? Thanks! 2014-04-06 14:59 GMT+08:00 Nandavar, Divakar Padiyar *divakar.padiyar-nanda...@hp.com* divakar.padiyar-nanda...@hp.com: Well, it seems to me that the problem is the above blueprint and the code it introduced. This is an anti-feature IMO, and probably the best solution would be to remove the above code and go back to having a single nova-compute managing a single vCenter cluster, not multiple ones. Problem is not introduced by managing multiple clusters from single nova-compute proxy node. Internally this proxy driver is still presenting the compute-node for each of the cluster its managing.What we need to think about is applicability of the live migration use case when a cluster is modelled as a compute. Since the cluster is modelled as a compute, it is assumed that a typical use case of live-move is taken care by the underlying cluster itself. With this there are other use cases which are no-op today like host maintenance mode, live move, setting instance affinity etc., In order to resolve this I was thinking of A way to expose operations on individual ESX Hosts like Putting host in maintenance mode, live move, instance affinity etc., by introducing Parent - Child compute node concept. Scheduling can be restricted to Parent compute node and Child compute node can be used for providing more drill down on compute and also enable additional compute operations. Any thoughts on this? Thanks, Divakar -Original Message- From: Jay Pipes [mailto:*jaypi...@gmail.com* jaypi...@gmail.com] Sent: Sunday, April 06, 2014 2:02 AM To: *openstack-dev@lists.openstack.org*openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute Importance: High On Fri, 2014-04-04 at 13:30 +0800, Jay Lau wrote: 2014-04-04 12:46 GMT+08:00 Jay Pipes *jaypi...@gmail.com*jaypi...@gmail.com : On Fri, 2014-04-04 at 11:08 +0800, Jay Lau wrote: Thanks Jay and Chris for the comments! @Jay Pipes, I think that we still need to enable one nova compute live migration as one nova
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
2014-04-09 19:04 GMT+08:00 Matthew Booth mbo...@redhat.com: On 09/04/14 07:07, Chen CH Ji wrote: we used to have one compute service corresponding to multiple hypervisors (like host and nodes concept ) our major issue on our platform is we can't run nova-compute service on the hypervisor and we need to find another place to run the nova-compute in order to talk to hypervisor management API through REST API It may not be directly relevant to this discussion, but I'm interested to know what constraint prevents you running nova-compute on the hypervisor. Actually, VMWare has two drivers, one is ESXDriver and the other is VCDriver. When using ESXDrvier, one nova compute can only manage one ESX host, but ESXDriver do not support some advanced features such as live migration, resize etc. And this driver has been deprecated. We are now talking about VCDriver which will talk to vCenter via wsdl API and the VCDriver is intend to support all VM operations, but we need some enhancement to make VCDriver can work well for some advacend features such as live migration. Matt -- Matthew Booth, RHCA, RHCSS Red Hat Engineering, Virtualisation Team GPG ID: D33C3490 GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Thanks, Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Hi Juan, thanks for your response. Comments inline. On Mon, 2014-04-07 at 10:22 +0200, Juan Manuel Rey wrote: Hi, I'm fairly new to this list, actually this is my first email sent, and to OpenStack in general, but I'm not new at all to VMware so I'll try to give you my point of view about possible use case here. Jay you are saying that by using Nova to manage ESXi hosts we don't need vCenter because they basically overlap in their capabilities. Actually, no, this is not my main point. My main point is that Nova should not change its architecture to fit the needs of one particular host management platform (vCenter). Nova should, as much as possible, communicate with vCenter to perform some operations -- in the same way that Nova communicates with KVM or XenServer to perform some operations. But Nova should not be re-architected (and I believe that is what has gone on here with the code change to have one nova-compute worker talking to multiple vCenter clusters) just so that one particular host management scheduler/platform (vCenter) can have all of its features exposed to Nova. I agree with you to some extent, Nova may have similar capabilities as vCenter Server but as you know OpenStack as a full cloud solution adds a lot more features that vCenter lacks, like multitenancy just to name one. Sure, however, my point is that Nova shouldn't need to be re-architected just to adhere to one particular host management platform's concepts of an atomic provider of compute resources. Also in any vSphere environment managing ESXi hosts individually, this is without vCenter, is completely out of the question. vCenter is the enabler of many vSphere features. And precisely that's is, IMHO, the use case of using Nova to manage vCenter to manage vSphere. Without vCenter we only have a bunch of hypervisors and none of the HA or DRS (dynamic resource balancing) capabilities that a vSphere cluster provides, this in my experience with vSphere users/customers is a no go scenario. Understood. Still doesn't change my opinion though :) Best, -jay I don't know why the decision to manage vCenter with Nova was made but based on the above I understand the reasoning. Best, --- Juan Manuel Rey @jreypo On Mon, Apr 7, 2014 at 7:20 AM, Jay Pipes jaypi...@gmail.com wrote: On Sun, 2014-04-06 at 06:59 +, Nandavar, Divakar Padiyar wrote: Well, it seems to me that the problem is the above blueprint and the code it introduced. This is an anti-feature IMO, and probably the best solution would be to remove the above code and go back to having a single nova-compute managing a single vCenter cluster, not multiple ones. Problem is not introduced by managing multiple clusters from single nova-compute proxy node. I strongly disagree. Internally this proxy driver is still presenting the compute-node for each of the cluster its managing. In what way? What we need to think about is applicability of the live migration use case when a cluster is modelled as a compute. Since the cluster is modelled as a compute, it is assumed that a typical use case of live-move is taken care by the underlying cluster itself. With this there are other use cases which are no-op today like host maintenance mode, live move, setting instance affinity etc., In order to resolve this I was thinking of A way to expose operations on individual ESX Hosts like Putting host in maintenance mode, live move, instance affinity etc., by introducing Parent - Child compute node concept. Scheduling can be restricted to Parent compute node and Child compute node can be used for providing more drill down on compute and also enable additional compute operations.Any thoughts on this? The fundamental problem is that hacks were put in place in order to make Nova defer control to vCenter, when the design of Nova and vCenter are not compatible, and we're paying the price for that right now. All of the operations you describe above -- putting a host in maintenance mode, live-migration of an instance, ensuring a new instance is launched near or not-near another instance -- depend on a fundamental design feature in Nova: that a nova-compute worker fully controls and manages a host that provides a place to put server instances. We have internal driver interfaces for the *hypervisor*, not for the *manager of hypervisors*, because, you know, that's what Nova does. The problem with all of the vCenter stuff is that
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
On Mon, 2014-04-07 at 15:47 +0100, Matthew Booth wrote: On 07/04/14 06:20, Jay Pipes wrote: On Sun, 2014-04-06 at 06:59 +, Nandavar, Divakar Padiyar wrote: Well, it seems to me that the problem is the above blueprint and the code it introduced. This is an anti-feature IMO, and probably the best solution would be to remove the above code and go back to having a single nova-compute managing a single vCenter cluster, not multiple ones. Problem is not introduced by managing multiple clusters from single nova-compute proxy node. I strongly disagree. Internally this proxy driver is still presenting the compute-node for each of the cluster its managing. In what way? What we need to think about is applicability of the live migration use case when a cluster is modelled as a compute. Since the cluster is modelled as a compute, it is assumed that a typical use case of live-move is taken care by the underlying cluster itself. With this there are other use cases which are no-op today like host maintenance mode, live move, setting instance affinity etc., In order to resolve this I was thinking of A way to expose operations on individual ESX Hosts like Putting host in maintenance mode, live move, instance affinity etc., by introducing Parent - Child compute node concept. Scheduling can be restricted to Parent compute node and Child compute node can be used for providing more drill down on compute and also enable additional compute operations. Any thoughts on this? The fundamental problem is that hacks were put in place in order to make Nova defer control to vCenter, when the design of Nova and vCenter are not compatible, and we're paying the price for that right now. All of the operations you describe above -- putting a host in maintenance mode, live-migration of an instance, ensuring a new instance is launched near or not-near another instance -- depend on a fundamental design feature in Nova: that a nova-compute worker fully controls and manages a host that provides a place to put server instances. We have internal driver interfaces for the *hypervisor*, not for the *manager of hypervisors*, because, you know, that's what Nova does. I'm going to take you to task here for use of the word 'fundamental'. What does Nova do? Apparently: 'OpenStack Nova provides a cloud computing fabric controller, supporting a wide variety of virtualization technologies, including KVM, Xen, LXC, VMware, and more. In addition to its native API, it includes compatibility with the commonly encountered Amazon EC2 and S3 APIs.' There's nothing in there about the ratio of Nova instances to hypervisors: that's an implementation detail. Now this change may or may not sit well with design decisions which have been made in the past, but the concept of managing multiple clusters from a single Nova instance is certainly not fundamentally wrong. It may not be pragmatic; it may require further changes to Nova which were not made, but there is nothing about it which is fundamentally at odds with the stated goals of the project. Why did I bother with that? I think it's in danger of being lost. Nova has been around for a while now and it has a lot of code and a lot of developers behind it. We need to remember, though, that's it's all for nothing if nobody wants to use it. VMware is different, but not wrong. Let's stay fresh. Please see my previous email to Juan about this. I'm not anti-VMWare. I'm just opposed to changing an important part of the implementation of Nova just so that certain vCenter operations can be supported. The problem with all of the vCenter stuff is that it is trying to say to Nova don't worry, I got this but unfortunately, Nova wants and needs to manage these things, not surrender control to a different system that handles orchestration and scheduling in its own unique way. Again, I'll flip that round. Nova *currently* manages these things, and working efficiently with a platform which also does these things would require rethinking some design above the driver level. It's not something we want to do naively, which the VMware driver is suffering from in this area. It may take time to get this right, but we shouldn't write it off as fundamentally wrong. It's useful to users and not fundamentally at odds with the project's goals. I'm not writing off vCenter or its capabilities. I am arguing that the bar for modifying a fundamental design decision in Nova -- that of being horizontally scalable by having a single nova-compute worker responsible for managing a single provider of compute resources -- was WAY too low, and that this decision should be revisited in the future (and possibly as part of the vmware driver refactoring efforts currently underway by the good folks at RH and VMWare). If a shop really wants to use vCenter for scheduling and orchestration of
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Hi Jay, Managing multiple clusters using the Compute Proxy is not new right? Prior to this nova baremetal driver has used this model already. Also this Proxy Compute model gives flexibility to deploy as many computes required based on the requirement. For example, one can setup one proxy compute node to manage a set of clusters and another proxy compute to manage a separate set of clusters or launch compute node for each of the clusters. Thanks, Divakar -Original Message- From: Jay Pipes [mailto:jaypi...@gmail.com] Sent: Wednesday, April 09, 2014 6:23 PM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute Importance: High Hi Juan, thanks for your response. Comments inline. On Mon, 2014-04-07 at 10:22 +0200, Juan Manuel Rey wrote: Hi, I'm fairly new to this list, actually this is my first email sent, and to OpenStack in general, but I'm not new at all to VMware so I'll try to give you my point of view about possible use case here. Jay you are saying that by using Nova to manage ESXi hosts we don't need vCenter because they basically overlap in their capabilities. Actually, no, this is not my main point. My main point is that Nova should not change its architecture to fit the needs of one particular host management platform (vCenter). Nova should, as much as possible, communicate with vCenter to perform some operations -- in the same way that Nova communicates with KVM or XenServer to perform some operations. But Nova should not be re-architected (and I believe that is what has gone on here with the code change to have one nova-compute worker talking to multiple vCenter clusters) just so that one particular host management scheduler/platform (vCenter) can have all of its features exposed to Nova. I agree with you to some extent, Nova may have similar capabilities as vCenter Server but as you know OpenStack as a full cloud solution adds a lot more features that vCenter lacks, like multitenancy just to name one. Sure, however, my point is that Nova shouldn't need to be re-architected just to adhere to one particular host management platform's concepts of an atomic provider of compute resources. Also in any vSphere environment managing ESXi hosts individually, this is without vCenter, is completely out of the question. vCenter is the enabler of many vSphere features. And precisely that's is, IMHO, the use case of using Nova to manage vCenter to manage vSphere. Without vCenter we only have a bunch of hypervisors and none of the HA or DRS (dynamic resource balancing) capabilities that a vSphere cluster provides, this in my experience with vSphere users/customers is a no go scenario. Understood. Still doesn't change my opinion though :) Best, -jay I don't know why the decision to manage vCenter with Nova was made but based on the above I understand the reasoning. Best, --- Juan Manuel Rey @jreypo On Mon, Apr 7, 2014 at 7:20 AM, Jay Pipes jaypi...@gmail.com wrote: On Sun, 2014-04-06 at 06:59 +, Nandavar, Divakar Padiyar wrote: Well, it seems to me that the problem is the above blueprint and the code it introduced. This is an anti-feature IMO, and probably the best solution would be to remove the above code and go back to having a single nova-compute managing a single vCenter cluster, not multiple ones. Problem is not introduced by managing multiple clusters from single nova-compute proxy node. I strongly disagree. Internally this proxy driver is still presenting the compute-node for each of the cluster its managing. In what way? What we need to think about is applicability of the live migration use case when a cluster is modelled as a compute. Since the cluster is modelled as a compute, it is assumed that a typical use case of live-move is taken care by the underlying cluster itself. With this there are other use cases which are no-op today like host maintenance mode, live move, setting instance affinity etc., In order to resolve this I was thinking of A way to expose operations on individual ESX Hosts like Putting host in maintenance mode, live move, instance affinity etc., by introducing Parent - Child compute node concept. Scheduling can be restricted to Parent compute node and Child compute node can be used for providing more drill down on compute and also enable additional compute operations.Any thoughts on this? The fundamental problem is that hacks were put in place in order to make Nova defer control to vCenter, when the design of Nova
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
- Original Message - I'm not writing off vCenter or its capabilities. I am arguing that the bar for modifying a fundamental design decision in Nova -- that of being horizontally scalable by having a single nova-compute worker responsible for managing a single provider of compute resources -- was WAY too low, and that this decision should be revisited in the future (and possibly as part of the vmware driver refactoring efforts currently underway by the good folks at RH and VMWare). +1, This is my main concern about having more than one ESX cluster under a single nova-compute agent as well. Currently it works, but it doesn't seem particularly advisable as on face value as such an architecture seems to break a number of the Nova design guidelines around high availability and fault tolerance. To me it seems like such an architecture effectively elevates nova-compute into being part of the control plane where it needs to have high availability (when discussing on IRC yesterday it seemed like this *may* be possible today but more testing is required to shake out any bugs). Now may well be the right approach *is* to make some changes to these expectations about Nova, but I think it's disingenuous to suggest that what is being suggested here isn't a significant re-architecting to resolve issues resulting from earlier hacks that allowed this functionality to work in the first place. Should be an interesting summit session. -Steve ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
@Divakar, yes, the Proxy Compute model is not new, but I'm not sure if this model can be accepted by community to manage both VM and PM. Anyway, I will try to file a bp and get more comments then. Thanks. 2014-04-09 22:52 GMT+08:00 Nandavar, Divakar Padiyar divakar.padiyar-nanda...@hp.com: Hi Jay, Managing multiple clusters using the Compute Proxy is not new right? Prior to this nova baremetal driver has used this model already. Also this Proxy Compute model gives flexibility to deploy as many computes required based on the requirement. For example, one can setup one proxy compute node to manage a set of clusters and another proxy compute to manage a separate set of clusters or launch compute node for each of the clusters. Thanks, Divakar -Original Message- From: Jay Pipes [mailto:jaypi...@gmail.com] Sent: Wednesday, April 09, 2014 6:23 PM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute Importance: High Hi Juan, thanks for your response. Comments inline. On Mon, 2014-04-07 at 10:22 +0200, Juan Manuel Rey wrote: Hi, I'm fairly new to this list, actually this is my first email sent, and to OpenStack in general, but I'm not new at all to VMware so I'll try to give you my point of view about possible use case here. Jay you are saying that by using Nova to manage ESXi hosts we don't need vCenter because they basically overlap in their capabilities. Actually, no, this is not my main point. My main point is that Nova should not change its architecture to fit the needs of one particular host management platform (vCenter). Nova should, as much as possible, communicate with vCenter to perform some operations -- in the same way that Nova communicates with KVM or XenServer to perform some operations. But Nova should not be re-architected (and I believe that is what has gone on here with the code change to have one nova-compute worker talking to multiple vCenter clusters) just so that one particular host management scheduler/platform (vCenter) can have all of its features exposed to Nova. I agree with you to some extent, Nova may have similar capabilities as vCenter Server but as you know OpenStack as a full cloud solution adds a lot more features that vCenter lacks, like multitenancy just to name one. Sure, however, my point is that Nova shouldn't need to be re-architected just to adhere to one particular host management platform's concepts of an atomic provider of compute resources. Also in any vSphere environment managing ESXi hosts individually, this is without vCenter, is completely out of the question. vCenter is the enabler of many vSphere features. And precisely that's is, IMHO, the use case of using Nova to manage vCenter to manage vSphere. Without vCenter we only have a bunch of hypervisors and none of the HA or DRS (dynamic resource balancing) capabilities that a vSphere cluster provides, this in my experience with vSphere users/customers is a no go scenario. Understood. Still doesn't change my opinion though :) Best, -jay I don't know why the decision to manage vCenter with Nova was made but based on the above I understand the reasoning. Best, --- Juan Manuel Rey @jreypo On Mon, Apr 7, 2014 at 7:20 AM, Jay Pipes jaypi...@gmail.com wrote: On Sun, 2014-04-06 at 06:59 +, Nandavar, Divakar Padiyar wrote: Well, it seems to me that the problem is the above blueprint and the code it introduced. This is an anti-feature IMO, and probably the best solution would be to remove the above code and go back to having a single nova-compute managing a single vCenter cluster, not multiple ones. Problem is not introduced by managing multiple clusters from single nova-compute proxy node. I strongly disagree. Internally this proxy driver is still presenting the compute-node for each of the cluster its managing. In what way? What we need to think about is applicability of the live migration use case when a cluster is modelled as a compute. Since the cluster is modelled as a compute, it is assumed that a typical use case of live-move is taken care by the underlying cluster itself. With this there are other use cases which are no-op today like host maintenance mode, live move, setting instance affinity etc., In order to resolve this I was thinking of A way to expose operations on individual ESX Hosts like Putting host in maintenance mode, live move, instance affinity etc., by introducing Parent - Child compute node concept. Scheduling can be restricted to Parent compute node and Child compute node can be used
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Hi, I'm fairly new to this list, actually this is my first email sent, and to OpenStack in general, but I'm not new at all to VMware so I'll try to give you my point of view about possible use case here. Jay you are saying that by using Nova to manage ESXi hosts we don't need vCenter because they basically overlap in their capabilities. I agree with you to some extent, Nova may have similar capabilities as vCenter Server but as you know OpenStack as a full cloud solution adds a lot more features that vCenter lacks, like multitenancy just to name one. Also in any vSphere environment managing ESXi hosts individually, this is without vCenter, is completely out of the question. vCenter is the enabler of many vSphere features. And precisely that's is, IMHO, the use case of using Nova to manage vCenter to manage vSphere. Without vCenter we only have a bunch of hypervisors and none of the HA or DRS (dynamic resource balancing) capabilities that a vSphere cluster provides, this in my experience with vSphere users/customers is a no go scenario. I don't know why the decision to manage vCenter with Nova was made but based on the above I understand the reasoning. Best, --- Juan Manuel Rey @jreypo On Mon, Apr 7, 2014 at 7:20 AM, Jay Pipes jaypi...@gmail.com wrote: On Sun, 2014-04-06 at 06:59 +, Nandavar, Divakar Padiyar wrote: Well, it seems to me that the problem is the above blueprint and the code it introduced. This is an anti-feature IMO, and probably the best solution would be to remove the above code and go back to having a single nova-compute managing a single vCenter cluster, not multiple ones. Problem is not introduced by managing multiple clusters from single nova-compute proxy node. I strongly disagree. Internally this proxy driver is still presenting the compute-node for each of the cluster its managing. In what way? What we need to think about is applicability of the live migration use case when a cluster is modelled as a compute. Since the cluster is modelled as a compute, it is assumed that a typical use case of live-move is taken care by the underlying cluster itself. With this there are other use cases which are no-op today like host maintenance mode, live move, setting instance affinity etc., In order to resolve this I was thinking of A way to expose operations on individual ESX Hosts like Putting host in maintenance mode, live move, instance affinity etc., by introducing Parent - Child compute node concept. Scheduling can be restricted to Parent compute node and Child compute node can be used for providing more drill down on compute and also enable additional compute operations.Any thoughts on this? The fundamental problem is that hacks were put in place in order to make Nova defer control to vCenter, when the design of Nova and vCenter are not compatible, and we're paying the price for that right now. All of the operations you describe above -- putting a host in maintenance mode, live-migration of an instance, ensuring a new instance is launched near or not-near another instance -- depend on a fundamental design feature in Nova: that a nova-compute worker fully controls and manages a host that provides a place to put server instances. We have internal driver interfaces for the *hypervisor*, not for the *manager of hypervisors*, because, you know, that's what Nova does. The problem with all of the vCenter stuff is that it is trying to say to Nova don't worry, I got this but unfortunately, Nova wants and needs to manage these things, not surrender control to a different system that handles orchestration and scheduling in its own unique way. If a shop really wants to use vCenter for scheduling and orchestration of server instances, what exactly is the point of using OpenStack Nova to begin with? What exactly is the point of trying to use OpenStack Nova for scheduling and host operations when you've already shelled out US $6,000 for vCenter Server and a boatload more money for ESX licensing? Sorry, I'm just at a loss why Nova was changed to accomodate vCenter cluster and management concepts to begin with. I just don't understand the use case here. Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
On 07/04/14 06:20, Jay Pipes wrote: On Sun, 2014-04-06 at 06:59 +, Nandavar, Divakar Padiyar wrote: Well, it seems to me that the problem is the above blueprint and the code it introduced. This is an anti-feature IMO, and probably the best solution would be to remove the above code and go back to having a single nova-compute managing a single vCenter cluster, not multiple ones. Problem is not introduced by managing multiple clusters from single nova-compute proxy node. I strongly disagree. Internally this proxy driver is still presenting the compute-node for each of the cluster its managing. In what way? What we need to think about is applicability of the live migration use case when a cluster is modelled as a compute. Since the cluster is modelled as a compute, it is assumed that a typical use case of live-move is taken care by the underlying cluster itself. With this there are other use cases which are no-op today like host maintenance mode, live move, setting instance affinity etc., In order to resolve this I was thinking of A way to expose operations on individual ESX Hosts like Putting host in maintenance mode, live move, instance affinity etc., by introducing Parent - Child compute node concept. Scheduling can be restricted to Parent compute node and Child compute node can be used for providing more drill down on compute and also enable additional compute operations.Any thoughts on this? The fundamental problem is that hacks were put in place in order to make Nova defer control to vCenter, when the design of Nova and vCenter are not compatible, and we're paying the price for that right now. All of the operations you describe above -- putting a host in maintenance mode, live-migration of an instance, ensuring a new instance is launched near or not-near another instance -- depend on a fundamental design feature in Nova: that a nova-compute worker fully controls and manages a host that provides a place to put server instances. We have internal driver interfaces for the *hypervisor*, not for the *manager of hypervisors*, because, you know, that's what Nova does. I'm going to take you to task here for use of the word 'fundamental'. What does Nova do? Apparently: 'OpenStack Nova provides a cloud computing fabric controller, supporting a wide variety of virtualization technologies, including KVM, Xen, LXC, VMware, and more. In addition to its native API, it includes compatibility with the commonly encountered Amazon EC2 and S3 APIs.' There's nothing in there about the ratio of Nova instances to hypervisors: that's an implementation detail. Now this change may or may not sit well with design decisions which have been made in the past, but the concept of managing multiple clusters from a single Nova instance is certainly not fundamentally wrong. It may not be pragmatic; it may require further changes to Nova which were not made, but there is nothing about it which is fundamentally at odds with the stated goals of the project. Why did I bother with that? I think it's in danger of being lost. Nova has been around for a while now and it has a lot of code and a lot of developers behind it. We need to remember, though, that's it's all for nothing if nobody wants to use it. VMware is different, but not wrong. Let's stay fresh. The problem with all of the vCenter stuff is that it is trying to say to Nova don't worry, I got this but unfortunately, Nova wants and needs to manage these things, not surrender control to a different system that handles orchestration and scheduling in its own unique way. Again, I'll flip that round. Nova *currently* manages these things, and working efficiently with a platform which also does these things would require rethinking some design above the driver level. It's not something we want to do naively, which the VMware driver is suffering from in this area. It may take time to get this right, but we shouldn't write it off as fundamentally wrong. It's useful to users and not fundamentally at odds with the project's goals. If a shop really wants to use vCenter for scheduling and orchestration of server instances, what exactly is the point of using OpenStack Nova to begin with? What exactly is the point of trying to use OpenStack Nova for scheduling and host operations when you've already shelled out US $6,000 for vCenter Server and a boatload more money for ESX licensing? I confess I wondered this myself. However, I have now spoken to real people who are spending real money doing exactly this. The drivers seem to be: * The external API * A heterogeneous cloud vSphere isn't really designed for the former and doesn't do it well. It obviously doesn't help with the latter at all. For example, users want to be able to give non-admin customers the ability to deploy across both KVM and VMware. To my mind, a VMware cluster is an obvious deployment target. I think it's reasonable for Nova
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
@Matt Booth: I think you make a lot of good points, but I think the main gist of the opposing argument, so to speak, is that currently, the way we differentiate between potential compute resources (whether they be an individual hypervisor or a cluster) is by having each have its own compute node. I think some of the reluctance here is to change that model -- the idea that a Nova compute node represents one resource which is, for all intents and purposes, atomic to OpenStack. While I get your point that this is an implementation detail, I think it's a rather large one, and a fundamental assumption in current OpenStack code (for the most part). If we change that assumption, we shouldn't really change it piecemeal. IMHO, this model (compute nodes as atomic resources) fits the overall design well. That being said, I personally would not be averse to something like expanding a NUMA-style API to cover the cluster, as I think this continues to fit the existing model -- a NUMA-style API breaks down an atomic resource, so for a VMWare cluster that would allow tuning to individual hypervisors, while for an individual hypervisor that would allow tuning to individual cores, etc. Best Regards, Solly Ross - Original Message - From: Matthew Booth mbo...@redhat.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Monday, April 7, 2014 10:47:35 AM Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute On 07/04/14 06:20, Jay Pipes wrote: On Sun, 2014-04-06 at 06:59 +, Nandavar, Divakar Padiyar wrote: Well, it seems to me that the problem is the above blueprint and the code it introduced. This is an anti-feature IMO, and probably the best solution would be to remove the above code and go back to having a single nova-compute managing a single vCenter cluster, not multiple ones. Problem is not introduced by managing multiple clusters from single nova-compute proxy node. I strongly disagree. Internally this proxy driver is still presenting the compute-node for each of the cluster its managing. In what way? What we need to think about is applicability of the live migration use case when a cluster is modelled as a compute. Since the cluster is modelled as a compute, it is assumed that a typical use case of live-move is taken care by the underlying cluster itself. With this there are other use cases which are no-op today like host maintenance mode, live move, setting instance affinity etc., In order to resolve this I was thinking of A way to expose operations on individual ESX Hosts like Putting host in maintenance mode, live move, instance affinity etc., by introducing Parent - Child compute node concept. Scheduling can be restricted to Parent compute node and Child compute node can be used for providing more drill down on compute and also enable additional compute operations.Any thoughts on this? The fundamental problem is that hacks were put in place in order to make Nova defer control to vCenter, when the design of Nova and vCenter are not compatible, and we're paying the price for that right now. All of the operations you describe above -- putting a host in maintenance mode, live-migration of an instance, ensuring a new instance is launched near or not-near another instance -- depend on a fundamental design feature in Nova: that a nova-compute worker fully controls and manages a host that provides a place to put server instances. We have internal driver interfaces for the *hypervisor*, not for the *manager of hypervisors*, because, you know, that's what Nova does. I'm going to take you to task here for use of the word 'fundamental'. What does Nova do? Apparently: 'OpenStack Nova provides a cloud computing fabric controller, supporting a wide variety of virtualization technologies, including KVM, Xen, LXC, VMware, and more. In addition to its native API, it includes compatibility with the commonly encountered Amazon EC2 and S3 APIs.' There's nothing in there about the ratio of Nova instances to hypervisors: that's an implementation detail. Now this change may or may not sit well with design decisions which have been made in the past, but the concept of managing multiple clusters from a single Nova instance is certainly not fundamentally wrong. It may not be pragmatic; it may require further changes to Nova which were not made, but there is nothing about it which is fundamentally at odds with the stated goals of the project. Why did I bother with that? I think it's in danger of being lost. Nova has been around for a while now and it has a lot of code and a lot of developers behind it. We need to remember, though, that's it's all for nothing if nobody wants to use it. VMware is different, but not wrong. Let's stay fresh. The problem with all of the vCenter stuff is that it is trying to say to Nova don't
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Well, it seems to me that the problem is the above blueprint and the code it introduced. This is an anti-feature IMO, and probably the best solution would be to remove the above code and go back to having a single nova-compute managing a single vCenter cluster, not multiple ones. Problem is not introduced by managing multiple clusters from single nova-compute proxy node. Internally this proxy driver is still presenting the compute-node for each of the cluster its managing.What we need to think about is applicability of the live migration use case when a cluster is modelled as a compute. Since the cluster is modelled as a compute, it is assumed that a typical use case of live-move is taken care by the underlying cluster itself. With this there are other use cases which are no-op today like host maintenance mode, live move, setting instance affinity etc., In order to resolve this I was thinking of A way to expose operations on individual ESX Hosts like Putting host in maintenance mode, live move, instance affinity etc., by introducing Parent - Child compute node concept. Scheduling can be restricted to Parent compute node and Child compute node can be used for providing more drill down on compute and also enable additional compute operations.Any thoughts on this? Thanks, Divakar -Original Message- From: Jay Pipes [mailto:jaypi...@gmail.com] Sent: Sunday, April 06, 2014 2:02 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute Importance: High On Fri, 2014-04-04 at 13:30 +0800, Jay Lau wrote: 2014-04-04 12:46 GMT+08:00 Jay Pipes jaypi...@gmail.com: On Fri, 2014-04-04 at 11:08 +0800, Jay Lau wrote: Thanks Jay and Chris for the comments! @Jay Pipes, I think that we still need to enable one nova compute live migration as one nova compute can manage multiple clusters and VMs can be migrated between those clusters managed by one nova compute. Why, though? That is what I am asking... seems to me like this is an anti-feature. What benefit does the user get from moving an instance from one VCenter cluster to another VCenter cluster if the two clusters are on the same physical machine? @Jay Pipes, for VMWare, one physical machine (ESX server) can only belong to one VCenter cluster, so we may have following scenarios. DC | |---Cluster1 | | | |---host1 | |---Cluser2 | |---host2 Then when using VCDriver, I can use one nova compute manage both Cluster1 and Cluster2, this caused me cannot migrate VM from host2 to host1 ;-( The bp was introduced by https://blueprints.launchpad.net/nova/+spec/multiple-clusters-managed- by-one-service Well, it seems to me that the problem is the above blueprint and the code it introduced. This is an anti-feature IMO, and probably the best solution would be to remove the above code and go back to having a single nova-compute managing a single vCenter cluster, not multiple ones. -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Hi Divakar, Can I say that the bare metal provisioning is now using kind of Parent - Child compute mode? I was also thinking that we can use host:node to identify a kind of Parent-Child or Hierarchy Compute. So can you please show some difference for your Parent - Child Compute Node and bare metal provisioning? Thanks! 2014-04-06 14:59 GMT+08:00 Nandavar, Divakar Padiyar divakar.padiyar-nanda...@hp.com: Well, it seems to me that the problem is the above blueprint and the code it introduced. This is an anti-feature IMO, and probably the best solution would be to remove the above code and go back to having a single nova-compute managing a single vCenter cluster, not multiple ones. Problem is not introduced by managing multiple clusters from single nova-compute proxy node. Internally this proxy driver is still presenting the compute-node for each of the cluster its managing.What we need to think about is applicability of the live migration use case when a cluster is modelled as a compute. Since the cluster is modelled as a compute, it is assumed that a typical use case of live-move is taken care by the underlying cluster itself. With this there are other use cases which are no-op today like host maintenance mode, live move, setting instance affinity etc., In order to resolve this I was thinking of A way to expose operations on individual ESX Hosts like Putting host in maintenance mode, live move, instance affinity etc., by introducing Parent - Child compute node concept. Scheduling can be restricted to Parent compute node and Child compute node can be used for providing more drill down on compute and also enable additional compute operations.Any thoughts on this? Thanks, Divakar -Original Message- From: Jay Pipes [mailto:jaypi...@gmail.com] Sent: Sunday, April 06, 2014 2:02 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute Importance: High On Fri, 2014-04-04 at 13:30 +0800, Jay Lau wrote: 2014-04-04 12:46 GMT+08:00 Jay Pipes jaypi...@gmail.com: On Fri, 2014-04-04 at 11:08 +0800, Jay Lau wrote: Thanks Jay and Chris for the comments! @Jay Pipes, I think that we still need to enable one nova compute live migration as one nova compute can manage multiple clusters and VMs can be migrated between those clusters managed by one nova compute. Why, though? That is what I am asking... seems to me like this is an anti-feature. What benefit does the user get from moving an instance from one VCenter cluster to another VCenter cluster if the two clusters are on the same physical machine? @Jay Pipes, for VMWare, one physical machine (ESX server) can only belong to one VCenter cluster, so we may have following scenarios. DC | |---Cluster1 | | | |---host1 | |---Cluser2 | |---host2 Then when using VCDriver, I can use one nova compute manage both Cluster1 and Cluster2, this caused me cannot migrate VM from host2 to host1 ;-( The bp was introduced by https://blueprints.launchpad.net/nova/+spec/multiple-clusters-managed- by-one-service Well, it seems to me that the problem is the above blueprint and the code it introduced. This is an anti-feature IMO, and probably the best solution would be to remove the above code and go back to having a single nova-compute managing a single vCenter cluster, not multiple ones. -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Thanks, Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
On Sun, 2014-04-06 at 06:59 +, Nandavar, Divakar Padiyar wrote: Well, it seems to me that the problem is the above blueprint and the code it introduced. This is an anti-feature IMO, and probably the best solution would be to remove the above code and go back to having a single nova-compute managing a single vCenter cluster, not multiple ones. Problem is not introduced by managing multiple clusters from single nova-compute proxy node. I strongly disagree. Internally this proxy driver is still presenting the compute-node for each of the cluster its managing. In what way? What we need to think about is applicability of the live migration use case when a cluster is modelled as a compute. Since the cluster is modelled as a compute, it is assumed that a typical use case of live-move is taken care by the underlying cluster itself. With this there are other use cases which are no-op today like host maintenance mode, live move, setting instance affinity etc., In order to resolve this I was thinking of A way to expose operations on individual ESX Hosts like Putting host in maintenance mode, live move, instance affinity etc., by introducing Parent - Child compute node concept. Scheduling can be restricted to Parent compute node and Child compute node can be used for providing more drill down on compute and also enable additional compute operations.Any thoughts on this? The fundamental problem is that hacks were put in place in order to make Nova defer control to vCenter, when the design of Nova and vCenter are not compatible, and we're paying the price for that right now. All of the operations you describe above -- putting a host in maintenance mode, live-migration of an instance, ensuring a new instance is launched near or not-near another instance -- depend on a fundamental design feature in Nova: that a nova-compute worker fully controls and manages a host that provides a place to put server instances. We have internal driver interfaces for the *hypervisor*, not for the *manager of hypervisors*, because, you know, that's what Nova does. The problem with all of the vCenter stuff is that it is trying to say to Nova don't worry, I got this but unfortunately, Nova wants and needs to manage these things, not surrender control to a different system that handles orchestration and scheduling in its own unique way. If a shop really wants to use vCenter for scheduling and orchestration of server instances, what exactly is the point of using OpenStack Nova to begin with? What exactly is the point of trying to use OpenStack Nova for scheduling and host operations when you've already shelled out US $6,000 for vCenter Server and a boatload more money for ESX licensing? Sorry, I'm just at a loss why Nova was changed to accomodate vCenter cluster and management concepts to begin with. I just don't understand the use case here. Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Thanks Jay Pipes. If go back to having a single nova-compute managing a single vCenter cluster, then there might be problems in a large sacle vCenter cluster. There are still problems that we can not handle: 1) The VCDriver can also manage multiple resource pools with a single nova compute, the resource pool is another concept, we can create multiple resource pools in one vCenter cluster or create multiple resource pools in one ESX host. In a large scale cluster, there can be thousands of resource pools, it would make the admin crazy for the configuration. ;-) 2) How to manage ESX host which not belong to any cluster or resource pools? Such as following case: DC | |--- ESX host1 | |--- ESX host2 3) There is another bp https://blueprints.launchpad.net/nova/+spec/vmware-auto-inventory filed by Shawn, this bp want to report all resources including clusters, resource pools, esx hosts, this bp can be treated as the base for VCDriver, as if the VCDriver can get all resources, then it would be very easy to do what we want. Thanks! 2014-04-06 4:32 GMT+08:00 Jay Pipes jaypi...@gmail.com: On Fri, 2014-04-04 at 13:30 +0800, Jay Lau wrote: 2014-04-04 12:46 GMT+08:00 Jay Pipes jaypi...@gmail.com: On Fri, 2014-04-04 at 11:08 +0800, Jay Lau wrote: Thanks Jay and Chris for the comments! @Jay Pipes, I think that we still need to enable one nova compute live migration as one nova compute can manage multiple clusters and VMs can be migrated between those clusters managed by one nova compute. Why, though? That is what I am asking... seems to me like this is an anti-feature. What benefit does the user get from moving an instance from one VCenter cluster to another VCenter cluster if the two clusters are on the same physical machine? @Jay Pipes, for VMWare, one physical machine (ESX server) can only belong to one VCenter cluster, so we may have following scenarios. DC | |---Cluster1 | | | |---host1 | |---Cluser2 | |---host2 Then when using VCDriver, I can use one nova compute manage both Cluster1 and Cluster2, this caused me cannot migrate VM from host2 to host1 ;-( The bp was introduced by https://blueprints.launchpad.net/nova/+spec/multiple-clusters-managed-by-one-service Well, it seems to me that the problem is the above blueprint and the code it introduced. This is an anti-feature IMO, and probably the best solution would be to remove the above code and go back to having a single nova-compute managing a single vCenter cluster, not multiple ones. -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Thanks, Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
On Mon, 2014-03-31 at 17:11 +0800, Jay Lau wrote: Hi, Currently with VMWare VCDriver, one nova compute can manage multiple clusters/RPs, this caused cluster admin cannot do live migration between clusters/PRs if those clusters/PRs managed by one nova compute as the current live migration logic request at least two nova computes. A bug [1] was also filed to trace VMWare live migration issue. I'm now trying the following solution to see if it is acceptable for a fix, the fix wants enable live migration with one nova compute: 1) When live migration check if host are same, check both host and node for the VM instance. 2) When nova scheduler select destination for live migration, the live migration task should put (host, node) to attempted hosts. 3) Nova scheduler needs to be enhanced to support ignored_nodes. 4) nova compute need to be enhanced to check host and node when doing live migration. What precisely is the point of live migrating an instance to the exact same host as it is already on? The failure domain is the host, so moving the instance from one cluster to another, but on the same host is kind of a silly use case IMO. But... if this really is something that is considered useful, then it seems to me that it would be more useful to simply expand the definition of the compute node object within Nova to be generic enough that a compute node could be a VCenter cluster/PR. That way there would be no need to hack the scheduler to account for more than the host (compute node internally in Nova)? In the same way, a cell could be a compute node as well, and we wouldn't need separate hacks in the scheduler and elsewhere that treated cells differently than regular compute nodes. Best, -jay I also uploaded a WIP patch [2] for you to review the idea of the fix and hope can get some comments from you. [1] https://bugs.launchpad.net/nova/+bug/1192192 [2] https://review.openstack.org/#/c/84085 -- Thanks, Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
On 04/03/2014 05:48 PM, Jay Pipes wrote: On Mon, 2014-03-31 at 17:11 +0800, Jay Lau wrote: Hi, Currently with VMWare VCDriver, one nova compute can manage multiple clusters/RPs, this caused cluster admin cannot do live migration between clusters/PRs if those clusters/PRs managed by one nova compute as the current live migration logic request at least two nova computes. A bug [1] was also filed to trace VMWare live migration issue. I'm now trying the following solution to see if it is acceptable for a fix, the fix wants enable live migration with one nova compute: 1) When live migration check if host are same, check both host and node for the VM instance. 2) When nova scheduler select destination for live migration, the live migration task should put (host, node) to attempted hosts. 3) Nova scheduler needs to be enhanced to support ignored_nodes. 4) nova compute need to be enhanced to check host and node when doing live migration. What precisely is the point of live migrating an instance to the exact same host as it is already on? The failure domain is the host, so moving the instance from one cluster to another, but on the same host is kind of a silly use case IMO. Here is where precise definitions of compute node, OS-EXT-SRV-ATTR:host, and OS-EXT-SRV-ATTR:hypervisor_hostname, and host as understood by novaclient would be nice. Currently the nova live-migration command takes a host argument. It's not clear which of the above this corresponds to. My understanding is that one nova-compute process can manage multiple VMWare physical hosts. So it could make sense to support live migration between separate VMWare hosts even if they're managed by a single nova-compute process. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Thanks Jay and Chris for the comments! @Jay Pipes, I think that we still need to enable one nova compute live migration as one nova compute can manage multiple clusters and VMs can be migrated between those clusters managed by one nova compute. For cell, IMHO, each cell can be treated as a small cloud but not a compute, each cell cloud should be able to handle VM operations in the small cloud itself. Please correct me if I am wrong. @Chris, OS-EXT-SRV-ATTR:host is the host where nova compute is running and OS-EXT-SRV-ATTR:hypervisor_hostname is the hypervisor host where the VM is running. Live migration is now using host for live migration. What I want to do is enable migration with one host and the host managing multiple hyperviosrs. I'm planning to draft a bp for review which depend on https://blueprints.launchpad.net/nova/+spec/vmware-auto-inventory Thanks! 2014-04-04 8:03 GMT+08:00 Chris Friesen chris.frie...@windriver.com: On 04/03/2014 05:48 PM, Jay Pipes wrote: On Mon, 2014-03-31 at 17:11 +0800, Jay Lau wrote: Hi, Currently with VMWare VCDriver, one nova compute can manage multiple clusters/RPs, this caused cluster admin cannot do live migration between clusters/PRs if those clusters/PRs managed by one nova compute as the current live migration logic request at least two nova computes. A bug [1] was also filed to trace VMWare live migration issue. I'm now trying the following solution to see if it is acceptable for a fix, the fix wants enable live migration with one nova compute: 1) When live migration check if host are same, check both host and node for the VM instance. 2) When nova scheduler select destination for live migration, the live migration task should put (host, node) to attempted hosts. 3) Nova scheduler needs to be enhanced to support ignored_nodes. 4) nova compute need to be enhanced to check host and node when doing live migration. What precisely is the point of live migrating an instance to the exact same host as it is already on? The failure domain is the host, so moving the instance from one cluster to another, but on the same host is kind of a silly use case IMO. Here is where precise definitions of compute node, OS-EXT-SRV-ATTR:host, and OS-EXT-SRV-ATTR:hypervisor_hostname, and host as understood by novaclient would be nice. Currently the nova live-migration command takes a host argument. It's not clear which of the above this corresponds to. My understanding is that one nova-compute process can manage multiple VMWare physical hosts. So it could make sense to support live migration between separate VMWare hosts even if they're managed by a single nova-compute process. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Thanks, Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
On Fri, 2014-04-04 at 11:08 +0800, Jay Lau wrote: Thanks Jay and Chris for the comments! @Jay Pipes, I think that we still need to enable one nova compute live migration as one nova compute can manage multiple clusters and VMs can be migrated between those clusters managed by one nova compute. Why, though? That is what I am asking... seems to me like this is an anti-feature. What benefit does the user get from moving an instance from one VCenter cluster to another VCenter cluster if the two clusters are on the same physical machine? Secondly, why is it that a single nova-compute manages multiple VCenter clusters? This seems like a hack to me... perhaps someone who wrote the code for this or knows the decision behind it could chime in here? For cell, IMHO, each cell can be treated as a small cloud but not a compute, each cell cloud should be able to handle VM operations in the small cloud itself. Please correct me if I am wrong. Yes, I agree with you that a cell is not a compute. Not sure if I said otherwise in my previous response. Sorry if it was confusing! :) Best, -jay @Chris, OS-EXT-SRV-ATTR:host is the host where nova compute is running and OS-EXT-SRV-ATTR:hypervisor_hostname is the hypervisor host where the VM is running. Live migration is now using host for live migration. What I want to do is enable migration with one host and the host managing multiple hyperviosrs. I'm planning to draft a bp for review which depend on https://blueprints.launchpad.net/nova/+spec/vmware-auto-inventory Thanks! 2014-04-04 8:03 GMT+08:00 Chris Friesen chris.frie...@windriver.com: On 04/03/2014 05:48 PM, Jay Pipes wrote: On Mon, 2014-03-31 at 17:11 +0800, Jay Lau wrote: Hi, Currently with VMWare VCDriver, one nova compute can manage multiple clusters/RPs, this caused cluster admin cannot do live migration between clusters/PRs if those clusters/PRs managed by one nova compute as the current live migration logic request at least two nova computes. A bug [1] was also filed to trace VMWare live migration issue. I'm now trying the following solution to see if it is acceptable for a fix, the fix wants enable live migration with one nova compute: 1) When live migration check if host are same, check both host and node for the VM instance. 2) When nova scheduler select destination for live migration, the live migration task should put (host, node) to attempted hosts. 3) Nova scheduler needs to be enhanced to support ignored_nodes. 4) nova compute need to be enhanced to check host and node when doing live migration. What precisely is the point of live migrating an instance to the exact same host as it is already on? The failure domain is the host, so moving the instance from one cluster to another, but on the same host is kind of a silly use case IMO. Here is where precise definitions of compute node, OS-EXT-SRV-ATTR:host, and OS-EXT-SRV-ATTR:hypervisor_hostname, and host as understood by novaclient would be nice. Currently the nova live-migration command takes a host argument. It's not clear which of the above this corresponds to. My understanding is that one nova-compute process can manage multiple VMWare physical hosts. So it could make sense to support live migration between separate VMWare hosts even if they're managed by a single nova-compute process. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Thanks, Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
2014-04-04 12:46 GMT+08:00 Jay Pipes jaypi...@gmail.com: On Fri, 2014-04-04 at 11:08 +0800, Jay Lau wrote: Thanks Jay and Chris for the comments! @Jay Pipes, I think that we still need to enable one nova compute live migration as one nova compute can manage multiple clusters and VMs can be migrated between those clusters managed by one nova compute. Why, though? That is what I am asking... seems to me like this is an anti-feature. What benefit does the user get from moving an instance from one VCenter cluster to another VCenter cluster if the two clusters are on the same physical machine? @Jay Pipes, for VMWare, one physical machine (ESX server) can only belong to one VCenter cluster, so we may have following scenarios. DC | |---Cluster1 | | | |---host1 | |---Cluser2 | |---host2 Then when using VCDriver, I can use one nova compute manage both Cluster1 and Cluster2, this caused me cannot migrate VM from host2 to host1 ;-( The bp was introduced by https://blueprints.launchpad.net/nova/+spec/multiple-clusters-managed-by-one-service Secondly, why is it that a single nova-compute manages multiple VCenter clusters? This seems like a hack to me... perhaps someone who wrote the code for this or knows the decision behind it could chime in here? For cell, IMHO, each cell can be treated as a small cloud but not a compute, each cell cloud should be able to handle VM operations in the small cloud itself. Please correct me if I am wrong. Yes, I agree with you that a cell is not a compute. Not sure if I said otherwise in my previous response. Sorry if it was confusing! :) Best, -jay @Chris, OS-EXT-SRV-ATTR:host is the host where nova compute is running and OS-EXT-SRV-ATTR:hypervisor_hostname is the hypervisor host where the VM is running. Live migration is now using host for live migration. What I want to do is enable migration with one host and the host managing multiple hyperviosrs. I'm planning to draft a bp for review which depend on https://blueprints.launchpad.net/nova/+spec/vmware-auto-inventory Thanks! 2014-04-04 8:03 GMT+08:00 Chris Friesen chris.frie...@windriver.com: On 04/03/2014 05:48 PM, Jay Pipes wrote: On Mon, 2014-03-31 at 17:11 +0800, Jay Lau wrote: Hi, Currently with VMWare VCDriver, one nova compute can manage multiple clusters/RPs, this caused cluster admin cannot do live migration between clusters/PRs if those clusters/PRs managed by one nova compute as the current live migration logic request at least two nova computes. A bug [1] was also filed to trace VMWare live migration issue. I'm now trying the following solution to see if it is acceptable for a fix, the fix wants enable live migration with one nova compute: 1) When live migration check if host are same, check both host and node for the VM instance. 2) When nova scheduler select destination for live migration, the live migration task should put (host, node) to attempted hosts. 3) Nova scheduler needs to be enhanced to support ignored_nodes. 4) nova compute need to be enhanced to check host and node when doing live migration. What precisely is the point of live migrating an instance to the exact same host as it is already on? The failure domain is the host, so moving the instance from one cluster to another, but on the same host is kind of a silly use case IMO. Here is where precise definitions of compute node, OS-EXT-SRV-ATTR:host, and OS-EXT-SRV-ATTR:hypervisor_hostname, and host as understood by novaclient would be nice. Currently the nova live-migration command takes a host argument. It's not clear which of the above this corresponds to. My understanding is that one nova-compute process can manage multiple VMWare physical hosts. So it could make sense to support live migration between separate VMWare hosts even if they're managed by a single nova-compute process. Chris ___
[openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Hi, Currently with VMWare VCDriver, one nova compute can manage multiple clusters/RPs, this caused cluster admin cannot do live migration between clusters/PRs if those clusters/PRs managed by one nova compute as the current live migration logic request at least two nova computes. A bug [1] was also filed to trace VMWare live migration issue. I'm now trying the following solution to see if it is acceptable for a fix, the fix wants enable live migration with one nova compute: 1) When live migration check if host are same, check both host and node for the VM instance. 2) When nova scheduler select destination for live migration, the live migration task should put (host, node) to attempted hosts. 3) Nova scheduler needs to be enhanced to support ignored_nodes. 4) nova compute need to be enhanced to check host and node when doing live migration. I also uploaded a WIP patch [2] for you to review the idea of the fix and hope can get some comments from you. [1] https://bugs.launchpad.net/nova/+bug/1192192 [2] https://review.openstack.org/#/c/84085 -- Thanks, Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
On 31 March 2014 10:11, Jay Lau jay.lau@gmail.com wrote: Hi, Currently with VMWare VCDriver, one nova compute can manage multiple clusters/RPs, this caused cluster admin cannot do live migration between clusters/PRs if those clusters/PRs managed by one nova compute as the current live migration logic request at least two nova computes. A bug [1] was also filed to trace VMWare live migration issue. I'm now trying the following solution to see if it is acceptable for a fix, the fix wants enable live migration with one nova compute: 1) When live migration check if host are same, check both host and node for the VM instance. 2) When nova scheduler select destination for live migration, the live migration task should put (host, node) to attempted hosts. 3) Nova scheduler needs to be enhanced to support ignored_nodes. 4) nova compute need to be enhanced to check host and node when doing live migration. I also uploaded a WIP patch [2] for you to review the idea of the fix and hope can get some comments from you. [1] https://bugs.launchpad.net/nova/+bug/1192192 [2] https://review.openstack.org/#/c/84085 Long term, finding a way to unify how cells and the VMware driver manages multiple hosts, seems like the best way forward. It would be a shame for this API to be different between cells and VMware, although right now, that might not work too well :( A better short term fix, might be to allow you to specify the same host as the instance, and the scheduling of the node could be delegated to the VMware driver, which might just delegate that to vCenter. I assume we still need some way to specify the node, and I can't immediately think of a good way forward. I feel this should really be treated as a blueprint, and go through the new blueprint review process. That should help decide the right approach to take. John ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Building on what John said, I'm a bit wary of introducing semantics into the Conductor's live migration code that are VMWare-specific. The conductor's live-migration code is supposed to be driver-agnostic. IMHO, it would be much better if we could handle this at a level where the code was already VMWare-specific. Best Regards, Solly Ross - Original Message - From: Jay Lau jay.lau@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Monday, March 31, 2014 10:36:17 AM Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute Thanks John. Yes, I also think that this should be a bp as it is going to make some changes to enable live migration with only one nova compute, will file a blueprint later. For your proposal specify the same host as the instance, this can resolve the issue of live migration with target host, but what about the case of live migration without target host? If we still allow specify the same host as the instance, the the live migration will goes to dead loop. So it seems we definitely need to find a way to specify the node for live migration, hope someone else can show some light here. Of course, I will file bp and go through the new bp review process for this feature. Thanks! 2014-03-31 21:02 GMT+08:00 John Garbutt j...@johngarbutt.com : On 31 March 2014 10:11, Jay Lau jay.lau@gmail.com wrote: Hi, Currently with VMWare VCDriver, one nova compute can manage multiple clusters/RPs, this caused cluster admin cannot do live migration between clusters/PRs if those clusters/PRs managed by one nova compute as the current live migration logic request at least two nova computes. A bug [1] was also filed to trace VMWare live migration issue. I'm now trying the following solution to see if it is acceptable for a fix, the fix wants enable live migration with one nova compute: 1) When live migration check if host are same, check both host and node for the VM instance. 2) When nova scheduler select destination for live migration, the live migration task should put (host, node) to attempted hosts. 3) Nova scheduler needs to be enhanced to support ignored_nodes. 4) nova compute need to be enhanced to check host and node when doing live migration. I also uploaded a WIP patch [2] for you to review the idea of the fix and hope can get some comments from you. [1] https://bugs.launchpad.net/nova/+bug/1192192 [2] https://review.openstack.org/#/c/84085 Long term, finding a way to unify how cells and the VMware driver manages multiple hosts, seems like the best way forward. It would be a shame for this API to be different between cells and VMware, although right now, that might not work too well :( A better short term fix, might be to allow you to specify the same host as the instance, and the scheduling of the node could be delegated to the VMware driver, which might just delegate that to vCenter. I assume we still need some way to specify the node, and I can't immediately think of a good way forward. I feel this should really be treated as a blueprint, and go through the new blueprint review process. That should help decide the right approach to take. John ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Thanks, Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
On 31 Mar 2014, at 18:13, Solly Ross sr...@redhat.com wrote: Building on what John said, I'm a bit wary of introducing semantics into the Conductor's live migration code that are VMWare-specific. The conductor's live-migration code is supposed to be driver-agnostic. IMHO, it would be much better if we could handle this at a level where the code was already VMWare-specific. In terms of driver specific features, we’re evaluating cluster support for Hyper-V in the next cycle which would encounter the same issue for live migration. Hyper-V does not require clustering for supporting live migration (it’s already available since Grizzly), but various users are requesting Windows clustering support for supporting specific scenarios, which requires a separate Nova Hyper-V failover clustering driver with resemblances to the VCenter driver in terms of cells / hosts management. Note: this is not related to Microsoft System Center. Evaluating such a feature solely in base of blueprints barely under drafting for other drivers and never discussed for approval isn’t obviously requested, but it might be useful to consider the possibility that VMWare’s might not be the only Nova driver with this requirement in the relatively short term future. Thanks, Alessandro Best Regards, Solly Ross - Original Message - From: Jay Lau jay.lau@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Monday, March 31, 2014 10:36:17 AM Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute Thanks John. Yes, I also think that this should be a bp as it is going to make some changes to enable live migration with only one nova compute, will file a blueprint later. For your proposal specify the same host as the instance, this can resolve the issue of live migration with target host, but what about the case of live migration without target host? If we still allow specify the same host as the instance, the the live migration will goes to dead loop. So it seems we definitely need to find a way to specify the node for live migration, hope someone else can show some light here. Of course, I will file bp and go through the new bp review process for this feature. Thanks! 2014-03-31 21:02 GMT+08:00 John Garbutt j...@johngarbutt.com : On 31 March 2014 10:11, Jay Lau jay.lau@gmail.com wrote: Hi, Currently with VMWare VCDriver, one nova compute can manage multiple clusters/RPs, this caused cluster admin cannot do live migration between clusters/PRs if those clusters/PRs managed by one nova compute as the current live migration logic request at least two nova computes. A bug [1] was also filed to trace VMWare live migration issue. I'm now trying the following solution to see if it is acceptable for a fix, the fix wants enable live migration with one nova compute: 1) When live migration check if host are same, check both host and node for the VM instance. 2) When nova scheduler select destination for live migration, the live migration task should put (host, node) to attempted hosts. 3) Nova scheduler needs to be enhanced to support ignored_nodes. 4) nova compute need to be enhanced to check host and node when doing live migration. I also uploaded a WIP patch [2] for you to review the idea of the fix and hope can get some comments from you. [1] https://bugs.launchpad.net/nova/+bug/1192192 [2] https://review.openstack.org/#/c/84085 Long term, finding a way to unify how cells and the VMware driver manages multiple hosts, seems like the best way forward. It would be a shame for this API to be different between cells and VMware, although right now, that might not work too well :( A better short term fix, might be to allow you to specify the same host as the instance, and the scheduling of the node could be delegated to the VMware driver, which might just delegate that to vCenter. I assume we still need some way to specify the node, and I can't immediately think of a good way forward. I feel this should really be treated as a blueprint, and go through the new blueprint review process. That should help decide the right approach to take. John ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Thanks, Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Thanks Solly and Alessandro! @Solly, Yes, I also want to make the code change not VMWare-specific, but perhaps we can consider of what @Alessandro said, we are going to have hyper-V cluster support in next cycle, and maybe Power HMC in the future. All of them can be managed in cluster level and one cluster can have multiple hypervisors. So I think that it might be a time to enhance live migration to handle not only the case of single hypervisor but also multiple hypervisors managed by one cluster. Hope we can also get some comments from VMWare guys. Thanks. 2014-04-01 6:57 GMT+08:00 Alessandro Pilotti apilo...@cloudbasesolutions.com: On 31 Mar 2014, at 18:13, Solly Ross sr...@redhat.com wrote: Building on what John said, I'm a bit wary of introducing semantics into the Conductor's live migration code that are VMWare-specific. The conductor's live-migration code is supposed to be driver-agnostic. IMHO, it would be much better if we could handle this at a level where the code was already VMWare-specific. In terms of driver specific features, we're evaluating cluster support for Hyper-V in the next cycle which would encounter the same issue for live migration. Hyper-V does not require clustering for supporting live migration (it's already available since Grizzly), but various users are requesting Windows clustering support for supporting specific scenarios, which requires a separate Nova Hyper-V failover clustering driver with resemblances to the VCenter driver in terms of cells / hosts management. Note: this is not related to Microsoft System Center. Evaluating such a feature solely in base of blueprints barely under drafting for other drivers and never discussed for approval isn't obviously requested, but it might be useful to consider the possibility that VMWare's might not be the only Nova driver with this requirement in the relatively short term future. Thanks, Alessandro Best Regards, Solly Ross - Original Message - From: Jay Lau jay.lau@gmail.com To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Sent: Monday, March 31, 2014 10:36:17 AM Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute Thanks John. Yes, I also think that this should be a bp as it is going to make some changes to enable live migration with only one nova compute, will file a blueprint later. For your proposal specify the same host as the instance, this can resolve the issue of live migration with target host, but what about the case of live migration without target host? If we still allow specify the same host as the instance, the the live migration will goes to dead loop. So it seems we definitely need to find a way to specify the node for live migration, hope someone else can show some light here. Of course, I will file bp and go through the new bp review process for this feature. Thanks! 2014-03-31 21:02 GMT+08:00 John Garbutt j...@johngarbutt.com : On 31 March 2014 10:11, Jay Lau jay.lau@gmail.com wrote: Hi, Currently with VMWare VCDriver, one nova compute can manage multiple clusters/RPs, this caused cluster admin cannot do live migration between clusters/PRs if those clusters/PRs managed by one nova compute as the current live migration logic request at least two nova computes. A bug [1] was also filed to trace VMWare live migration issue. I'm now trying the following solution to see if it is acceptable for a fix, the fix wants enable live migration with one nova compute: 1) When live migration check if host are same, check both host and node for the VM instance. 2) When nova scheduler select destination for live migration, the live migration task should put (host, node) to attempted hosts. 3) Nova scheduler needs to be enhanced to support ignored_nodes. 4) nova compute need to be enhanced to check host and node when doing live migration. I also uploaded a WIP patch [2] for you to review the idea of the fix and hope can get some comments from you. [1] https://bugs.launchpad.net/nova/+bug/1192192 [2] https://review.openstack.org/#/c/84085 Long term, finding a way to unify how cells and the VMware driver manages multiple hosts, seems like the best way forward. It would be a shame for this API to be different between cells and VMware, although right now, that might not work too well :( A better short term fix, might be to allow you to specify the same host as the instance, and the scheduling of the node could be delegated to the VMware driver, which might just delegate that to vCenter. I assume we still need some way to specify the node, and I can't immediately think of a good way forward. I feel this should really be treated as a blueprint, and go through the new blueprint review process. That should help decide the right
